G33k, Programmes

Bind all your gprc endpoints with a simple and magic coRouter

grpc-services are quite simple, focused and performance oriented services.
Instead of calling them with a format that needs complex marshalling / unmarshalling, and making use of a standard http 1.1 connection, grpc has a dedicated (therefore faster) generated code for each type.
This protocol also indexes data fields by number, which permits to shrink the input by many folds
(if you know the order of the keys in the structure, then you don’t need to indicate them in your payload)

grpc work efficiently with naive Netty servers (all you have to do is to call .bindService()).
But the integration is not so smooth when you currently have reactive spring components to integrate with (authentication, libraries, aspects).
When I have discovered coRouters, I was able to understand that one of its purpose was to dynamically declare endpoints based upon spring beans.

Then if you mix these two concepts together (grpc and coRouters), you can build autowired grpc services on spring.
How do you do that ?
Simply by copy / pasting this snippet in your codebase : https://gist.github.com/libetl/a655de480ed4d123e0c10fe557ea4271

G33k, Programmes

Rendering react in Java

Before telling me that I am crazy, think about it again.

There are plenty of ways to use server-side rendering today, and the practice of sending displayable data through the network has proven to be efficient :
– rendering data logged in the same tool as for backend services
– less client business logic, less coupling between frontends
– multi-channel broadcast on your clients devices (mobile, desktop, television)
Besides, some frameworks are written on top of virtual machines to do the job. Which one specifically ? Node.js

Java has a poor support today, and many companies are considering rewritting their stack just for the dream of going multi-channel.
Some others cannot, … well it is quite expensive.

While the idea of going rogue and starting to rewrite the app seems like the only way to go, there is an intermediairy solution for companies that mainly has code written in Java :
Make your Java api return react elements.

Howto ?
React components are just initiated (or can always be summarized) by a composition of two elements :
– a type
– properties
If there are some advanced features (forward references, lazy loading, derived states, react portals), these can always be wrapped in simple components in the javascript / typescript client.
Example of json that you can write in your http response :

{"type":"table","props":{"id":"test","className":"my-table","children":{"type":"tablerow","props":{"vAlign":"center","children":{"type":"tablecell","props":{"children":{"type":"MyText","props":{"big":true,"bold":true,"danger":false,"children":"Hello"}}}}}}}}

Then this is how an user interface can be sent through a rest endpoint :

@Component
class RenderText {

    fun page() = tag { table }.new {
        id = "test"
        className = "my-table"
    }.setChildren {
        tag { tr }.new { vAlign = "center" }.setChildren {
            new { td }.setChildren {
                MyLib.MyText::class.new {
                    big = true
                    bold = true
                    danger = false
                    children = "Hello"
                }
            }
        }
    }.toString()
}

Of course, just like in javascript, this is programmatic, you can embed whatever logic you want.
To be able to build this above, you need to declare your react components interface in the JVM.

package com.mycompany.service.react

object MyLib {

    interface MyTextProps: React.Props<MyTextProps> {
        var bold: Boolean?
        var danger : Boolean?
        var big : Boolean?
    }

    interface MyText : React.FC<MyTextProps>
}

You may now ask how can we have the react typings without leaving the JVM. In my case I have used (https://github.com/Kotlin/dukat) to convert the react typescript types back into Kotlin (and then back to the JVM)
You can just copy / paste react 16.13.1 types in kotlin : https://gist.github.com/libetl/7f4784eeaa5320b14b33567c0544c52a#file-react-kt

Now that you have your components and the react library, there is one missing gap : how do I instantiate interfaces ?
I have added just a small set of helpers method to achieve this goal, by creating proxy instances.
These proxy instances are objects which can extend a complex interface without necessarily implementing all the behaviors.
That lets you bind data to your components without needing you to plug the events and interactions… since these will be lost during the serialization.
Just copy paste those “medium sized” builder methods : https://gist.github.com/libetl/7f4784eeaa5320b14b33567c0544c52a#file-helpers-kt.

Your react code in java should now compile… Enjoy some cheap server-side rendering in the JVM now.
That is it for the backend part. Your API is now able to render a react as json.
Next, you have to convert that json into react elements.
I have a javascript ES5 file to do that : https://gist.github.com/libetl/7f4784eeaa5320b14b33567c0544c52a#file-expand-react-json-js.

Finally, you need the html page to fetch the API that you have created and to transform the result into components :

<!DOCTYPE html>
<html lang=“en”>
  <head>
    <meta charset=“UTF-8” />
    <title>Test server side rendering</title>
  </head>
  <body>
    <div id=“root”></div>
    <script
      crossorigin
    ></script>
    <script
      crossorigin
    ></script>
    <script
      crossorigin
    ></script>
    <!– my lib export an umd variable called “components” –>
    <script
      crossorigin
    ></script>
    <script
      crossorigin=“anonymous”
    ></script>
    <script src=“expand-react-json.js”></script>
    <script type=“text/javascript”>
      window.addEventListener(“DOMContentLoaded”function (event) {
        // api name here is server-driven-ui, but call it your name
        fetch(“server-driven-ui”)
          .then(function (response) {
            return response.json();
          })
          .then(function (json) {
            ReactDOM.render(
              expandReactJson(Reactcomponents“”json),
              document.getElementById(“root”)
            );
          });
      });
    </script>
  </body>
</html>

That is it,
And let’s be patient. Someday we are going to hear about Jetpack Compose for the web.

Until next time, goodbye.

G33k, Programmes

Kotlin Coroutines utilities

If you prefer to combine Kotlin Coroutines with popular solutions like Flowable, Streams or RxJava, where every strategy is readily available, this post is not for you.

Otherwise, if you are interested in writing your own asynchronous strategies, here are two of them (in addition to my post from April 2019 : https://libetl.wordpress.com/2019/04/30/learn-how-to-group-expensive-calls-with-the-coroutines/).

Batching strategy
That strategy helps you stream your “Extract, transform, load” program by starting a parallel execution of a correctly throttled massive process.
In other words, regardless of how many rows are in your dataset, you are able to process them all.

Not too slow, and not too fast to avoid sending a throughput that your downstream actor cannot bear.
The batching strategy is then a kind of streaming facility.

It basically consists in keeping either n workers busy with one row unless if all the rows have been processed.
The strategy is initialized on the first use, and the following datasets can be processed in less “heating” time
Source code of that strategy : https://gist.github.com/libetl/71b826a0db248e6770a2c0b5c0ae6d18#file-batchcoroutinesstrategy-kt

Caching Strategy
Want to keep long time computation results in your program memory after having them processed ? That sounds interesting when your client is requesting some data and you cannot respond in a reasonable amount of time (more than 5 seconds).
Give your client an UUID and tell it to come back later with that UUID.

When a client request an UUID that is not yet computed, you can just reply “oh it is not ready yet”.
If it is done, “here are the results”,
otherwise “sorry, apparently that UUID does not correspond to a task done on this machine”

That strategy consists in a cache object (map of uuid to results), a worker to run async tasks, a “cacheAdder”, a method to poll the status of a task.
Basically, the job starts by sending a message to the worker, which after completion sends the result to the cacheAdder. The cache is configured to automatically make the elements expire 10 minutes after the last read.
Source code of that strategy : https://gist.github.com/libetl/71b826a0db248e6770a2c0b5c0ae6d18#file-cachingcoroutinesstrategy-kt

Can I combine them ?
Absolutely, here are the declarations to have a batching strategy with cache :

private val batch =
    batchingStrategy.batchOf(
        workers = 20,
        coroutineContext = coroutineContext
    ) {
        letsProcess(it)
        // this is where you tell what to do
        // for each element in your dataset
    }

private val batchWithCache =
    cachingStrategy.cache(
        workers = 20,
        coroutineContext = coroutineContext
    ) {
        batch(it).await()
        // "it" represents your data
        // the result of "await" is the
        // global result of the operation.
        // you can add further operations there
    }

G33k, Programmes

How do I mock javax.sql.DataSource

This is the annoying question I have had to ask myself.

Integration tests are pretty slow. Even when they use a dedicated database like H2 or Derby, they are still accessing dark infrastructure software layers to read / edit / write / flush data.
I was wanting tests able to run in no time and still performing data manipulation.

Rather than trying to mock each Data Access layer (and I have got many of them), how can I simply mock responses from databases without having to do bespoke mocks engineering.

Actually DataSource interface (which is a really primitive interface) can be mocked if we simulate the right methods.
DataSource also has a lot of dependencies and axioms that won’t be called by your application … so don’t create mocks for them.
Otherwise you will spend weeks defining your tests strategy.

I have digged into the Java Database Connectivity interfaces to understand the main interactions between roles and I could create several mocks to allow the critical path of the data flow go without any error

You will need to mock : DataSource, PreparedStatement, ResultSet and Connection,

Now the question is : Do I have to prepare tables before the tests ?
Well to be honest, only one Map. You can register a mapping between SQL requests and results (we will call it registeredSQLServerMappings).

And you can verify the results of your call by making the mocks update a mutable list, let’s call it calledStatements.

Here is the gist : https://gist.github.com/libetl/48beff8234a7e034762fa23f6692cb86. Anyone can copy and the only pre-requisite to use is to have mockK in the classpath (and kotlin).

To use it in your test,
– have a BeforeEach to do
val dataSource = mockk<DataSource>().simulateDatabase()
– register some mappings like with that example :
registeredSQLServerMappings["SELECT ID, TICKET_NUMBER, FIRST_NAME, LAST_NAME FROM DOCUMENT"] = listOf(
mapOf("ID" to 1234, "TICKET_NUMBER" to "ABCD", "FIRST_NAME" to "John", "LAST_NAME" to "Doe")
).toResultSet()

– use that DataSource in your data layer bean.
– Congratulations, you can now use a mocked database in your unit tests and they run in less than one second each.

G33k, Programmes

GraphQL in Kotlin : how to register generic scalars without sweating

If you are working like I do, you certainly are wrapping the smallest pieces of data to ensure they follow strict validation rules.
And you are likely to wrap the small field values in wrapping type, for these reasons :

– your IDE will tell you that you are manipulating an EmailAddress, an IPAddressV6 or a DomainName instead of a poor a char string.

– your tests will not need to concentrate on the possible data flows when irrelevant data is passed to downstream interactions

– your doc will be fluent to read, because you call a spade a spade.

– your code will stay human readable, because you will be able to add (a lot) of operations around your wrapping types (DomainName.getExtension(), EmailAddress.getDomainName(), IPAddressV6.isItPrivate())

These wrapper types can be bound to scalar in GraphQL, and I use graphql-kotlin to NOT need to create a clone of my mapping in a graphqls file (https://github.com/ExpediaGroup/graphql-kotlin)

But if you have plenty of them, you may end up wrapping each single wrapping type to a coercing function.

…Or even worse…

You will want to make all these independant wrapping types implement a type common interface.

Hang on, why don’t you make that generic and abstract, so you don’t have to care anymore about coercing in graphQL ?

Here is my attempt on the topic.

Whenever I use a wrapping type, I always use a @JsonCreator(mode = JsonCreator.Mode.DELEGATING) around it to make it also usable with downstream services not having graphQL but REST.

So during the schema creation, you may want to automatically create a coercing function for each like this below :

val alreadyCreatedWrapperTypes = mutableMapOf<KClass<*>, GraphQLScalarType>()
val hooks = object : SchemaGeneratorHooks {
   
override fun willGenerateGraphQLType(type: KType): GraphQLType? =
       
when (type.classifier as? KClass<*>) {
            
            in alreadyCreatedWrapperTypes.keys -> alreadyCreatedWrapperTypes[type.classifier]
           
else -> if ((type.classifier as? KClass<*>)?.constructors?.size == 1 &&
                (type.
classifier as? KClass<*>)?.constructors?.first()
                    ?.
annotations?.any { a ->
                   
a.
annotationClass == JsonCreator::class &&
                        (a
as JsonCreator).mode == JsonCreator.Mode.DELEGATING } == true) {
               
val newValue = wrappedValue<Any?>(type.classifier as KClass<*>)
                alreadyCreatedWrapperTypes[type.
classifier as KClass<*>] = newValue
                newValue
            }
else null
       
}
}

So above, you say that if I happen to find A @JsonCreator(DELEGATING) annotation on my data class and if I see only one constructor with only one parameter, then it certainly looks like a wrapper type.

Then after, you need to convert the wrapper type into the coercing function and to save the new scalar like this :

fun <T> wrappedValue(clazz: KClass<*>) = GraphQLScalarType.newScalar().name(
clazz.simpleName).description(
"Generic scalar (${clazz.simpleName})"
).coercing(object : Coercing<T, Any> {
override fun serialize(input: Any) =
(input::class.memberProperties.first() as KProperty<*>).getter.call(input)

override fun parseValue(input: Any): T =
clazz.constructors.first().call(input) as T

override fun parseLiteral(input: Any): T =
when (input) {
is StringValue -> clazz.constructors.first().call(input.value) as T
is IntValue -> clazz.constructors.first().call(input.value.toInt()) as T
is LongValue -> clazz.constructors.first().call(input.value) as T
else -> throw CoercingParseLiteralException("Invalid input '$input' for wrapped value")
}
}).build()

This will result in wrapped types that will keep their validation during the API call, but you don’t need to wrap them from the client.


e.g, use this syntax : {register(emailAddress: “john.doe@acme.com”){…}} and get the EmailAddress constructor validation for free.

G33k, Programmes

Coroutines in Spring API : give me !

Coroutines are a sharply increasing practice among Android developers, but it is seems quite limited to the Android ecosystem, even though the original intent of them is to create an agnostic framework to implement asynchronous and cheap concurrency programs.

There already is a framework that can help combine together coroutines and spring mvc : https://github.com/konrad-kaminski/spring-kotlin-coroutine

This framework is working nicely for hello world or CRUD Apis, but hardly adapts when you have spring security configured, with a LocaleContextHolder, a MDC map, or maybe also a RequestContextHolder attributes object.

This very rough implementation below will help you declare a CoroutineContext bean that you can bind directly in your RestController if you want to use coroutines seamlessly.
It will make a coroutineContext parameter available, and will also make the controller wait until the result has been resolved.

package com.company.myapi

import kotlinx.coroutines.ExecutorCoroutineDispatcher
import org.slf4j.MDC
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import org.springframework.context.annotation.Primary
import org.springframework.context.i18n.LocaleContextHolder
import org.springframework.core.MethodParameter
import org.springframework.security.core.context.SecurityContextHolder
import org.springframework.security.core.parameters.DefaultSecurityParameterNameDiscoverer
import org.springframework.web.bind.support.WebDataBinderFactory
import org.springframework.web.context.request.NativeWebRequest
import org.springframework.web.context.request.RequestContextHolder
import org.springframework.web.context.request.async.DeferredResult
import org.springframework.web.method.support.AsyncHandlerMethodReturnValueHandler
import org.springframework.web.method.support.HandlerMethodArgumentResolver
import org.springframework.web.method.support.HandlerMethodReturnValueHandler
import org.springframework.web.method.support.ModelAndViewContainer
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer
import org.springframework.web.servlet.mvc.method.annotation.DeferredResultMethodReturnValueHandler
import java.lang.reflect.Method
import java.util.concurrent.ExecutorService
import java.util.concurrent.Executors
import kotlin.coroutines.Continuation
import kotlin.coroutines.CoroutineContext
import kotlin.coroutines.intrinsics.COROUTINE_SUSPENDED
import kotlin.reflect.jvm.kotlinFunction

object CoroutinesInRestController {

@Configuration
internal class CoroutinesInjection {
@Bean
@Primary
fun parameterNameDiscovererWithCoroutines() =
object: DefaultSecurityParameterNameDiscoverer(listOf()) {
override fun getParameterNames(method: Method) =
((super.getParameterNames(method)?.toList() ?: listOf()) +
if (method.isSuspend) listOf("__continuation__") else listOf()).toTypedArray()
}
@Bean
fun coroutineContext() = object : ExecutorCoroutineDispatcher() {
override val executor = Executors.newFixedThreadPool(128)

override fun dispatch(context: CoroutineContext, block: Runnable) {
val securityContext = SecurityContextHolder.getContext()
val requestAttributes = RequestContextHolder.currentRequestAttributes()
val locale = LocaleContextHolder.getLocale()
val contextMap = MDC.getCopyOfContextMap()
executor.execute {
SecurityContextHolder.setContext(securityContext)
RequestContextHolder.setRequestAttributes(requestAttributes)
LocaleContextHolder.setLocale(locale)
MDC.setContextMap(contextMap)
block.run()
}
}

override fun close() {
executor.shutdown()
}
}
}

@Configuration
internal class CoroutinesWebMvcConfigurer : WebMvcConfigurer {

@Autowired
private lateinit var coroutineContext: CoroutineContext

override fun addArgumentResolvers(resolvers: MutableList<HandlerMethodArgumentResolver>) {
resolvers.add(0, coroutineArgumentResolver(coroutineContext))
}

override fun addReturnValueHandlers(handlers: MutableList<HandlerMethodReturnValueHandler>) {
handlers.add(0, returnValueHandler())
}
}

private const val DEFERRED_RESULT = "deferred_result"

private fun <T> isContinuationClass(clazz: Class<T>) = Continuation::class.java.isAssignableFrom(clazz)
val Method?.isSuspend: Boolean get() = this?.kotlinFunction?.isSuspend ?: false

fun coroutineArgumentResolver(coroutineContext: CoroutineContext) =
object : HandlerMethodArgumentResolver {
override fun supportsParameter(parameter: MethodParameter) =
parameter.method.isSuspend && isContinuationClass(parameter.parameterType)

override fun resolveArgument(parameter: MethodParameter, mavContainer: ModelAndViewContainer,
webRequest: NativeWebRequest, binderFactory: WebDataBinderFactory) =
object : Continuation<Any> {
val deferredResult = DeferredResult<Any>()

override val context: CoroutineContext
get() = coroutineContext

override fun resumeWith(result: Result<Any>) {
if (result.isSuccess) {
deferredResult.setResult(result.getOrNull())
} else {
deferredResult.setErrorResult(result.exceptionOrNull())
}
}
}.apply {
mavContainer.model[DEFERRED_RESULT] = deferredResult
}
}

fun returnValueHandler() =
object: AsyncHandlerMethodReturnValueHandler {
private val delegate = DeferredResultMethodReturnValueHandler()

override fun supportsReturnType(returnType: MethodParameter): Boolean =
returnType.method.isSuspend

override fun handleReturnValue(returnValue: Any?, type: MethodParameter,
mavContainer: ModelAndViewContainer, webRequest: NativeWebRequest) {
val result = mavContainer.model[DEFERRED_RESULT] as DeferredResult<*>

return delegate.handleReturnValue(result, type, mavContainer, webRequest)
}

override fun isAsyncReturnValue(returnValue: Any, returnType: MethodParameter): Boolean =
returnValue === COROUTINE_SUSPENDED
}
}

This special implementation will take care of keeping the ThreadLocal values up to date in each coroutine scope, so you can continue processing your request without standing out of the servlet context values.

That was useful for me, and maybe it will be for you too, so good luck.

G33k, Programmes

Routing tool in java or kotlin

This is how “experimented” developers (= developers who know that programs can fail) write code
kameamea

It is ok if you are alone on your project, you can even identify easily your legacy code visually by doing that.
It is not (always) ok if you work in a team. Sometimes one might just ask you : can you write a truth table of your algorithm.

Unless if you think your program is simple enough not to need documentation, you will end up writing something like this in your documentation source :

              arch     │ hasJava │ has.NET │ choose
            ───────────┼─────────┼─────────┼─────────
                       │         │         │ RSA
                       │   yes   │         │ JKS
              win      │         │         │ PKCS12
             win|linux │         │   yes   │ PKCS12 

I do think there is another way :
why don’t you use a table in your code ?
You can either do that by using configuration tools or business rules management systems.
Or by using a very naive util class :

https://gist.github.com/libetl/1e446d60da00d1af74a2303918496f1c#file-router-kt

It will allow you to configure routing tables directly inlined in code and then to use them at runtime.
The leftmost parameters will have more priority than the others.
You can also specify several values to match several possibilites rather than copy pasting the rows, just write a|b (a vertical bar) as separator.

Take a look at the RouterTest.kt file and don’t hesitate to copy paste some tables in your code to try out.

G33k, Programmes

Bad code practices : don’t review them, but write code to ban them.

This is getting boring when you are in charge of software quality :

“- stop sending primitive values in your parameters”
“- remove this foreign data structure from the code, it is error prone”
“- why can’t you just isolate this in a separate role ?”

I found myself repeating over and over these kind of sentences. But in 2019, there are new tools that help me become more persuasive. Instead of just using the well known Programming Mistake Detector tools like sonar or checkstyle…
…Some of them now allow to write rules as code, and to make them mandatory at build time.

So now, the code is responsible not only for doing his job, but also to make sure that the developer does not mess with it.

So I have (re)-defined some roles in domain driven design projects, here they are below :

package com.company.program.domain.transverse

import org.springframework.stereotype.Component
import org.springframework.stereotype.Repository
import org.springframework.stereotype.Service

object DomainDrivenDesignAnnotations {
/**
* The following role is the intention revealing code of the API.
* It must be written in declarative design and cannot be a facade to the code.
* The abstraction level must be respected in order to let the action
* describe the end to end.
*
* The action can only have a constructor and a public method, 0 private method
*/
@Component
annotation class Action

/**
* The following role is an orchestrator of several Repositories, Gateways
* and computation. It can act as a facility to tackle the complexity
* of the business logic that the action class cannot describe without
* running the risk of becoming too difficult to read.
*
* The domain service can have several public methods, however we strongly
* recommend that all those methods are semantically relating to the same
* use case
*/
@Service
annotation class DomainService

/**
* The following role is a caller to an external / foreign service
* You will find a SOAP/REST api call (and only one)
*
* Only one API call per Gateway is allowed, unless if the foreign API
* has different methods to get the same value
*
* A gateway must take in parameters Aggregates, Entities Or Value Types,
* and must return Aggregates, Entities or Value Types
*/
@Repository
annotation class Gateway

// Repository is already declared in spring, no need to redeclare

/**
* The following data structure belongs to a model that is not part of
* the program domain model, because its structure is not compatible.
*
* To avoid that case, we can try to reuse the value types, entities and
* aggregates from the domain model in the repositories or gateways.
*
* Which is not possible all the time because of the interface contracts.
*
* This means that the data structure must be hold inside the infra package,
* and should not be read or written by the actions and domain services.
*
* The foreign model must come with a converter
*
* ____________________________________________________________________
* | you are not allowed to pull any of these roles in the domain |
* | package, neither in an Action or a DomainService |
* -------------------------------------------------------------------
*/
annotation class ForeignModel

/**
* The following data structure is the root information to pass in an action
* or to be returned from an action.
*
* Only composite data structures can be Aggregate, and an aggregate
* cannot be a value type unless if the action has no access to
* Gateways and Repositories
*/
annotation class Aggregate

/**
* Indicates that the following data structure has no reference
* and cannot identify any repository value
*/
annotation class ValueType

/**
* Indicates that the following data structure contains at least one reference
* in a storage either in this program or in a foreign service
* Therefore must be considered as globally unique and identifiable in the system
*/
annotation class Entity
}

I took some time to describe those roles to my team (+ contributors) and to let them know how it impacts positively the transparency and the proficiency of the code.

The majority of them acquired the paradigm, and some others needed (and still need) to be guided.

So I have written some linter rules, with ktlint, that everyone can check out to apply them in any project : https://github.com/libetl/domaindrivendesign-ktlint-rules

You can add support for ktlint into your project (and also detekt to catch even more code smells), and then you can add those user-defined rules to enforce the use of domain driven design patterns.
Just add a ktlintRules(project(“:domain-driven-design-ktlint-rules”)) directive in your gradle project classpath.

Here are the user-defined rules contained in this project :

AClassWithoutFunctionMustBeADataClass.kt
ActionOnlyHasOnePublicMethod.kt
AllClassMembersMustBePrivateAndImmutable.kt
AllNonForeignDataClassesMembersMustBeImmutable.kt
DataClassNotAnnotated.kt
EachRoleShouldBeInTheRightPackage.kt
GatewayOrRepositoryMustHaveOnlyOneTemplateVariable.kt
NeedsOneCallToAnActionFromAController.kt
NoBreakOrContinue.kt
NoForOrWhileInActionClass.kt
NoForeignInfraUsageInInfra.kt
NoForeignModelInAnnotatedComponentContract.kt
NoGenericCatch.kt
NoPrimitiveObsessionInAnnotatedComponent.kt
NoTemplateUseInActionOrDomainService.kt

From now on, all these rules are correctly respected in the project and everyone can focus on flow instead of form during code reviews.

G33k, Programmes

Learn how to group expensive calls with the coroutines

In a micro-service fashion, everything is streamlined to small domains, small actions and therefore fast responses.

This is theorically true, BUT. Sometimes we may want to access to expensive in time information. Either a partner api which is slow, or a restricted access to a confidential network which takes seconds to route correctly.

You will run the risk of having the client api renewing the same request multiple times because the data has no fallback information (you cannot find a fallback for a credit card for example)

In this case, the Nagle’s algorithm’m will not work. The retries will happen each N seconds, and then the packets will get acknowledged before the grouping can happen.

It is time if not too late to group your api calls together. All the inbound retries will subscribe to the same backend call. And one successful technology to do that is the Coroutines in Kotlin.

We are going to create two roles in this process : a router to map each request to one task, and several workers whose role is to handle a task. The workers, which are not threads but coroutines, will always be changing contexts to avoid wasting cpu time.

The behavior is : The client api will send its input to the router, which will reply either with a new task or a currently running task. It will wait until completion of the task. The only intelligence there is to compare the input with the other saved inputs to see if two of them are the same. It will help the router to decide to merge two requests in one.

The repository in github is this one : https://github.com/libetl/coroutines-pooling-client

The test runs two scenarios : one having 100 requests of different calls with different inputs, and one having 100 requests exactly identical. Each backend call will be logged.

In the first scenario, each inbound call leads to a call to the backend (100 different inputs)

In the second scenario, there are 100 inbound calls but only 10 calls to the backend (you can even decrease that number if the latency of the backend increases). Why ? As you have guessed, some calls are grouped together.

Coroutines can help you implement a wide range of different strategies for your use cases, but what I foresee the most is the need for seamless implementations when you need your code to stay stupid and simple. And the coroutines framework easily help to do that by letting the business code dislocated from the boilerplate use of coroutines.

So now perf developers and craftsmen can become friends again.

Cuisine, Programmes

My Chocolate Cake recipe

Here is a chocolate cake recipe.
Please note that this is not only a recipe, but it is code too…
You can compile that in Kotlin successfully and you can imagine a cookbook site using this kind of implementation.

fun main(args: Array<String>) {
    val chocolateCake = `recipe consisting of` { -
            (7 `ounces of` "black chocolate") +
            (4 `ounces of` "flour") +
            (3 `ounces of` "sugar") +
            (1 ["baking powder"]) +
            (1 ["vanilla sugar powder bag"]) +
            (2 `glasses of` "oil") +
            (1 `ounce of` "white cheese") +
            (3 ["eggs"]) +
            (1 `tea spoon of` "orange zest") +
            (1 `ounce of` "butter")}
            .`then apply` { "black chocolate" `melted in` "a pan containing water" }
            .`then make sure that all the` {listOf(
                    "flour", "sugar", "black chocolate", "baking powder", "vanilla sugar powder"
            )`is disposed in the` "bowl"}
            .`mix everything`
            .`scramble the`("eggs")
            .`then pour all the` {listOf("oil", "white cheese", "eggs", "orange zest"
            ) `in the` "bowl"}
            .`mix everything`
            .`beat with a whip`
            .`bake for`(45.minutes)
    print(chocolateCake)
}

Here is the necessary support code to make that compile :

data class Operation(val verb: String,
                     val ingredient: Ingredient ? = null,
                     val where: String ? = null,
                     val duration: String = "immediate")

data class Ingredient(val name: String,
                      val quantity: Int,
                      val unit: ConversionUnit)

data class Recipe(val ingredients: List < Ingredient > = emptyList(),
                  val operations: List < Operation > = emptyList()) {

    fun operationsBuilder(ingredientNames: List < String >, verb: String, place: String ? = null, duration: String ? = null) =
            ingredientNames.map {
                ingredientName ->
                Operation(verb,
                        ingredient = this@Recipe.ingredients.find { it.name == resolveRealNameFrom(ingredientName) }
                                ?: throw IllegalArgumentException("ingredient not found"),
                        where = place,
                        duration = duration ?: "immediate")
            }

    fun`then apply`(operationsToDo: Recipe.() -> List<Operation>) = Recipe(ingredients, operations = operations + operationsToDo.invoke(this))

    fun`then make sure that all the`(operationsToDo: Recipe.() -> List<Operation>) = Recipe(ingredients, operations = operations + operationsToDo.invoke(this))

    fun`then pour all the`(operationsToDo: Recipe.() -> List<Operation>) = `then make sure that all the`(operationsToDo)

    infix fun List<String>.`is disposed in the`(place: String) = operationsBuilder(this, verb = "dispose", place = place)
    infix fun List<String>.`in the`(place: String) = this `is disposed in the` place

    infix fun String.`melted in`(place: String) = operationsBuilder(listOf(this), verb = "melt", place = place,
            duration = 10.minutes)

    val`mix everything` get() = Recipe(ingredients, operations = operations + Operation("mix"))

    val`beat with a whip` get() = Recipe(ingredients, operations = operations + Operation("beat with a whip"))

    fun`scramble the`(ingredientName: String) =
            Recipe(ingredients, operations = operations + operationsBuilder(listOf(ingredientName), "scramble"))

    fun`bake for`(duration: String) =
            Recipe(ingredients, operations + Operation(verb = "bake", duration = duration))

    operator fun plus(newIngredient: Ingredient) = Recipe(ingredients + newIngredient)
}

enum class ConversionUnit { OUNCES, QUANTITY, GLASSES, TEA_SPOON }

infix fun Int.`ounces of`(name: String) = Ingredient(name, this, ConversionUnit.OUNCES)
infix fun Int.`ounce of`(name: String) = this `ounces of` name
infix fun Int.`glasses of`(name: String) = Ingredient(name, this, ConversionUnit.GLASSES)
infix fun Int.`tea spoons of`(name: String) = Ingredient(name, this, ConversionUnit.TEA_SPOON)
infix fun Int.`tea spoon of`(name: String) = this `tea spoons of` name
fun`recipe consisting of`(theIngredientsImplementation: () -> Recipe) =
        theIngredientsImplementation.invoke()

operator fun Int.get(name: String) = Ingredient(resolveRealNameFrom(name),
        quantity = this, unit = ConversionUnit.QUANTITY)

val Int.minutes get() = "$this minutes"

fun resolveRealNameFrom(name: String) =
        name.replace(Regex("s$"), "")
                .replace(Regex(" bag$"), "")

operator fun Ingredient.unaryMinus() = Recipe(listOf(this))

Enjoy the cake.