Blog Title

Designing APIs for vibe coding

2025-10-03T00:00:00Z

Vibe coding is a point of no return. The fine art of designing APIs and crafting great DevEx? It needs to adapt to this new reality.

So how do we actually do that?

Before vibe coding

I've been doing API design for a while now, nitpicking on parameter names, function overloads, you name it. Over the years, I picked up from my mentors a few principles that I'd call Developer Experience canon.

First, your API needs to be idiomatic for the target audience. Being idiomatic covers a lot of ground: obvious stuff like naming conventions, but also the more subtle unwritten rules. If factory methods in a language follow a certain pattern in popular libraries, you better have a damn good reason to do it differently.

Integrability is another big one. Your API doesn't live in a vacuum—it's used alongside other APIs. If you're returning something iterable, use the standard library iterator type, as this most likely makes your type usable with other APIs.

And finally, discoverability: can people actually find your API features? Developers discover APIs in different ways: skimming docs, reading examples, or just downloading a template and CTRL + Space-ing their way through. As an API designer, you need to optimize for one or more of these learning paths.

To get concrete, let's look at the design choices behind the Context API from the Restate TypeScript SDK:

import * as restate from "@restatedev/restate-sdk";

async function myFn(ctx: restate.Context, input: Input): Promise<Output> {
  await ctx.run(mySideEffect);
}

Here's the problem: people don't know what Restate can do unless they read a good chunk of the docs and examples. So we put all the important features in one type called Context (a pretty common name for this kind of thing). Users can just hit CTRL + SPACE and boom—all the features we want them to discover:

You get run for recording side effects, sleep for delaying execution, serviceClient for calling other services, and more.

We went even further: when you define a restate function using restate.serve({...options}), the Context parameter is required by the type system—you'll get a type error without it. Technically Restate functions don't need the Context, but we make it mandatory anyway so users know it exists.

This works well for us, but we could've gone different routes.

For example, we could've just had individual functions you import and ditched the Context entirely:

import * as restate from "@restatedev/restate-sdk";

async function myFn(input: Input): Promise<Output> {
  await restate.run(mySideEffect);
}

Sure, this looks cleaner in examples, but discoverability takes a hit. CTRL + Space would give you a ton of results, including stuff you shouldn't use inside a restate function (like serve). How do you fix that? Maybe another import like import * as restateFeature from "@restatedev/restate-sdk/features"? But then users need to know about that import. And good luck naming that namespace!

Every design has tradeoffs. With our Context approach, you have to pass it through your code instead of just importing a function anywhere. There's no free lunch!

What changed with vibe coding

Let's look at the example that gave me the idea for this post:

This API is another example of optimizing for autocompletion: you pass the service to ctx.serviceClient, and some TypeScript type manipulation plus Proxy gives you a fully typed client with autocompletion.

Let me run a test with Copilot using GPT as the model:

In my function, can you call anotherFn using the context?

Hallucinating methods that don't exist. Let me try again:

Still making things up. Different prompt:

In my function, can you send a request to anotherFn using the context?

Switch to Claude Sonnet 4, and after a few tries, we finally get it right:

With older models, Copilot would make up even more unique stuff, trying to invent a fetch-like API:

ctx.request({
  service: "anotherService",
  handler: "anotherFn",
  body: "my input"
})

This is creative, but completely made up.

Sure, LLMs get better over time. Eventually they get trained on your APIs or pick up feedback. And yeah, they're probabilistic—they won't always nail it.

But here's the thing: a vibe coder will throw prompts like these at your API, and you want them building features, not debugging LLM hallucinations. They won't discover your API by reading docs—they'll discover it by asking the LLM to solve a problem.

This leads to a new design principle: maximize your API's vibe-ability—the ease with which developers can discover and correctly use your API through LLM-assisted coding.

Why is the vibe-ability so low?

Not totally sure, but I've got a few theories for serviceClient.

First, the type manipulation/proxy approach. The LLM needs specific training on that: it can't just look at the Context type, pick the right method, and fill in the parameters. It needs to deeply understand TypeScript's type system, perhaps using the LSP output in a feedback loop. That, or it needs to be trained on your API examples.

Second theory: in the concept space of TypeScript and requests, fetch and plain function calls are the closest neighbors the LLM can think of. During training, the model probably saw fetch way more than client APIs like ours.

How do I increase vibe-ability?

I don't have a scientific answer—and I'd love to know if one exists. But after testing different prompts against our APIs, my empirical takeaway is simple: flat and verbose beats concise, conformity beats originality.

It's a bit unfortunate—this likely means less innovative API designs will make it into the wild. Perhaps even worse, adoption will become a tougher game for new programming languages.

Now every API I design gets a final test: I throw it at ChatGPT, Gemini, and Claude, ask them to assume the API exists, then see how close they get to what I've designed. To check if the new API vibes for real 🔮👩‍💻🌌

Rust macros are not just about DRY

2025-02-05T00:00:00Z

In the last weeks I've been re-working some internals of Restate that enable the SDKs used by our users to interact with the Restate server. At a glance, the Restate server talks with the SDKs using a protocol on top of HTTP that transports few Protobuf messages, prefixed by a header containing message type, length and flags. Easy enough.

If you've ever found yourself implementing something similar, you know there's a lot of boring matches involved in the process: convert from the message type in the code to some MessageType enum, then from MessageType enum to the prost::Message implementation, and so on and so forth. And often you also have some additional intrinsic properties of the Message themselves that you need to represent in the MessageType enum, leading to again more matches.

Sounds like a perfect case for a nice shiny macro right?

The argument against macros

Let's face it, unless you're a macro magician, macros aren't super approachable to develop and maintain for all Rust developers. And I think this is true both for the world of procedural macros and for macro_rules!.

In order to develop a procedural macro, you need to dig deep into the Rust syntax tree data model, requiring a lot of code to handle all the different cases correctly, especially when adding generics to the mix. On top, it requires to set up a different crate, perhaps leading you to other issues that you didn't need to deal with in the first place. If you've never developed a Rust procedural macro, I suggest to look at thiserror, it's one of the best procedural macro implementations I've read so far.

When it comes to macro_rules!, well well well... Substitutions can be challenging to understand, both from a syntax perspective but also because of the matching/expansion rules Rust applies, that sometimes are sort of unexpected. There's a reason why the Rust Macro book has a section only about Minutiae, which you definitely encounter and exploit too.

At the end of the day, procedural macros and macro_rules! are respectively a framework and a separate language within Rust, and as such it needs people that know them in order to mantain them.

Because of that, it is fair that some people refrain from creating macros, especially when they don't bring enough value. You probably heard or read somewhere arguments like "if you need 100 lines of macro_rules to generate 120 lines of code, then it's not worth it". I think there's more to macros though, than just ratio between macro LOCs vs generated LOCs.

Macros are not just about DRY

Macros have way more benefits than just DRY, which I can sum up as follows: macros are an excellent tool to develop a DSL evaluated at compile time!

Let's go back to our protocol: we need to describe the message types with their various intrinsic properties, and whether we achieve that through ifs, matches, a bunch of traits with dynamic dispatch, that's just an implementation detail. This is exactly what I would use a DSL for!

By skimming through the protocol message macro here:

// Syntax: <Message type> <Message kind> <flags>* = <code>,
gen_message!(
    Start Control = 0x0000,
    Call Command allows_ack = 0x040D,
    Call CompletionNotification noparse = 0x800D,
    Signal Notification noparse = 0xFBFF,
    // [...]
);

You can quite easily deduct for a given message type what is the message kind, the message code, the flags, and so on. With the help of this macro, I can define with a simple DSL my data model with all its properties and the macro generates for me the code implementing it, including the MessageType enum, methods like MessageType::code(), MessageType::requires_ack(), encoding/decoding of messages, and so on. Even better, I can now change and/or expand the generated code, without needing to go through the match statements one by one fixing the individual bits.

At the end of the day, macros are an awesome tool to define DSLs to describe your data model at compile time!

We have other good examples of macros like this in Restate, for example the define_table! macro, that we use to define our SQL tables in Datafusion:

define_table!(sys_journal(
    /// [Invocation ID](/operate/invocation#invocation-identifier).
    id: DataType::LargeUtf8,
    /// The index of this journal entry.
    index: DataType::UInt32,
    /// The entry type. You can check all the available entry types in [`entries.rs`](https://github.com/restatedev/restate/blob/main/crates/types/src/journal/entries.rs).
    entry_type: DataType::LargeUtf8

    // [...]
));

This macro generates for us few data structures, including the Schema used by DataFusion, and the RowBuilder to fill the DataFusion rows during a table scan.

What's even more interesting in this macro is that we reuse the /// doc comments by copying them in a static string (compiled behind a feature gate) and we read them later with a tool to generate our SQL documentation.

Use with moderation

I hope I gave you some good examples of how to use Rust macros. As every "advanced" feature in every programming language, you should ponder pragmatically whether your problem can use a macro or not.

Few tips I can give are:

Refrain from overcomplicated macro input syntaxes when you can and rather try to stick to familiar things such as assignment like syntax or struct/enum definition like syntax.
Document the macro input syntax and the generated output (of course!), no matter if it's an internal macro or something you expose in a Rust crate you publish/maintain.
Before delving into proper macro_rules! implementations, read the Rust macro book, and generally stick to the patterns described in the book. Most of the macro_rules! I developed so far are a combination of push down accumulators and TT munchers.
If you use RustRover/Intellij for Rust, crank up max heap before touching a macro code!

I hope you liked this post, I plan to do more of that in the future, so stay tuned for updates!

JUnit 5, Parallel tests, Extensions and ThreadLocal

2022-02-19T00:00:00Z

In the Apache Flink community we're in the process of porting our huge test codebase to JUnit 5. In order to leverage as much as we can the new JUnit 5 features, in the past days I've spent some time playing around with it.

In this blog post I'll talk about my story about enabling the project to use the new JUnit 5 features, including parallel execution, parametrized tests and extensions, and how they're going to help us improve our test codebase.

Our test codebase

In Flink we've all kind of tests you can think of:

Simple unit tests with a certain degree of mocking
Integration tests
End-to-end tests
Various other tests for utilities, such as tests for our bash scripts

In most of the codebase, we refer to integration tests as tests that define and run a streaming job, but use a mocked sink and source. While end-to-end tests are like integration tests, but they use real external systems, such as Kafka, deployed with TestContainers.

In particular in Flink SQL, unsurprisingly, we have a lot of integration tests, because each single feature requires to be "understood" by all the different moving parts of our stack. For example, take the built-in function COALESCE: it has a runtime implementation, a Table API expression ^[1], a custom logic for arguments type inference and return type inference, an optimizer rule that removes the call when possible. Each of these single pieces need to work in harmony, and integration tests usually gives us the guarantee that everything fits together.

Another aspect of JUnit 5 tests is that we have a lot of test bases and of parametrized test bases, à la Junit 4. This is due to the organic growth of the project and the effort to try to standardize certain test aspects, like starting and stopping a Flink MiniCluster, the embedded Flink cluster to run test jobs.

Goals

In porting to JUnit 5, we want to:

Have less test bases, but more extensions, hence composition over inheritance. This simplifies contributing new tests, as adding a new "capability" to the test suite won't require new ad-hoc test bases.
Improve error reporting and test cases separation, in order to make the contributor experience nicer both when running tests from the IDE and with Maven
Speedup the test suite as much as possible

In particular the last point is a hot topic, as today a CI run usually takes between 1 and a half and 2 hours, having a significant impact on the development loop of the project.

Parallel tests

Parallel tests are, in my opinion, the killer feature of JUnit 5 and the real incentive to port JUnit 4 tests to JUnit 5.

With JUnit 4 you can parallelize test execution using the build tool, for example using maven-surefire-plugin fork JVM feature. It runs tests in parallel by spawning several JVM processes, where each of them gets assigned a split of the overall list of tests to run. JUnit 5 on the other hand runs all the tests within the same JVM: the test runner manages a thread pool and takes care of assigning test cases to threads.

I think the JUnit 5 approach fits best in our use case, as spawning several JVMs is very resource intensive on constrained machines such as CI runners, so it's a constant source of issues. This statement is also valid for contributor's machines, as these days just running the browser with 20+ tabs open, Spotify, Slack and the IDE can easily eat up to 16Gb of RAM. Plus they work in any IDE without additional configuration, they can help you find out thread safety bugs and the granularity of the execution is easy and flexible to configure.

To start using JUnit 5 parallel tests, we just had to create a file called junit-platform.properties in our test resources and add the following:

junit.jupiter.execution.parallel.enabled = true
junit.jupiter.execution.parallel.config.strategy = dynamic
junit.jupiter.execution.parallel.mode.default = same_thread

This configuration enables to opt-in specific tests/classes to run in parallel. To flag a test class to run in parallel, we need to annotate it with @Execution(CONCURRENT). Thanks to this configuration we can gradually enable parallel execution only for tests we know are safe.

JUnit 5 offers the ability to configure the granularity of the parallel test execution, e.g. run all test cases from all classes in parallel, run all test cases from a class sequentially but run the test classes in parallel, etc. Check out all the available options in the JUnit 5 documentation.

So I've decided I wanted to try this new feature with some complex parametrized test base. I ended up choosing CastRulesTest, a parametrized unit test base checking the runtime implementation of the CAST logic. Because there is no shared state whatsoever, this test suite is embarrassingly parallelizable. Just adding the annotation @Execution(CONCURRENT) gave me a 3x times faster execution time for the whole suite.

Integration tests in parallel

That 3x totally got my attention, so I wanted to try to apply the same annotation to integration tests as well. As my next target, I've chosen BuiltInFunctionTestBase. As the name implies, this is a parametrized integration test base we use to test the correct behaviour of our built-in functions. We have around 10 classes where we use this test base, for a total of 1200+ integration test cases.

Porting the test suite to JUnit 5

The first thing I had to do was to port to JUnit 5 the base class and its inheritors. My initial thought was to use @TestFactory feature, a new JUnit 5 feature to spawn dynamic tests, allowing you to group test cases. Think to it as a more powerful @ParametrizedTest.

This would have allowed me to have a nice nested view of the tests in the reports. For example, look at MathFunctionsITCase: by mixing DynamicTest and DynamicContainer I could have achieved a report like:

MathFunctionsITCase:
- PLUS:
  - f0 + 6 = 20: Success
  - f0 + 6 = 10: Failure
- MINUS: [...]

Because the test base itself already had some models to define test cases, including input/output data, query expressions and configuration (see TestSpec), what I had to do was simply to convert these models to DynamicTest/DynamicContainer. A little refactor of the testFunction method to wrap the logic into DynamicTest did it. Now every concrete test class just had to implement the abstract method getTestSpecs to return the test cases defined with my TestSpec class, so the final implementation of the @TestFactory just looked like:

@TestFactory
Stream<DynamicContainer> tests() {
    return getTestSpecs().map(TestSpec::toDynamicContainer);
}

Every implementation class provided a DynamicContainer, containing a set of tests for a specific built-in function, like PLUS, and each container had a set of DynamicTest with the specific test cases for that built-in function, like f0 + 6 = 20.

Last but not least, to let the query run, I needed the extension to set up MiniCluster once per class. This is already available in our flink-test-utils, so with some copy-paste I enabled it:

public static final MiniClusterWithClientExtension MINI_CLUSTER_RESOURCE =
        new MiniClusterWithClientExtension(
                new MiniClusterResourceConfiguration.Builder()
                        .setNumberTaskManagers(1)
                        .build());

@RegisterExtension
public static final AllCallbackWrapper<MiniClusterWithClientExtension> ALL_WRAPPER =
        new AllCallbackWrapper<>(MINI_CLUSTER_RESOURCE);

Tried a first run without parallel tests, and everything ran fine. Tried to add @Execution(CONCURRENT), and Intellij IDEA welcomed my idea with a long report full of red crosses.

Fixing the `MiniCluster` extension first

Looking at the logs, it became evident how the problem was MiniClusterWithClientExtension, given several tests were trying to push jobs to a MiniCluster already shut down.

The MiniClusterWithClientExtension was developed by wrapping the JUnit 4 MiniClusterWithClientResource rule in a custom interface defining before and after. Then, to register it, you could either use AllCallbackWrapper or EachCallbackWrapper to define whether to have one MiniCluster per test class, or per single test.

In JUnit 4 rules have a before and after extension point, and then when you register them, depending on whether you use @Rule or @ClassRule, the rule is executed for each test or once per class. In JUnit 5 the user cannot pick whether the extension is used globally or per method: it's the extension itself that defines where it can hook in the lifecycle.

An example of this shift of concept between JUnit 4 and 5 is provided by the JUnit 5 TestContainers integration, which depending on whether the container field is static or not, decides to share it between test methods or not^[2].

Our AllCallbackWrapper or EachCallbackWrapper were circumventing the new JUnit 5 Extension paradigm, bringing back the same semantics of @Rule or @ClassRule. And this worked fine, until I tried to use parallel execution.

The MiniClusterWithClientExtension#before method was starting MiniCluster, creating some ClusterClient and then setting up a thread local for the environment configuration lookup.

The interaction between our ThreadLocal configuration when using AllCallbackWrapper didn't sound right, so I tried to do a little experiment. Take this simple extension:

public class TestExtension
        implements BeforeEachCallback, BeforeAllCallback, AfterEachCallback, AfterAllCallback {
    @Override
    public void beforeAll(ExtensionContext context) throws Exception {
        System.out.println("before all: " + Thread.currentThread().getName());
    }

    @Override
    public void beforeEach(ExtensionContext context) throws Exception {
        System.out.println(
                "before " + context.getDisplayName() + ": " + Thread.currentThread().getName());
    }

    @Override
    public void afterEach(ExtensionContext context) throws Exception {
        System.out.println(
                "after " + context.getDisplayName() + ": " + Thread.currentThread().getName());
    }

    @Override
    public void afterAll(ExtensionContext context) throws Exception {
        System.out.println("after all: " + Thread.currentThread().getName());
    }
}

This is simply going to print the thread where the various hooks are executed, and it also prints the name of single test in case in beforeEach/afterEach. Then I tried to run this test:

@ExtendWith(TestExtension.class)
@Execution(ExecutionMode.CONCURRENT)
public class MyTest {

    @ParameterizedTest(name = "{0}")
    @ValueSource(ints = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10})
    void name(int i) {
        System.out.println("test: " + i + ", thread: " + Thread.currentThread().getName());
    }
}

And here is the result:

before all: ForkJoinPool-1-worker-1
before 2: ForkJoinPool-1-worker-3
before 3: ForkJoinPool-1-worker-4
before 5: ForkJoinPool-1-worker-6
before 9: ForkJoinPool-1-worker-1
before 4: ForkJoinPool-1-worker-5
before 7: ForkJoinPool-1-worker-0
before 6: ForkJoinPool-1-worker-7
before 1: ForkJoinPool-1-worker-2
test: 7, thread: ForkJoinPool-1-worker-0
test: 9, thread: ForkJoinPool-1-worker-1
test: 4, thread: ForkJoinPool-1-worker-5
test: 1, thread: ForkJoinPool-1-worker-2
test: 6, thread: ForkJoinPool-1-worker-7
test: 2, thread: ForkJoinPool-1-worker-3
test: 3, thread: ForkJoinPool-1-worker-4
test: 5, thread: ForkJoinPool-1-worker-6
after 4: ForkJoinPool-1-worker-5
after 7: ForkJoinPool-1-worker-0
after 6: ForkJoinPool-1-worker-7
after 1: ForkJoinPool-1-worker-2
after 5: ForkJoinPool-1-worker-6
after 9: ForkJoinPool-1-worker-1
after 3: ForkJoinPool-1-worker-4
after 2: ForkJoinPool-1-worker-3
before 8: ForkJoinPool-1-worker-3
test: 8, thread: ForkJoinPool-1-worker-3
before 10: ForkJoinPool-1-worker-6
after 8: ForkJoinPool-1-worker-3
test: 10, thread: ForkJoinPool-1-worker-6
after 10: ForkJoinPool-1-worker-6
after all: ForkJoinPool-1-worker-1

As I was expecting, beforeEach and afterEach runs in the same thread of the test, as effectively they just "wrap" the test method, as described here.

In other words:

There is no guarantee about which thread is used to execute beforeAll and afterAll, but
beforeEach and afterEach are guaranteed to be executed within the same thread of the test

So here was the solution: we just had to set/unset that ThreadLocal everytime in beforeEach/afterEach, no matter whether the MiniCluster instance was meant to be per test class or per test method.

I was ready to declare victory, so I did some refactoring of MiniClusterWithClientExtension to always create one MiniCluster per test class, I removed the wrapping around AllCallbackWrapper and then I tried to run again my tests: still everything was red.

Back on `@ParametrizedTest`

After some investigation, I found out that DynamicTest lifecycle doesn't work per DynamicTest instance, while parallelization does. For example, for this test class:

@ExtendWith(TestExtension.class)
@Execution(ExecutionMode.CONCURRENT)
public class MyTest {

    @TestFactory
    public Stream<DynamicTest> tests() {
        return IntStream.rangeClosed(1, 10)
                .mapToObj(i ->
                  DynamicTest.dynamicTest(
                    String.valueOf(i),
                    () -> System.out.println("test: " + i + ", thread: " + Thread.currentThread().getName())
                  ));
    }
}

You get this output:

before all: ForkJoinPool-1-worker-1
before tests(): ForkJoinPool-1-worker-1
after tests(): ForkJoinPool-1-worker-1
test: 2, thread: ForkJoinPool-1-worker-3
test: 6, thread: ForkJoinPool-1-worker-7
test: 3, thread: ForkJoinPool-1-worker-4
test: 9, thread: ForkJoinPool-1-worker-1
test: 7, thread: ForkJoinPool-1-worker-0
test: 1, thread: ForkJoinPool-1-worker-2
test: 4, thread: ForkJoinPool-1-worker-5
test: 5, thread: ForkJoinPool-1-worker-6
test: 8, thread: ForkJoinPool-1-worker-3
test: 10, thread: ForkJoinPool-1-worker-2
after all: ForkJoinPool-1-worker-1

As you see, each DynamicTest is executed in parallel, as you would expect, but the beforeEach and afterEach is executed only once per @TestFactory. Because of this behaviour, the tests were not picking the correct ThreadLocal, hence failing the test because they could not find the MiniCluster.

It seems like there is no solution to this problem, and there is an issue open in the JUnit 5 issue tracker about whether this should be supported or not.

So I had to fall back to the good old @ParametrizedTest: I just removed the nested DynamicContainer, and I created my own DynamicTest like data structure to wrap a test Executable and its display name:

/** Single test case. */
static class TestCase {

    private final String name;
    private final Executable executable;

    TestCase(String name, Executable executable) {
        this.name = name;
        this.executable = executable;
    }

    @Override
    public String toString() {
        return name;
    }
}

And then I kept the same code as before in the test base, but I had to add a method to flatten the previously used DynamicContainers:

abstract Stream<TestSpec> getTestSpecs();

final Stream<TestCase> getTestCases() {
    return this.getTestSpecs()
            .flatMap(testSpec -> testSpec.getTestCases(this.getConfiguration()));
}

@ParameterizedTest
@MethodSource("getTestCases")
final void test(TestCase testCase) throws Throwable {
    testCase.executable.execute();
}

The report doesn't look as nice as with @TestFactory, but it finally works fine with parallel execution.

Results and conclusion

Before the parallelization, this test suite ran in around 50 seconds from Intellij IDEA on my i7 11th Gen packed with 16Gb of RAM. After the parallelization the suite runs in around 20 seconds^[3]. Not bad.

I think JUnit 5 is a very powerful tool, but you need to be careful when developing extensions that are supposed to work with parallel tests. These are my gotchas from this experience:

Make sure thread locals are set in beforeEach, so they're set in the right thread, and cleaned up in afterEach, to avoid next tests reusing that thread to pick up the old ThreadLocal values
Be aware that the method beforeEach could be invoked in parallel on the same instance of the extension, so you need to make sure the access to extension fields is thread safe, or...
Use the ExtensionContext.Store, as it's thread safe, it's hierarchically organized per hook context, and it's particularly useful to auto cleanup objects built in the before[..] methods
Rather than providing accessors in the extension class, use parameter injection implementing ParameterResolver together with the ExtensionContext.Store. This simplifies the implementation when storing your extension state in ExtensionContext.Store.

And last but not least, don't use @TestFactory in conjunction with parallel execution if your test requires ThreadLocal or other thread dependant thing correctly configured.

The DSL provided by Flink SQL to define relational queries without SQL ↩︎
TestContainers - JUnit 5 integration ↩︎
Please take these numbers with a grain of salt, my measurements were very empirical, and not proper benchmarks ↩︎

Vert.x container images with jlink

2020-10-08T00:00:00Z

In this blog post I'm gonna show you how I managed to reduce the container image size of an Eclipse Vert.x application, creating a smaller JDK with jdeps and jlink.

The application is the data plane of Knative Eventing Kafka Broker, an implementation of Knative Broker tailored on Kafka.

Fat-jar

We didn't handwritten nor generated a module-info.java for our application, we just ship a fat-jar. To generate a fat-jar, configure the maven-shade-plugin:

<plugin>
   <groupId>org.apache.maven.plugins</groupId>
   <artifactId>maven-shade-plugin</artifactId>
   <version>${maven.shade.plugin.version}</version>
   <configuration>
     <minimizeJar>true</minimizeJar>
   </configuration>
   <executions>
     <execution>
       <phase>package</phase>
       <goals>
         <goal>shade</goal>
       </goals>
       <configuration>
         <transformers>
           <transformer
             implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
             <mainClass>
               ${main-class}
             </mainClass>
           </transformer>
         </transformers>
       </configuration>
     </execution>
   </executions>
</plugin>

Jdeps

jdeps is the tool you must use to figure which JDK modules you depend on. If you run jdeps just with the fat-jar as argument, you'll get a list of all the packages in your jar and the JDK modules it depends on:

% jdeps receiver/target/receiver-1.0-SNAPSHOT.jar
[...]
   dev.knative.eventing.kafka.broker.core             -> java.lang                                          java.base
   dev.knative.eventing.kafka.broker.core             -> java.lang.invoke                                   java.base
   dev.knative.eventing.kafka.broker.core             -> java.net                                           java.base
   dev.knative.eventing.kafka.broker.core             -> java.time                                          java.base
   dev.knative.eventing.kafka.broker.core             -> java.time.format                                   java.base
   dev.knative.eventing.kafka.broker.core             -> java.util                                          java.base
   dev.knative.eventing.kafka.broker.core             -> java.util.concurrent                               java.base
   dev.knative.eventing.kafka.broker.core             -> java.util.function                                 java.base
   dev.knative.eventing.kafka.broker.core             -> java.util.stream                                   java.base
[...]

Note that jdeps analyzes only the imports so, if you perform some reflections at runtime of JDK classes, they won't be found by the tool.

To get an output that you can directly pass to jlink:

% jdeps -q --print-module-deps --ignore-missing-deps receiver/target/receiver-1.0-SNAPSHOT.jar
java.base,java.compiler,java.naming,java.security.jgss,java.security.sasl,java.sql,jdk.management,jdk.unsupported

This is the list of jdk modules we depend on. Doing a quick check, I've found that:

java.base it's the module that contains all the core features of the jdk
java.compiler contains the compiler types. It's brought in by Guava and Vert.x CLI feature.
java.naming contains some JNDI types, to perform names lookups. This is required to perform DNS queries by Vert.x
java.security.jgss and java.security.sasl contains some security protocols implementation. They're used by the Java Kafka client.
java.sql contains JDBC. Jackson databind and Google Gson (that we import transitively via Protobuf json) depends on it because they provide marshallers/unmarshallers for JDBC types.
jdk.management contains some interfaces to manage the JDK. This is used by Micrometer to instrument the JVM and collect metrics.
jdk.unsupported contains sun.misc.Unsafe. Netty and Protobuf use it to perform off-heap allocations.

Zero days since it was DNS

Because some reflection is happening behind the hood to choose the Vert.x DNS resolver, jdeps doesn't discover the module jdk.naming.dns, that you need in your Vert.x application to enable the DNS.

To resolve the deps and add the dns module:

MODS=$(jdeps -q --print-module-deps --ignore-missing-deps receiver/target/receiver-1.0-SNAPSHOT.jar)
echo "Computed mods = '$MODS'"
# Patch adding the dns
MODS="$MODS,jdk.naming.dns"

Create the JDK

Now you just need to invoke jlink to generate your custom JDK:

% jlink --verbose \
  --no-header-files \
  --no-man-pages \
  --compress=2 \
  --strip-debug \
  --add-modules "$MODS" \
  --output ./jdk
java.base file:///usr/lib/jvm/java-15-openjdk-15.0.0.36-1.rolling.fc32.x86_64/jmods/java.base.jmod
java.compiler file:///usr/lib/jvm/java-15-openjdk-15.0.0.36-1.rolling.fc32.x86_64/jmods/java.compiler.jmod
java.logging file:///usr/lib/jvm/java-15-openjdk-15.0.0.36-1.rolling.fc32.x86_64/jmods/java.logging.jmod
java.management file:///usr/lib/jvm/java-15-openjdk-15.0.0.36-1.rolling.fc32.x86_64/jmods/java.management.jmod
java.naming file:///usr/lib/jvm/java-15-openjdk-15.0.0.36-1.rolling.fc32.x86_64/jmods/java.naming.jmod
java.security.jgss file:///usr/lib/jvm/java-15-openjdk-15.0.0.36-1.rolling.fc32.x86_64/jmods/java.security.jgss.jmod
java.security.sasl file:///usr/lib/jvm/java-15-openjdk-15.0.0.36-1.rolling.fc32.x86_64/jmods/java.security.sasl.jmod
java.sql file:///usr/lib/jvm/java-15-openjdk-15.0.0.36-1.rolling.fc32.x86_64/jmods/java.sql.jmod
java.transaction.xa file:///usr/lib/jvm/java-15-openjdk-15.0.0.36-1.rolling.fc32.x86_64/jmods/java.transaction.xa.jmod
java.xml file:///usr/lib/jvm/java-15-openjdk-15.0.0.36-1.rolling.fc32.x86_64/jmods/java.xml.jmod
jdk.management file:///usr/lib/jvm/java-15-openjdk-15.0.0.36-1.rolling.fc32.x86_64/jmods/jdk.management.jmod
jdk.naming.dns file:///usr/lib/jvm/java-15-openjdk-15.0.0.36-1.rolling.fc32.x86_64/jmods/jdk.naming.dns.jmod
jdk.unsupported file:///usr/lib/jvm/java-15-openjdk-15.0.0.36-1.rolling.fc32.x86_64/jmods/jdk.unsupported.jmod

Providers:
  java.base provides java.nio.file.spi.FileSystemProvider used by java.base
  java.naming provides java.security.Provider used by java.base
  java.security.jgss provides java.security.Provider used by java.base
  java.security.sasl provides java.security.Provider used by java.base
  jdk.naming.dns provides javax.naming.spi.InitialContextFactory used by java.naming
  java.management provides javax.security.auth.spi.LoginModule used by java.base
  java.logging provides jdk.internal.logger.DefaultLoggerFinder used by java.base
  jdk.management provides sun.management.spi.PlatformMBeanProvider used by java.management

This command will grab the modules you provided from your local machine and will create a JDK without manual, header files, debug symbols:

jdk
├── bin
│   ├── java
│   └── keytool
├── conf
│   ├── logging.properties
│   ├── net.properties
│   ├── sdp
│   └── security
[...]
├── lib
[...]
└── release

Complete script and Dockerfile

This is the complete Dockerfile (simplified) that builds the project and generates the JDK:

ARG JAVA_IMAGE=docker.io/adoptopenjdk:14-jdk-hotspot
ARG BASE_IMAGE=docker.io/ubuntu:bionic

FROM ${JAVA_IMAGE} as builder

WORKDIR /app

# https://github.com/AdoptOpenJDK/openjdk-docker/issues/260
RUN apt-get update && apt-get install -y binutils

# Copy all my project stuff inside the container
COPY . .

# Build the project
RUN ./mvnw package -DskipTests -Deditorconfig.skip --no-transfer-progress

# Generate the sdk using the script (shown below)
RUN ./generate_jdk.sh /app/receiver/target/receiver-1.0-SNAPSHOT.jar

# Prod image
FROM ${BASE_IMAGE} as running

# Create appuser and directories
RUN groupadd -g 999 appuser && useradd -r -u 999 -g appuser appuser && \
  mkdir /tmp/vertx-cache && chown -R appuser:appuser /tmp/vertx-cache && \
  mkdir /app

# Copy jar and jdk inside /app
COPY --from=builder /app/jdk /app/jdk
COPY --from=builder /app/receiver/target/receiver-1.0-SNAPSHOT.jar /app/app.jar
RUN chown -R appuser:appuser /app

# Set appuser and configure PATH
USER appuser
WORKDIR /app
# Add jdk bin to the path, so you can run the java command
ENV PATH="/app/jdk/bin:${PATH}"

And the generate_jdk.sh script:

#!/usr/bin/env sh

if [ $# -eq 0 ]
  then
    echo "No arguments supplied, You must provide the jar"
fi

echo "Computing mods for $1"
MODS=$(jdeps -q --print-module-deps --ignore-missing-deps "$1" | sed -e 's/^[ \t]*//')
echo "Computed mods = '$MODS'"

# Patch adding the dns
MODS="$MODS,jdk.naming.dns"
echo "Patched modules shipped with the generated jdk = '$MODS'"

jlink --verbose --no-header-files --no-man-pages --compress=2 --strip-debug --add-modules "$MODS" --output /app/jdk

You can run the built container with:

% docker run imagename java -jar /app/app.jar

Conclusions

Using jdeps and jlink our container image size is 2.5x smaller, which is a great achievement for us. We also plan to reduce it even further, in fact we want to:

Investigate some modules we depend on, but that we probably don't need: java.compiler and java.sql are two good candidates.
Rebase our base image on alpine, which uses musl as libc. Look at Project Portola for more details.

Check out the full PR: https://github.com/knative-sandbox/eventing-kafka-broker/pull/265

Kubernetes controllers - The Empire Strikes Back

2020-10-08T00:00:00Z

In this blog post I want to introduce you to some important steps forward we made in our Extending Kubernetes API In-Process project. If you don't know what I'm talking about, check out the previous post: Kubernetes Controllers - A new hope.

We implemented the high-level ABI to start watchers on Kubernetes resources, as argued in Next steps paragraph. This allows us to identify watch requests sent by controllers and to deduplicate them.

Then we worked on modifying our ABI to make it asynchronous, in order to invoke the controllers only on-demand. Now, we don't spin up a thread for each module, but we wake controllers up asynchronously when there is a new task to process.

Thanks to these changes, we're now able to use Rust's async/await inside the module. This allowed us to realign our fork of kube-rs, bringing back all the original interfaces, and run kube-runtime inside the modules.

The `kube-watch-abi` ABI

The kube-watch-abi ABI was originally composed by an import and an export:

#[link(wasm_import_module = "kube-watch-abi")]
extern "C" {
    // Returns the watch identifier
    fn watch(watch_req_ptr: *const u8, watch_req_len: usize) -> u64;
}

#[no_mangle]
pub extern "C" fn on_event(watch_id: u64, ev_ptr: *const u8, ev_len: usize) {
    // handle event
}

The watch_req is a description of the watch to register:

pub(crate) struct WatchRequest {
    pub(crate) resource: Resource,
    pub(crate) watch_params: WatchParams
}

Controller

When the module invokes the watch import, it registers a new watch and returns a watch identifier. This identifier is stored together with a reference to the Stream<WatchEvent> in a global map. In similar fashion to the request ABI discussed in the previous post, we serialize the WatchRequest data structure in order to pass it to the host.

Then, every time the host has a new WatchEvent that the controller needs to handle, it will invoke on_event with the serialized WatchEvent. Using the watch identifier, the controller will get the associated Stream from the global map and append the deserialized WatchEvent to it.

The host

When the controller invokes watch, the host checks if there is a registered watch for that resource. If there is, then it just registers the invoker as interested to that watch, otherwise it starts a new watch. Every time a watch receives a new event from Kubernetes, the host resolves all the modules interested to that particular event. For each module, it allocates some module memory to pass the event and it finally wakes the controller up by invoking on_event.

Rust module with async-await

Callbacks are bad!

The kube-watch-abi is the de-facto an asynchronous API: watch starts the asynchronous operation, on_event notifies on the completion of the operation.

In the initial implementation of the watch, on module side, I just modified the kube client watch to provide a callback:

pub fn watch<F: 'static + Fn(WatchEvent<K>) + Send>(&self, lp: &ListParams, version: &str, callback: F) {
    // Invoke watch
}

Every time on_event was invoked, the module resolved the callback from the watch identifier and invoked it. This was fine as an initial approach, but we soon hit a problem: Porting all of the existing Kubernetes client and runtime code from async/await to the more primitive callbacks could have caused a lot of issues, making the fork diverge too much from the upstream code.

How `async`/`await` works

Here's a little refresh on async/await from Asynchronous Programming in Rust book:

async/.await is Rust's built-in tool for writing asynchronous functions that look like synchronous code. async transforms a block of code into a state machine that implements a trait called Future. Whereas calling a blocking function in a synchronous method would block the whole thread, blocked Futures will yield control of the thread, allowing other Futures to run._

Although there are a lot of details about how Rust implements the async/await feature, these are the relevant concepts for this post:

Future is the type that represents an asynchronous result, e.g. async fn returns a Future. You can wait for the result of a Future using .await.
Stream is the same as Future, but it returns several elements before None, which notifies the end of the stream.
In order to use .await, you must be in an async code block.
In order to run an async code block, you need to use a task executor.

For example:

use futures::executor::LocalPool;

fn main() {
    // Create the executor
    let mut pool = LocalPool::new();

    // Spawn an async task
    poll.spawner().spawn(async {
        // execute `my_async_fn()` and await for the result
        my_async_fn().await
    });

    // Block the main thread until all the tasks are done
    pool.run();
}

In our async/await implementation we implemented the Future and Stream traits, we reused LocalPool from the futures crate as a task executor, and we generalized on_event to notify the asynchronous operation completion.

The controller lifecycle

We first analyzed the controller lifecycle:

When host invokes run, the controller starts a bunch of watchers and then it waits for new events
Every time a new event comes in, the host wakes the controller up by calling on_event

This means that during the run phase, the controller starts a bunch of asynchronous tasks, one or more of them waiting for asynchronous events. After the run phase completes, there is no need to keep the module running. When we wake the controller up again, we need to check for any tasks that can continue and run them up to the point where we have all the tasks waiting for an external event.

To implement this lifecycle, we need to execute both on run and on on_event the executor method run_until_stalled(), which will run all tasks in the executor pool and returns if no more progress can be made on any task. This allows us to implement run as follows:

#[no_mangle]
pub extern "C" fn run() {
    // Get the global executor
    let exec = kube::abi::get_mut_executor();
    // Start the main task
    exec.deref().borrow_mut().spawner().spawn(main()).unwrap();
    // Give a little push to the executor.
    // This runs until all tasks up to when they're all waiting for async results completion.
    exec.deref().borrow_mut().run_until_stalled();
}

Our custom `Future`/`Stream`

In order to encapsulate the pending asynchronous operation, we implemented our Future. The implementations are straightforward and pretty much the same as explained in the Rust async book:

/// Shared state between the future and the waiting thread
struct AbiFutureState {
    // The result value
    value: Option<Vec<u8>>,
    // Was the future completed?
    completed: bool,
    // Object that notifies the completion to the executor
    waker: Option<Waker>,
}

impl Future for AbiFuture {
    type Output = Option<Vec<u8>>;

    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
        // Look at the shared state to see if the timer has already completed.
        let mut shared_state = self.shared_state.lock().unwrap();
        if shared_state.completed {
            Poll::Ready(shared_state.value.take())
        } else {
            shared_state.waker = Some(cx.waker().clone());
            Poll::Pending
        }
    }
}

A similar implementation exists for the Stream trait. To create a Future, we use this method:

pub fn start_future(async_operation_id: u64) -> AbiFuture {
    // Create the future state
    let state = Arc::new(Mutex::new(
        AbiFutureState {
            value: None,
            completed: false,
            waker: None
        }
    ));
    // Insert in the global map of pending future states this future state
    get_pending_futures().deref().borrow_mut().insert(async_operation_id, state.clone());

    // Return the new future
    AbiFuture {
        shared_state: state
    }
}

Generalizing `on_event` to `wakeup_future`/`wakeup_stream`

At this point, we took the concept of on_event and generalized to async/await, introducing wakeup_future/wakeup_stream ABI exports. Every time the controller invokes an asynchronous ABI import (like watch), it gets the identifier that we use to instantiate our Future/Stream implementation. When the host completed the asynchronous operation, it invokes wakeup_future/wakeup_stream. The controller marks the Future/Stream as completed, including the result value, and invokes LocalPool::run_until_stalled() to wake up tasks waiting for that future/stream to complete.

#[no_mangle]
pub extern "C" fn wakeup_future(future_id: u64, ptr: *const u8, len: usize) {
    // Retrieve the global pending future states map
    let fut_state = get_pending_futures();
    let waker = {
        let state_arc = fut_state.deref().borrow_mut().remove(&future_id).unwrap();
        let mut state = state_arc.lock().unwrap();

        // If the pointer is not null, fill the Future result value
        if !ptr.is_null() {
            state.value = Some(unsafe {
                Vec::from_raw_parts(
                    ptr as *mut u8,
                    len as usize,
                    len as usize,
                )
            });
        }

        state.completed = true;
        state.waker.take()
    };
    // Use the waker to notify the executor this future is completed
    if let Some(waker) = waker {
        waker.wake()
    }

    // Let's try to execute stuff up to the point where there isn't anything else to execute
    get_mut_executor().deref().borrow_mut().run_until_stalled();
}

The complete flow

You can find the complete code regarding async/await support here: executor.rs

More async ABI methods!

After we implemented async/await in our WASM modules, we refactored the request ABI discussed in our previous post as an asynchronous ABI method:

#[link(wasm_import_module = "http-proxy-abi")]
extern "C" {
    fn request(ptr: *const u8, len: usize) -> u64;
}

Now the returned value is the asynchronous operation identifier and, to signal the completion of the request, the host invokes wakeup_future.

We also included a new ABI method to sleep the execution of the module:

#[link(wasm_import_module = "delay-abi")]
extern "C" {
    fn delay(millis: u64) -> u64;
}

This is necessary to run the kube-runtime, which performs some sleeps before synchronizing the internal cache again.

Redesigning the host as an event-driven application

The first implementation of the new kube-watch-abi ABI was a little rough: a lot of blocking threads, shared memory across threads, some unsafe sprinkled here and there to make the code compiling. Because of that, we redesigned the host to transform it in a full asynchronous application made of channels and message handlers. For every asynchronous ABI method there is a channel that delivers the request to a message handler, which processes the request, computes one or more responses and sends them back to another channel. This last channel delivers messages to the AsyncResultDispatcher, owner of the module instances, that invokes the wakeup_future/wakeup_stream of the interested controller.

Today we have 3 different message handlers, one for each async ABI method:

kube_watch::Watchers that controls the watch operation. This message handler is also able to deduplicate the watch operations
http::start_request_executor to execute HTTP requests
delay::start_delay_executor to execute delay requests

When the host loads all the modules, it executes the ABI method run for each module, then it transfers the ownership of module instances to AsyncResultDispatcher that will start listening for new AsyncResult messages on its ingress channel.

Because all the message handlers and channels are async/await based, if all the handlers are in idle, virtually no resource is wasted with threads waiting.

Since AsyncResultDispatcher controls all the different module instances, it avoids invoking the same controller in parallel: LocalLoop is a single threaded async task executor, hence a module cannot process multiple async results in parallel.

Compiling kube-runtime to WASM

Thanks to all the async changes, we managed to realign most of the APIs of kube-rs to the original ones. This allowed us to port kube-runtime to our WASM controllers.

The kube_runtime crate contains sets of higher level abstractions on top of the Api and Resource types so that you don't have to do all the watch/resourceVersion/storage book-keeping yourself.

The problem we experienced with compiling kube-runtime to WASM is that it depends on tokio::time::DelayQueue, a queue that yields components up to a specified deadline. DelayQueue uses the Future type called Delay to effectively implement delays. The problem with this Delay is that it's implemented using the internal ticker of the Tokio async task executor Runtime, which we don't use inside WASM modules.

In order to fix this issue, we had to fork the implementation of tokio::time::DelayQueue and reimplement the Delay type using the delay ABI shown previously:

pub struct Delay {
    fut: Pin<Box<dyn Future<Output=()> + Send>>,
    deadline: Instant
}

impl Delay {
    pub(crate) fn new_timeout(deadline: Instant) -> Delay {
        let now = Instant::now();
        match deadline.checked_duration_since(now) {
            Some(dur) => Delay {
                fut: Box::pin(kube::abi::register_delay(dur)),
                deadline
            },
            None => Delay {
                fut: Box::pin(futures::future::ready(())),
                deadline: now
            }
        }
    }

    // [...]
}

impl Future for Delay {
    type Output = ();

    fn poll(mut self: Pin<&mut Self>, cx: &mut task::Context<'_>) -> Poll<Self::Output> {
        self.fut.as_mut().poll(cx)
    }
}

With the custom implementation of DelayQueue, the rust-runtime compiled successfully to WASM, and we managed to port our controllers to use it!

async fn main() {
    let client = Client::default();

    let simple_pods: Api<SimplePod> = Api::namespaced(client.clone(), "default");
    let pods: Api<Pod> = Api::namespaced(client.clone(), "default");

    Controller::new(simple_pods, ListParams::default())
        .owns(pods, ListParams::default())
        .run(reconcile, error_policy, Context::new(Data { client }))
        .for_each(|res| async move { match res {
            Ok((obj, _)) => println!("Reconciled {:?}", obj),
            Err(e) => println!("Reconcile error: {:?}", e),
        }}).await;
}

Show me the code!

You can check out the complete code of controllers today here: simple-pod controller

If you want to look at all the different changes the project went through, look at these PRs:

Next steps

Now the controller module looks like a real Kubernetes controller: the difference is minimal to a controller targeting the usual deployment style. We also opened up a door for important optimizations, thanks to the watch ABI method. The host refactoring and the async ABI methods should also simplify the future interaction with Golang WASM controllers, because our ABI now resembles the asynchronous semantics of their WASM ABI.

Our next goals are:

Hack the Golang compiler to fit our ABI (or a similar one). For more info check out the previous blog post
Perform a comparison in terms of resource utilization between this deployment style using WASM modules and the usual one of 1 container per controller.
Figure out how to handle the different ServiceAccounts per controller

Stay tuned!

Kubernetes Controllers - A new hope

2020-07-28T00:00:00Z

Today I'm going to introduce you an idea Markus Thömmes and I had to package, deploy and operate Kubernetes controllers in a modern and efficient way that may have a fundamental impact on the Kubernetes ecosystem.

Introduction

Kubernetes is the standard de-facto container orchestration engine of these days. As every orchestrator of "something" (bare metal machines, VMs, containers, etc), one day you need to expand the available abstractions the system provides to you. For Kubernetes, that day came around 3 years ago where they introduced the concept of Custom Resource.

To put it simply, everything you create/read/update/delete in Kubernetes is an API Resource. A CRD (Custom Resource Definition) defines a new resource you can create/read/update/delete in your cluster, which is not included in the built-in ones, while a CR (Custom Resource) is an instance of the CRD. For example, a Pod is a Kubernetes resource from the core API group, a Broker is a CRD introduced when you install Knative Eventing on your Kubernetes cluster. With a definition of a new resource, you need to implement the business logic to orchestrate it, that is to handle create/read/update/delete operations on the resource. For example, you might want spawn a new etcd pod every time somebody creates a new EtcdCluster CR. In order to do that, you need to implement a Kubernetes controller: an application that listens for events on one or more API resources and reacts performing some operations on the cluster.

Since the introduction of CRDs, we've seen a huge expansion of extensions to Kubernetes APIs. Kubernetes vendors are using CRDs to provide distributions well integrated with their underlying systems, software vendors are developing controllers to simplify the operations on Kubernetes environment (for example Strimzi to deploy Kafka on Kubernetes), new projects are born to provide middleware and tooling to develop applications on top of Kubernetes (Knative to create serverless applications). Marketing around Kubernetes even forged a new name for the last use case: Kubernetes native applications, aka applications where the API-surface relies on the CRDs. In OpenShift there is even a system, based on a bunch of CRDs, to install other CRDs with their respective controllers.

Although there is a distinction between controllers and operators, for the sake of simplicity in this post I'm going to always refer to controller as the entity that implements the business logic of a CRD.

This concept works pretty well for Kubernetes, although there are still some open questions about the controller itself. In this post I'm going to introduce you a solution Markus Thömmes and I designed that may revolutionize how we package, deploy and operate Kubernetes controllers.

Kubernetes controllers today

Today a Kubernetes controller consists of more or less:

A main loop that does long polling to Kubernetes to watch one or more resources
On every event, a function called reconciler is invoked to align the cluster state to the desired state. For example: if the desired state expects a Pod to be running, but for some reason there isn't, then the reconciler creates it.
The reconciler optionally set the status of the CR, like Ready, SomeError, ...

After we implement the controller, we need to configure and apply the Custom Resource Definition and we need to setup a Service Account, that is an account to connect to our Kubernetes cluster in order to perform the operations the reconciler does.

During the years a lot of tooling was created, mostly in Golang, to implement these controllers, depending on the user needs. To mention some of them:

Other projects, like Knative, opt for creating and maintaining their own tooling to implement controllers.

These tools usually provide:

A library to improve the event handling logic: "smart" events queue that deduplicate and filter events for example
A code generator component that, starting from the CRD defined in the code, scaffolds the controller code and generates the client code to interact with the CRD

In essence, all of these tools work the same though in that ultimately they will generate a binary containing the controller. Each binary can in theory contain a number of controllers, for example Knative's framework actively encourages that with a main interface to start multiple controllers at same time. Crucially though, collapsing more than one controller into one binary and thus one process in the Kubernetes cluster needs to happen at compile time. With the systems commonly used, we'll get at least one process (aka Pod) per project that extends Kubernetes (i.e. at least one Pod for Knative CRDs, at least one Pod for Strimzi CRDs etc.). Each of those processes will be mostly idle for most of their lifetime, but they will steadily consume resources nonetheless. In short: We're flooding our Kubernetes clusters with applications that suck resources and remain in idle for most of their time!

Extending Kubernetes APIs in process

The idea behind our solution is to create a runtime plugin system where every plugin is a Kubernetes controller.

This plugin system should have certain traits:

Plugins should be well isolated between each other so no single plugin's failure/bad behaviour can cause trouble in other plugins
The host should be able to control and restrict what the plugin can do
Users should be able to develop plugins with different languages
The host should be able to load and unload plugins at runtime, without restarting the system

This plugin system can be seen as a mega controller. It can run as its own process, or it could even be merged with the kube-controller-manager, which is the process with all the controllers that manage the Kubernetes built-in resources:

Isolated, multi-language, lightweight plugin system? That sounds hard...

The requirements of the plugin system are not trivial, but lucky for us WebAssembly (Wasm) comes in rescue!

Wasm is an instruction set, targetable from every programming language, that runs in an isolated Virtual Machine.

From webassembly.org: "WebAssembly (abbreviated Wasm) is a binary instruction format for a stack-based virtual machine. Wasm is designed as a portable compilation target for programming languages, enabling deployment on the web for client and server applications".

That sounds like the perfect fit for our idea:

Because it's isolated by default, we define all the possible interactions between the plugins (controllers) and the host, piece by piece
Because it runs in a sandboxed virtual machine, it can easily isolate catastrophic failures and guarantee safe access to specific memory regions
Because it's a general purpose instruction set, more similar to an ISA of a modern CPU other than the JVM bytecode, most languages can target it (GC-enabled, dynamic/static typed, ...).

A list of supported languages is available here and here. Wasm is flexible enough that it can be executed with ahead of time (AOT) compilation or with a just in time (JIT) interpreter.

Although WebAssembly sounds like a web or browser technology, in the last years several people are experimenting with this technology outside the browser. Most notably, CloudFlare Function as a Service offering called Workers uses Wasm to run C/Rust code using Google's V8 isolations.

With WebAssembly, our plugin system looks like:

When we develop a controller, the compiled artifact is a binary Wasm module. A Wasm engine loads this module, eventually compiles it in AOT cases, and executes it.

At this point, we need to define the interaction semantics between the host and the plugins. In order to do that, we define the ABI (Application Binary Interface) of our plugin system: the functions the host imports in our plugin and the functions the plugin exports in our host.

Show me your ABI!

First of all, an example of a Wasm ABI. This is the Rust code for a Wasm module:

#[link(wasm_import_module = "my-abi")]
extern "C" {
  fn println(ptr: *const u8, len: usize);
}

#[no_mangle]
pub extern "C" fn foo() {}

This Wasm module exports the function foo, which means that, after the module has been loaded, the engine can invoke foo to execute it. On the other side, when we load the module in the engine, we must link some logic to the function println in order to run it, otherwise we'll get a link error.

We can get all imports and exports of a module using a tool called wasm-nm.

% wasm-nm module.wasm -i #Get the imports
i println

% wasm-nm module.wasm -e #Get the exports
e foo

This is the most important design choice of the project: the ABI influences the capabilities the host needs to implement, the portability of existing controllers, the APIs on the plugin side and so on. When we design the ABI we also need to take the differences of the programming languages into account: as an example, some may require additional interfaces to the host to run asynchronous code. The async schedulers in Rust and the goroutine scheduler in Golang both require a syscall to sched_yield to yield a thread execution in this example.

WASI

Lucky for us, part of the job that includes the low-level primitives is covered by a work in progress spec called WASI (WebAssembly System Interface). Their goal is to provide a reduced set of POSIX-like APIs: read/write files, get system clock, get environment variables and program arguments, read/write a socket and so on. Although, as they state in their rationale document, it's not goal of WASI to include primitives to open sockets/files, this is left to the users. From their docs:

One of WebAssembly's unique attributes is the ability to run sandboxed without relying on OS process boundaries. Requiring a 1-to-1 correspondence between wasm instances and heavyweight OS processes would take away this key advantage for many use cases. Fork/exec are the obvious example of an API that's difficult to implement well if you don't have POSIX-style processes, but a lot of other things in POSIX are tied to processes too. So it isn't a simple matter to take POSIX, or even a simple subset of it, to WebAssembly.

A socket is a process wide resource. This means that if module A opens a TCP socket, module B might have access to it if we don't implement proper security countermeasures.

Designing the ABI

Given our knowledge and studies of the current state of art for Wasm ABIs, we found out there are 3 possible non mutually exclusive approaches to the ABI design:

High level ABI: Expose the Kubernetes client functionalities
Medium level ABI: Proxy the HTTP client functionalities (send HTTP request, receive HTTP response)
Low level ABI: Proxy the syscalls (epoll, read, write, bind, ...)

We think that a good solution should mix all these 3 "levels". Some low-level APIs are always necessary for basic things like logging, getting configuration from environment, setting timeouts and so on. Proxying HTTP can be useful to invoke services outside Kubernetes (e.g. to trigger a cloud vendor API to enable a service). Exposing the Kubernetes client APIs unlocks a great potential to optimize the controllers beyond the optimized resource usage indicated above. Most prominently: the so called informer infrastructure can be shared between all the controllers. An informer is the part of a controller that listens for events on given resources. It builds up local caches that effectively reflect the state of said resources in-memory. In our case, the host would setup these informers and watches just once per resource. If two controllers want to watch ConfigMaps for example, the host would only need to setup one watch and keep one cache, reducing the amount of network traffic due to event delivery and memory consumption due to caching drastically. That effect becomes more and more pronounced as more extensions are being added.

Following the above approach, the host has always the full control of process resources (open files, open sockets, etc), we allow the modules to perform only certain operations and, most important, we open up for huge optimizations.

Our prototype

Markus and I built a prototype of the host and of the 2 controllers, a simple pod spawner and the Memcached example from operator-sdk. You can find all the code here: https://github.com/slinkydeveloper/extending-kubernetes-api-in-process-poc

The host is implemented in Rust and Wasmer, a popular Wasm engine supporting different compilation backends and with several language bindings. We implemented the controllers with Rust using a hacked version of kube-rs client. The host logic is pretty simple: when it starts, it reads the contents of the specified dir, looking for .yaml containing the module manifests. An example manifest:

name: memcached
abi: rust_v1alpha1

name is the operator name and abi is the abi the host should use to interact with the module. This allows to support different ABIs at the same time, mainly to overcome the differences between the programming languages and to support old modules running on new host versions.

Then, for each module, it compiles and runs it in a separate thread invoking the exported run function. The module, through the custom ABI we designed, can interact with Kubernetes to start watching resources and react to the events.

The prototype contains several simplifications that we can overcome easily (like watching the modules directory more than loading all the modules once), while others require more engineering as discussed later in this post.

The ABI

In order to create a running prototype, we ended up with a pretty simple ABI that mixes the low level WASI syscalls with a medium level HTTP client proxy functionality:

% wasm-nm memcached.wasm -i
i request
i clock_time_get
i fd_write
i poll_oneoff
i random_get
i proc_exit
i environ_sizes_get
i environ_get

% wasm-nm memcached.wasm -e
e run
e allocate

The exports are run to start the controller and allocate to allocate memory on the module. The implementer of the controller just implements the resources watch loop inside the run function.

In Wasm the module cannot access to the host memory, but the host can access to the module memory and copy bytes inside it. Because the Wasm module might have a memory allocator, a GC or any other mechanism to manage memory, the module should export a function that allocates memory to let the host copy bytes back to the module.

request is the function to perform a blocking and buffered HTTP request, while all the other imports come from WASI. The user interacts with our hacked version of the Kubernetes client, which under the hood invokes request to perform HTTP requests using reqwest crate. All our APIs are blocking because currently there is no out of the box async/await support in Wasm modules, I'll cover this limitation later.

The request flow should give you an idea of what it takes to implement an ABI:

The host takes care of handling the authentication, in order to avoid the controller having to have access to files at all.

The controller

An example code for the controller looks like:

let client = Client::default();

let foos: Api<SimplePod> = Api::namespaced(client.clone(), "default");
let inform = Informer::new(foos).params(ListParams::default().timeout(1));

let pods: Api<Pod> = Api::namespaced(client.clone(), "default");

loop {
    let events = inform.poll().expect("Poll error");

    for e in events {
        match e {
            Ok(WatchEvent::Added(o)) | Ok(WatchEvent::Modified(o)) => {
                reconcile_pod(&pods, &o.name(), &o.spec.image).expect("Reconcile error");
            }
            Ok(WatchEvent::Error(e)) => println!("Error event: {:?}", e),
            Err(e) => println!("Error event: {:?}", e),
            _ => {}
        }
    }
}

You can look at the full controller code for the above sample here. The user potentially doesn't know that their code is run inside an isolation. Everything is hidden behind the usual Kubernetes client APIs. In other words: The programming model does not change.

Next steps

There are some things we still haven't tested, but that we're willing to do in order to give us a clear idea of how to grow this idea:

Re-design the ABI to support unbuffered & asynchronous HTTP requests. This requires several additions to both the Wasm engine and the host code, most notably the engine should have the ability to yield the execution and the module should have an async executor to execute async operations.
The proposed ABI mixes medium level and low level. We want to try to implement a high level function in our ABI for watching Kubernetes resources, in order to register a single watch for all the controllers.
The actual ABI doesn't have error management at all, but that shouldn't be hard to implement: the serialized ABI data structures might be enums with error as one of the variants, so then the error is propagated back to the user as a Result.

Wait, where is Golang?

Our dream is to port existing controllers to this approach. Most of Kubernetes world today lives in Golang, but up until this point I didn't talk at all about Golang. Why?

The reason is that we found some critical issues within the Wasm/Golang support, among the others Golang assumes that the Wasm engine runs inside a JS VM using wasm_exec.js. In order to understand the impact of this choice, let me go through the Golang standard library implementation in Wasm.

Today in Golang, when you compile to Wasm, all the standard library I/O operations go through the syscall/js module, which invokes a predefined set of imports in your module. From the doc: Package js gives access to the WebAssembly host environment when using the js/wasm architecture. Its API is based on JavaScript semantics.:

Global() allows you to get the global object
Invoke() allows you to use a js.Value as a function and invoke it

Now let's analyze how the net/http client implementation on Wasm: When the user invokes RoundTrip, the request is transformed in a Javascript object, then using the syscall Global and Call the javascript global function fetch is invoked and finally, on the returned promise, the two callbacks are set to handle response and error.

The problem with this approach, although it works well in the browsers, is that the engine needs to implement all these JS functions used by Golang standard library. If we're not using a Javascript based runtime, we need to reimplement this. Further, it requires us to secure access to the global object fields, in order to retain the isolation properties.

To make things even worse, at the moment there is no way to define imports and exports within the module, so we cannot define and evolve an ABI that fits our needs. The only workable solution with the existing tools consists in define "fake" Javascript functions on the host side for the imports and setting "fake" fields on the "fake" global object for the exports, but this approach makes the code far more complex and convoluted, both on host and Wasm module side.

While the official Golang distribution doesn't support definining imports and exports, TinyGo supports it

Markus managed to run the Memcached operator example of operator-sdk, using NodeJS as a host, hacking some polyfills to match the expectations of wasm_exec.js. The final module ABI is:

% wasm-nm test.wasm -i
i debug
i runtime.resetMemoryDataView
i runtime.wasmExit
i runtime.wasmWrite
i runtime.nanotime1
i runtime.walltime1
i runtime.scheduleTimeoutEvent
i runtime.clearTimeoutEvent
i runtime.getRandomData
i syscall/js.finalizeRef
i syscall/js.stringVal
i syscall/js.valueGet
i syscall/js.valueSet
i syscall/js.valueIndex
i syscall/js.valueSetIndex
i syscall/js.valueCall
i syscall/js.valueNew
i syscall/js.valueLength
i syscall/js.valuePrepareString
i syscall/js.valueLoadString
i syscall/js.copyBytesToGo
i syscall/js.copyBytesToJS

% wasm-nm test.wasm -e
e run
e resume
e getsp

I was initially puzzled by the resume and getsp exports of the Wasm module, but then digging in the wasm_exec.js codebase their existence was justified: using resume and getsp the host can yield the execution of the module, perform some async operations, and then resume the execution of the code. That's the very same solution I would love to implement in our Rust ABI prototype and, as far as I know, WASI is looking for a generalized solution to support async runtimes inside the modules.

This experiment proves that Golang/Wasm support can definitely work and we think that, without the constraints of syscall/js and with the ability to define our ABI, we could port all the existing controllers to compile in Wasm isolated modules.

Some interesting issues to follow about Golang and Wasm related to this discussion:

There is still a long road ahead

Our long-term goal is to improve the Kubernetes ecosystem, creating a production ready plugin system with low footprint. We encourage the Kubernetes community to give us feedback on our findings and our prototype, so we can work together on shaping the future of Kubernetes controllers!

Check out the second part of this blog post series: Kubernetes controllers: The Empire Strikes Back

Reproducibility of benchmarks in Kubernetes

2019-12-02T00:00:00Z

Hi everybody! Today I'm going to talk you about how I've managed to run a benchmark inside Kubernetes in a reproducible manner.

Reproducibility: What and Why

Reproducibility means that different runs of the same benchmark, testing the same system, running in the same environment, should lead to similar results.

This is one of the most important traits that every benchmark should respect, because without it, the test can't be trusted.

For example, let's assume your boss gave you to optimize the most important system in production. You begin writing a benchmark to understand how it performs and, without caring about reproducibility, you jump to start searching where the performance hotspots are and operate to solve them. Now you run again the tests and results are better than the beginning but, while you're already feeling the bonus on the next paycheck, a colleague comments your "40% performance boost" PR saying "I tried to run the benchmark and results look worse than the beginning!". What the heck? Did your PR improves the performance or not?

You can't really answer that question, because your benchmark is not reproducible! You can try to run it several times but you'll continue to get deniable and not correlated results, that can answer positively or negatively to your question.

Making the test reproducible, for a good part, depends on the environment where you run the test.

Kubernetes is a virtualized environment designed to scale up & down workloads, depending on resource demands. So it can arbitrarily schedule your application to run where it wants, imposing precise cpu/memory resources. I'll show you what countermeasures I've took in my methodology to run benchmarks inside K8s to prevent such problems.

System under test: Knative Eventing

A couple of months I've started working on a project called Knative Eventing, an event mesh for Kubernetes. One of the goals of Knative Eventing is to enable message consuming & producing through HTTP, acting as a bridge between a "traditional" messaging system (such as Kafka) and an HTTP application.

I won't cover all aspects of Knative Eventing, If you want to learn more about it check out the Knative Eventing documentation

Knative, among the others, provides the concept of Channel, a flow of events from one or more producers to one or more subscribed consumers:

To push events into the channel you interact with its HTTP interface, while to receive events from the channel you subscribe to it, specifying at what HTTP endpoint the channel should send the events. Behind the hood, a pod called dispatcher is actually serving the HTTP interface for inbound events, managing the interaction with the messaging system and dispatching the events to the subscribers.

In this post I will use the test that calculates KafkaChannel's throughput and latency.

Test & Cluster

The test components are:

a sender container that forces a configurable load for a certain amount of time on the Channel
a receiver container that subscribes to the channel
an aggregator container that fetches results from sender and receiver containers and calculates latencies and throughputs

All these three components runs inside the cluster.

I won't spend words in this post to explain how such components, designed together with Knative community, work in deep. If you want to know more about it, look at the README.

The cluster where I'm running the tests is composed by three bare metal machines in the same rack and they're running only these tests. It's quite important to run on bare metal, otherwise you will need to make further steps to make your virtualization environment reproducible, depending on the VM system you use.

Step 0: How to determine reproducibility?

The question that arises is: what metric should be used to determine reproducibility? A wise answer could be that the standard deviation of the metric used to determine a performance improvement should be used to determine reproducibility.

In my case I'm going to use standard deviation of the percentiles of E2E latency (from sender to receiver) across several runs. The lower is the standard deviation, more reproducible is the test.

To improve reproducibility, I'll start by configuring and running the test 5 times, to calculate a baseline standard deviation. Then I'll show you the tweaks I've made to reduce the standard deviation to an acceptable value:

Configure the test to don't blow up the system
Pin containers to nodes
Restart the system after each run
Configure the resource limits

Step 1: Configure the test to don't blow up the system

The first step is to configure the test to correctly generate a load that doesn't blow up the system. System must be stressed, but in such a way that doesn't lead to a complete degradation, or even a crash.

I've configured my test to force 500 requests per second for 30 seconds, which I've found, experimentally, that is a good configuration the system can hold. Bear in mind that different "requests per second" configurations leads to different latencies!

I've collected the 99%, 99.9% and 99.99% percentiles but I'll focus on 99% percentile because I've managed to do only few and short test runs, and in such situations outliers are more visible and not filtered out in higher percentiles. In a "production run" of the test, you should run it for more than 30 seconds, to understand if higher latencies happens frequently.

After a first run, just configuring the test and running it for 5 times, I've these results:

	P99	P99.9	P99.99
Run 1	266.266179	276.945500	284.709000
Run 2	264.750750	278.127000	283.149000
Run 3	250.629000	263.994500	271.937000
Run 4	250.594875	261.605000	272.635000
Run 5	266.224393	282.690500	290.529000

The SD of P99 is 8.312 and, in particular, the relative standard deviation is 3.2%.

From experimental evidence I've found that the relative standard deviation is not linearly related with the test configuration, which means that the more stress is applied by the load generator, the more could be the relative standard deviation.

Let's try to dig into why these numbers are so different and how I've lowered them.

Step 2: Pin containers to nodes

The first thing you can notice is that the third an fourth run performed with generally lower numbers than the others. Digging a bit with kubectl describe nodes I've found that Kubernetes was scheduling on each run pods in different nodes. Sometimes it scheduled the sender and receiver in the same node of Kafka Channel dispatcher, letting them communicate with lower latencies!

To let Kubernetes deploy the pods always in the same nodes, I've configured the affinity of sender, receiver and all SUTs (system under test, which in my case means the Kafka Channel dispatcher and the Kafka cluster). To do it, I've defined three labels:

bench-role: kafka: Where Kafka cluster and Zookeeper are deployed
bench-role: eventing: Where the kafka dispatcher is deployed
bench-role: sender: Where both sender and receiver are deployed

And then, I set these labels in my cluster using:

kubectl label nodes node_name bench-role=eventing

On every deployment/pod descriptor, I've configured affinity in my various deployment descriptors.

I deployed Kafka using Strimzi and, thanks to its Kafka CRD, I can easily configure the affinity too (I've omitted irrelevant parts of this config):

kafka:
  template:
    pod:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: bench-role
                    operator: In
                    values:
                      - kafka
zookeeper:
  template:
    pod:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: bench-role
                    operator: In
                    values:
                      - kafka

For kafka-ch-dispatcher, I just modified the original dispatcher yaml adding the nodeSelector (which is in fact a short version of the nodeAffinity) and I redeployed from source using ko:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: kafka-ch-dispatcher
  namespace: knative-eventing
spec:
  template:
    spec:
      nodeSelector:
        bench-role: eventing
      containers:
        - name: dispatcher

Now the sender and receiver:


apiVersion: v1
kind: Pod
metadata:
  name: channel-perf-send
  namespace: perf-eventing
spec:
  nodeSelector:
    bench-role: sender
  containers:
    - name: sender

---

apiVersion: v1
kind: Pod
metadata:
  name: channel-perf-receive
  namespace: perf-eventing
spec:
  nodeSelector:
    bench-role: sender
  containers:
    - name: receiver

Step 3: Restart the system after each run

After pinning the workload to different nodes, I've ran again the tests:

	P99	P99.9	P99.99
Run 1	263.552250	268.646500	272.223000
Run 2	266.060133	280.979000	285.983000
Run 3	266.994500	282.858000	292.864000
Run 4	268.234000	297.516000	326.862000
Run 5	265.809929	281.717000	288.665000

As you may notice, the first four runs looks incrementally worse. This happens because every run depends on the SUTs states caused by the previous run. The Kafka cluster and/or the Kafka channel dispatcher could be in a degradated state before a new run begins and this obviously reduces the chances to have same results over multiple runs. All systems involved in the road from sender to receiver must be reset, so every run starts stressing the system under the same conditions, ensuring that the latency of a run doesn't depend on previous runs.

In my case just deleting all pods does the trick, since the Deployments spin up a new ones:

kubectl delete pods -n knative-eventing --all
kubectl delete kt --all-namespaces --all # Delete all KafkaTopics
kubectl delete pods -n kafka --all

kubectl wait pod -n knative-eventing --for=condition=Ready --all
kubectl wait pod -n kafka --for=condition=Ready --all

Step 4: Configure the resource limits

As explained at beginning of this post, Kubernetes is designed to scale up & down workloads. What if the scheduler decides to schedule up and down our benchmark resources while the test is running? The benchmark needs to have granted the resources it needs & these should not change while is running. To do so, resource request & limits must be configured the same for every test and SUT, like:

resources:
  requests:
    cpu: 16
    memory: "8Gi"
  limits:
    cpu: 16
    memory: "8Gi"

This leads Kubernetes to schedule pods with QoS class Guaranteed, so it can't scale up & down resources.

The nodes where I'm running the benchmarks are configured with AMD EPYC 7401P 24 cores CPUs (so 48 logical cores) and 24Gb of RAM. I've tried to match these limits as following:

Kafka has 16 cpus and 8Gi of memory, same for Zookeeper
Kafka channel dispatcher has 44 cpus and 22Gi of memory
Sender has 16 cpus and 8Gi of memory, same for receiver

The problem is, even if containers are configured with Guaranteed QoS, there are no guarantees that the workload is pinned and it has exclusive access to the cores. By default, even in Guaranteed QoS, Kubernetes can move the workload to different cores depending on whether the pod is throttled and which CPU cores are available at scheduling time. The Kube scheduler does it defining the CFS Quota for the running container, so it asks to the kernel scheduler to allocate a fixed time to such containers.

Luckily there is a way to force the CPU pinning, enabling the static CPU management. This can be done only configuring the Kubelet config file for each node. To do so:

If the node is already running and connected to the cluster, it must be drained using kubectl drain <node name> --delete-local-data --force --ignore-daemonsets
Kubelet config file should contain the following entry: cpuManagerPolicy: static
Kubernetes system containers should have statically assigned resources. To do it, Kubelet config file should contain a configuration like:

systemReserved:
  cpu: "1"
  memory: "1Gi"

Results

I've tried to ran the tests after all these tweaks:

	P99	P99.9	P99.99
Run 1	265.955238	271.344500	276.415000
Run 2	264.850000	271.462000	279.283000
Run 3	266.283643	291.772500	335.116000
Run 4	266.065179	272.497000	279.553000
Run 5	264.828300	271.254500	278.362000

This results looks far better! Now relative SD of P99 is down to 0.26% (0.7014114246) vs the initial 3.2%!

I still have some outliers at higher percentiles, but now the results looks more trusty than the previous 3.2% of relative SD.

To wrap up, I want to underline that these tweaks worked for me but they could not be enough for all benchmark configurations.

Get in touch with me if you have more tweaks to show, and stay tuned for more updates!

Debts Manager Tutorial Part 3: Vert.x Web API Contract & Service

2019-02-13T00:00:00Z

Today I'm going to bootstrap the project starting from the contract I have already created in the previous chapter. The aim of this chapter is to show you Vert.x Web, Vert.x Web API Contract and Vert.x Web API Service. The combination of these three packages provides all functionalities you need to create a REST API. Then I'm going to use pmlopes' vertx-starter to scaffold the project.

If you are new to Vert.x, before going further I strongly suggest you to read vertx-core documentation and vertx-web documentation

Vert.x Web, API Contract and API Services

Vert.x Web is a library built on top of Vert.x to create web applications. It provides high level features like routing, request body handling, authorization, etc. The core concept of Vert.x Web is the Router which an object that can route requests to one or more Routes based on a set of rules like HTTP method, HTTP path, accepted content types, etc. On each Route you can define one or more Handlers that contain the logic to process the request. When a Router receives a request it creates a RoutingContext object which has all methods to read the request, write the response, call the next Handler and fail the context. Each Handler that you register consumes the request's RoutingContext. Vert.x Web also provides some common handlers like the BodyHandler that parses the request body and the AuthHandler that manages authN/Z of the request

Vert.x Web API Contract generates a Router starting from an OpenAPI definition. Everything revolves around an object called RouterFactory: you create the contract, you specify to the RouterFactory what are the handlers for the defined operations and it generates the Vert.x Web Router for you. The RouterFactory does some magics behind the hood to provide you a Router that:

Parses and validates the incoming requests (all kind of parameters and form or json bodies)
Correctly routes the requests to the right operation handlers by generating the correct Route instances
Has the required security handlers mounted in the right Routes

Vert.x Web API Service is a code generator based on concept of Vert.x Service Proxy. An event bus service is a Java interface that helps you to define in a more natural way an event bus message consumer. This approach leads to different benefits like the straightforward process to test a message consumer like any other Java class. As every EB event consumer, a service can inhabit inside the same verticle or it can be deployed in another application somewhere else in your microservices network. You can use the Vert.x Web API Service in order to mix Contract Driven capabilites provided by Vert.x WAC and these event bus service features. When you use EB API Services, you don't put the business logic inside the Route handlers, but you can distribute it inside different services. Then by using Vert.x WAC, you can manage the linking between these services and the Router instance

Define the API Services

Now I need to define how to group the API Operations defined inside the contract into different API Services. An important thing to keep in mind about API Services is that, with a good design, you can turn each API Service into a microservice in a few minutes. Starting from this assumption, I want to group operations by different subdomains of my API. Debts Manager API handles users, transactions and status, so I'm going to organize operations to create a Users Service, a Transactions Service and a Status Service. Here is the mapping between services and operations:

Transactions Service	Users Service	Status Service
`getTransactions` `getTransaction` `createTransaction` `updateTransaction` `deleteTransaction`	`login` `register` `getConnectedUsers` `connectUser` `getUsers`	`getUserStatus`

I need to assign an event bus address to each service. The interesting fact is that you can deploy more than one service instance on an address and the event bus will manage the load balancing between these. The event bus address could be any string in any format, although this it makes sense to use a domain like format to identify it. My choice is:

TransactionService is available at transactions.debts_manager
UsersService is available at users.debts_manager
StatusService is available at status.debts_manager

Specify the services inside the spec

In order to make the RouterFactory been able to correcly link the Router to the services, it must know what are the services' addresses. To define these associations, you have a couple of different methods. I'm going to focus on the configuration based method: inside the OpenAPI document, for each operation, I define the related service event bus address; then just calling mountServicesFromExtensions(), the RouterFactory will inspect the OpenAPI document to find all those associations. E.G. the getTransaction definition now looks like this:

summary: 'Get a Transaction'
operationId: getTransaction
responses: ...
security: ...
x-vertx-event-bus: transactions.debts_manager

Look at x-vertx-event-bus documentation for more details.

Bootstrap the project

vertx-starter contains a collection of different project templates called presets. I'm going to focus on "OpenAPI Server with Event Bus" preset to scaffold debts manager.

To help the scaffolder generates the right service interfaces, you must define the service address - service name mapping. Just a couple of entries inside Debts manager API spec and you are ready to scaffold the project:

components:
  schemas: ...
  securitySchemes: ...
  x-vertx-service-gen:
    transactions.debts_manager: TransactionsService
    users.debts_manager: UsersService
    status.debts_manager: StatusService

Now open the vertx-starter page and with just a couple of clicks you have a zip with the project scaffolded!

Let's dig into the generated code:

The scaffolder created for you:

The POJOs which represents the API data models
The service interfaces
package-info.java files required to trigger Vert.x annotation processing
The stubs of service implementations
The stubs of service tests
The entrypoint of Vert.x application which is called MainVerticle
An ApiClient that can be used for testing purposes
The build tool configuration file (pom.xml in my case)

As every scaffolder, it is necessary to make some adjustments for the project needs.

Configure the project

I made a couple of changes to adapt the project skeleton to my needs. In particular I configured in my pom:

Logback logging
Compilation with Java 11 with maven-compiler-plugin
JUnit 5 with maven-surefire-plugin
vertx-maven-plugin to execute and package my Vert.x application

I also substituted the generated openapi.json with my original debts_manager_api.yaml to keep readibility of the spec document. The generated one is a bundled version with all $refs solved. If you need to bundle again your spec in future, I suggest you to use Speccy which is a very good tool that can also convert Json Schema to OpenAPI Schema while bundling the spec.

`vertx-maven-plugin` and application properties

Vert.x Maven plugin provides a couple of facilities to help you execute and package the Vert.x application. The usage is very simple: you must add it to your build plugins and then you must define the FQCN of the Verticle to run:

<properties>
  <vertx.verticle>io.slinkydeveloper.debtsmanager.MainVerticle</vertx.verticle>
</properties>

<plugin>
  <groupId>io.reactiverse</groupId>
  <artifactId>vertx-maven-plugin</artifactId>
  <version>1.0.18</version>
  <executions>
    <execution>
      <id>vmp</id>
      <goals>
        <goal>initialize</goal>
        <goal>package</goal>
      </goals>
    </execution>
  </executions>
  <configuration>
    <redeploy>true</redeploy>
    <config>test_config.json</config>
  </configuration>
</plugin>

test_config.json is a JSON containing application properties like PostgreSQL and Redis connection parameters. Vert.x Core includes this basic support to json configuration files: you can load it into your verticle with Vert.x command line, DeploymentConfig or directly with vertx-maven-plugin. If you want to configure a more complex properties management, there is a package called vertx-config that enables you to load HOCON configuration files, load configuration from a remote server, etc.

`MainVerticle`

The generated MainVerticle contains two methods:

startHttpServer() to create the RouterFactory, define the various handlers, generate the Router instance and start the HTTP server
startServices() to instantiate and mount the event bus services

As I said before, the HTTP Server and the corresponding Router don't depend on the event bus services, so you can move these two methods into two separate verticles. For simplicity, I'm going to mantain everything inside one verticle. In next chapters, we will spend time into splitting the verticle.

To mount a service to the event bus I use an helper object called ServiceBinder that lookups for the generated message handler and binds the service instance to the event bus:

TransactionsService transactionsService = TransactionsService.create(vertx);
serviceBinder
  .setAddress("transactions.debts_manager")
  .register(TransactionsService.class, transactionsService);

As I previously said, the RouterFactory can lookup into the OpenAPI document for the associations between services and operations with mountServicesFromExtensions(). This makes the code of startHttpServer() quite simple for the moment:

private Future<Void> startHttpServer() {
  Future<Void> future = Future.future();
  OpenAPI3RouterFactory.create(this.vertx, "openapi.yaml", openAPI3RouterFactoryAsyncResult -> {
    if (openAPI3RouterFactoryAsyncResult.succeeded()) {
      OpenAPI3RouterFactory routerFactory = openAPI3RouterFactoryAsyncResult.result();

      // Mount services on event bus based on extensions
      routerFactory.mountServicesFromExtensions();

      // Add security handlers
      routerFactory.addSecurityHandler("loggedUserToken", new LoggedUserTokenHandler());

      // Generate the router
      Router router = routerFactory.getRouter();

      // Start the HTTP Server and bind Router
      server = vertx.createHttpServer(new HttpServerOptions().setPort(8080).setHost("localhost"));
      server.requestHandler(router).listen();
      future.complete();
    } else {
      // Something went wrong during router factory initialization
      future.fail(openAPI3RouterFactoryAsyncResult.cause());
    }
  });
  return future;
}

Configure JWT AuthN/Z

Vert.x Web already provides a good support for JWT thanks to Vert.x Auth JWT, so I don't need to write an handler that manages the AuthN/Z.

To get JWT running you need an RSA key pair to sign your tokens. I opted for the JWK standard to store it and I generated the key pair and the key store using mkjwk.org.

Vert.x Auth JWT provides JWTAuth auth provider, which is the object that can authenticate, authorize and generate tokens. Vert.x Web has an handler called JWTAuthHandler that uses this auth provider to validate and extract the payload of the token of incoming requests. I modify the start() method of MainVerticle to load the jwk from filesystem and create the JWTAuth:

loadResource(jwkPath).setHandler(ar -> {
  if (ar.failed()) future.fail(ar.cause());
  else {
    JsonObject jwkObject = ar.result().toJsonObject();
    JWTAuth auth = JWTAuth.create(vertx, new JWTAuthOptions().addJwk(jwkObject));
    startServices();
    startHttpServer(auth).setHandler(future.completer());
  }
});

And of course I modify the startHttpServer() to use JWTAuthHandler:

routerFactory.addSecurityHandler("loggedUserToken", JWTAuthHandler.create(auth));

In next chapters you will see how to create a token during the login process.

Conclusion

The application is bootstrapped, now we are ready to deep down into application logic! In next two chapters I'm going to show you how I have implemented the persistence layer and the event sourcing layer.

Stay tuned for more updates!

Debts Manager Tutorial Part 2: Contract Design

2019-01-20T00:00:00Z

Hi guys! Welcome back to this tutorial!

In this second chapter of Debts Manager Tutorial I would like to show you how I have designed the REST API of Debts Manager. I'm going to follow the API First approach, documenting all aspects of the API Design with OpenAPI 3.

This post doesn't aim to provide you a full guide of how to design REST APIs: if you want more resources to learn it, look at the end of this post

Analysis

The REST APIs, in contrast with RPC, are driven by the data the services wants to expose. In the previous chapter I gave you an idea of the entities we must expose. Now I tabulate these and the relative operations on it.

Entity	Create	Retrieve	Update	Delete
User	✔️	✔️	❌	❌
User relationship	✔️	✔️	❌	❌
Transaction	✔️	✔️	✔️	✔️
Status	❌	✔️	❌	❌

This table is a pretty good starting point, but I must refine the analysis enforcing our methods with policies and logics.

These policies are primarly based on who is making the request. I'm going to define a login phase together with JWT to provide authorization and authentication. Each endpoint, except login and register, is secured with a JWT auth. My objective is expose, for each user, only a subset of data relative to the user itself.

Models

Before defining the endpoints I must formally describe the data models representing the service entities. OpenAPI has its own Json Schema dialect to define models: OpenAPI Schema. This is an extended subset of Json Schema Draft 5. Meanwhile I'm writing, there is a proposal to allow usage of every version of Json Schema, including the newer versions, with an extension https://github.com/OAI/OpenAPI-Specification/issues/1532.

I place these schemas in main OpenAPI file under components and schemas keywords. I can refeer to it using Json schema references ($ref keyword).

The simplest model here is the user. I want to expose only the username, so I represent it with a simple string. This is the definition using OpenAPI Schema:

Username:
  minLength: 5
  type: string

The status is represented by a map with users as keys and total debts\credits as values. In OpenAPI Schema:

Status:
  description: 'Map with username as keys and debt as value'
  type: object
  additionalProperties:
    type: number

In JSON maps are usually represented or as json array of tuples key-value or as a json object. The json object is the natural way to represent it, but it has an important restriction: keys are strings. In my case I need to represent a map string → number, so json object representation fits good. The map values schema are defined using additionalProperties and, only with Json Schema Draft 7 or newer, keys schema are defined using propertyNames.

The main transaction model is described below:

Transaction:
  type: object
  properties:
    id:
      type: string
    from:
      $ref: '#/components/schemas/Username'
    to:
      $ref: '#/components/schemas/Username'
    at:
      description: "Insertion datetime"
      format: date-time
      type: string
    value:
      type: number
    description:
      minLength: 1
      type: string
  required:
    - from
    - to
    - id
    - description
    - value
    - at

$ref keyword points to the Username schema I defined before.

This model doesn't fit good for my usage, because for each Transaction endpoint I want to apply some policies. A very common example is the id field: when user inserts a new transaction I want to designate the database to fill the id value. When the user creates a new transaction it shouldn't add the id field: that means that I can't use the Transaction model to describe the "create transaction" request body. Let's look at all restrictions I want to apply on various transaction endpoints:

id and at are filled by the backend when user adds a new transaction and they are immutable from the API perspective
When user updates a transaction he can't update the from (sender) and to (receiver) fields
When user adds a new transaction he doesn't need to fill the from field because the backend fills it with the logged user

To apply these restrictions I create a new model for each endpoint. I'm going to refactor Transaction into 3 different models: UpdateTransaction, NewTransaction and Transaction.

These new models lead to a new problem: duplication of model fields definitions. Json schema solves the duplication with schema composition keywords: allOf, anyOf and oneOf. In particular I will use allOf to achieve inheritance of schemas.

This is the final result:

UpdateTransaction:
  type: object
  properties:
    value:
      type: number
    description:
      minLength: 1
      type: string
NewTransaction:
  allOf:
    - $ref: '#/components/schemas/UpdateTransaction'
    - properties:
        to:
          $ref: '#/components/schemas/Username'
      required:
        - to
        - description
        - value
      type: object
Transaction:
  allOf:
    - $ref: '#/components/schemas/NewTransaction'
    - required:
        - from
        - to
        - id
        - description
        - value
        - at
      type: object
      properties:
        from:
          $ref: '#/components/schemas/Username'
        id:
          type: string
        at:
          description: "Insertion datetime"
          format: date-time
          type: string

The schemas inheritance tree is UpdateTransaction←NewTransaction←Transaction

Endpoints

OpenAPI document structures the endpoint definitions as follow:

paths:
  /pathA: {}
  /pathB:
    get: {}
    post: {}
    put: {}
  /pathC/{paramA}: {}

OpenAPI path strings allow path parameters using {paramName} and doesn't require an explicit definition of query parameters.

In OpenAPI terminology an operation is an API endpoint identified by a path and an HTTP method. Every operation could be uniquely identified with an operationId. The OpenAPI Specification (OAS) documents this field as optional, but I strongly suggest to specify it if you don't want to see your tooling explode. Most code generation tooling asserts that operationId is present. If it's not present they try to infeer it from path and http method producing unexpected results.

For each operation we are going to define:

operationId
parameters (if any): List of header, path, query and cookie parameters
requestBody (if any): Content type and content schema of request bodies
responses: Status code with response content type and schemas

I also fill the security field for each operation to require a JWT token to execute it.

Transactions and Status

Let's start with transaction CRUDs:

Operation	`operationId`	CRUD	Path	HTTP Method
Create a new transaction	`createTransaction`	Create	`/transactions`	POST
Get a single transaction	`getTransaction`	Retrieve	`/transactions/{transactionId}`	GET
Get user related transactions	`getTransactions`	Retrieve multiple	`/transactions`	GET
Update a transaction	`updateTransaction`	Update	`/transactions/{transactionId}`	PUT
Delete a transaction	`deleteTransaction`	Delete	`/transactions/{transactionId}`	DELETE

In OpenAPI:

  /transactions:
    get:
      operationId: getTransactions
      responses:
        '200':
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: '#/components/schemas/Transaction'
        '401':
          description: 'Expired token'
      security:
        - loggedUserToken: []
    post:
      operationId: createTransaction
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/NewTransaction'
        required: true
      responses:
        '201':
          description: 'Successful response.'
        '401':
          description: 'Expired Token'
        '403':
          description: "Trying to create a transaction with receiver not connected to logged user"
      security:
        - loggedUserToken: []
  '/transactions/{transactionId}':
    get:
      operationId: getTransaction
      responses:
        '200':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Transaction'
        '401':
          description: 'Expired Token'
        '403':
          description: "Trying to get a transaction where `from` or `to` is not the logged user"
      security:
        - loggedUserToken: []
    put:
      operationId: updateTransaction
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/UpdateTransaction'
        required: true
      responses:
        '202':
          description: 'Successful response.'
        '401':
          description: 'Expired Token'
        '403':
          description: 'Trying to update a transaction where `from` is not the logged user'
      security:
        - loggedUserToken: []
    delete:
      operationId: deleteTransaction
      responses:
        '204':
          description: 'Successful response.'
        '401':
          description: 'Expired Token '
        '403':
          description: 'Trying to remove a transaction where `from` is not the logged user'
      security:
        - loggedUserToken: []
    parameters:
      - name: transactionId
        in: path
        required: true
        schema:
          type: string

Note that for all operations under /transactions/{transactionId} path I haven't redefined every time the parameter transactionId: I have defined once at path level.

Status has only the retrieve operation, but I want to let user customize the output based on transactions insertion datetime: clients can use query parameter till to ask the status till the date time provided, excluding newer transactions. You can use it to throw back in your house mate face that he didn't pay the bills for a quite long time.

/status:
  get:
    operationId: getUserStatus
    parameters:
      - name: till
        in: query
        required: false
        schema:
          type: string
          format: 'date-time'
    responses:
      '200':
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/Status'
      '401':
        description: 'Expired token'
    security:
      - loggedUserToken: []

User and User relationships

The service supports creation and retrieval of users and user relationships. For simplicity I avoided to include U and D operations for user and user relationships.

I want to expose an endpoint to retrieve all registered users and an endpoint to retrieve only users that have a relationship with logged user:

/users:
  get:
    operationId: getUsers
    parameters:
      - name: filter
        required: true
        in: query
        schema:
          type: string
    responses:
      '200':
        content:
          application/json:
            schema:
              type: array
              items:
                $ref: '#/components/schemas/Username'
      '401':
        description: 'Expired token'
    security:
      - loggedUserToken: []
/users/connected:
  get:
    operationId: getConnectedUsers
    responses:
      '200':
        content:
          application/json:
            schema:
              type: object
              properties:
                allowedTo:
                  description: "Users that logged user can bill"
                  type: array
                  items:
                    $ref: '#/components/schemas/Username'
                allowedFrom:
                  description: "Users that can bill the logged user"
                  type: array
                  items:
                    $ref: '#/components/schemas/Username'
      '401':
        description: 'Expired token'
    security:
      - loggedUserToken: []

In getUsers we defined a very basic search functionality with the optional query parameter filter

In getConnectedUser I prefeered to define the request schema directly inside the request body definition because It's a schema strictly related to this operation and It isn't parent of any other schema.

This is the endpoint to create a user connection (user relationship):

/users/connected/{userToConnect}:
  post:
    operationId: connectUser
    parameters:
      - name: userToConnect
        required: true
        in: path
        schema:
          $ref: '#/components/schemas/Username'
    responses:
      '201':
        description: 'User connected'
      '401':
        description: 'Expired token'
    security:
      - loggedUserToken: []

When an user wants to start using this API he must authenticate with his credentials following this process:

User calls the /login endpoint passing his credentials in the request body
The backend checks if credentials are correct
The backend writes the response with a JWT token containing the username inside the payload
User stores the received JWT token

For each request the server must authorize the user. The user must include inside each request the header Authorization: Bearer <jwt token>. When the backend receives the request it checks the signature validity and the token expiration time. If the token is valid It parses the payload, where It can read the username of the logged user.

This is the login operation definition:

/login:
  post:
    operationId: login
    requestBody:
      content:
        application/json:
          schema:
            required:
              - username
              - password
            type: object
            properties:
              username:
                $ref: '#/components/schemas/Username'
              password:
                type: string
      required: true
    responses:
      '200':
        description: 'Returns the JWT token'
        content:
          text/plain: {}
      '400':
        description: 'Wrong username or password'

The register operation creates a new user and logins it:

/register:
  post:
    operationId: register
    requestBody:
      content:
        application/json:
          schema:
            required:
              - username
              - password
            type: object
            properties:
              username:
                $ref: '#/components/schemas/Username'
              password:
                type: string
      required: true
    responses:
      '200':
        description: 'Returns the JWT Token'
        content:
          text/plain: {}
      '400':
        description: 'Username already exists'

I don't cover in this tutorial the logout process, but I want to give you a tip: create a whitelist or blacklist of tokens.

As you already saw, each secured operation has the security field:

security:
  - loggedUserToken: []

The security field is called security requirement and it tells the user that he needs loggedUserToken security schema to access to this endpoint. Security schemas must be defined under #/components/securitySchemes:

securitySchemes:
  loggedUserToken:
    type: http
    scheme: bearer

Some resources to learn Web API Design and OpenAPI

I give you a couple of useful links:

Rest API Tutorial: Very simple and coincise tutorial for newbies of REST APIs world
API Stylebook: Collection of API styleguides from different IT companies
OpenAPI Specification repository: Contains the spec and examples
OpenAPI Directory: Collection of OpenAPIs of different public APIs

Conclusion

You can find the complete OpenAPI definition here: /src/main/resources/debts_manager_api.yaml

After you learnt how to design a REST API, approacching to OpenAPI is very simple. The operation definition is very intuitive because of 1:1 mapping with HTTP (methods, parameters, status codes, content types and so on). The tricky and magic part, for me, is definining and organizing the JSON Schemas. When you define simple models, you tend to put everything inside the same file. But when you raise the complexity using composed schemas, you get flooded by smaller and unclear schemas. My suggestion for you is to document the schemas with title and description keywords and organize these in multiple files.

In next chapter I'm going to bootstrap the project and start writing first Vert.x code, stay tuned!

Debts Manager Tutorial Part 1: Introduction

2018-12-13T00:00:00Z

Some months ago I decided to create a complete Vert.x application to show you capabilities of Vert.x for building Web APIs and, at the same time, I wanted to try some patterns I never used or applied. I'm going to create a production ready application to finally manage the debts with my house mate with a fully powered Vert.x application!

Some notes before starting: I'm going to make this guide as complete as possible, but keep in mind that this is a side project and It could contain bugs and It could be incomplete. I will try to cover all interesting aspects about API design, implementation, testing and I will show you how I implemented Event Sourcing and CQRS. I don't plan to write a frontend for it (I don't want to hurt your eyes), but if you want to help me I'm glad to accept it!

The code is already available on GitHub but It could change while I'm writing the guide.

What Debts Manager should do

The purpose of Debts Manager is to manage the debts between two users of the service. The idea is similar to Splitwise, but it will support only bills between two users. Every user should be registered to use the application. Then, if you want to receive bills from another user, you must connect to that user. When you are connected, you can bill him creating a transaction. For example:

User A registers to the platform
User B registers to the platform
User B allows user A to bill himself. It does connecting to user A
User A bills user B of 5 Euros for last grocery shopping

The final result is: user B now has a debt of 5 Euros with user A. Debts Manager will show to both users their status with various debts/credits

The connection between users are unidirectional, which means that if users want to bill each other they must create two diffent connections. There is no group concept, I wanted to keep things as simple as possible.

Design

Before going further I want to show you a couple of things of the overall design of the application. These are required to undestand various aspects of the tutorial.

Persistence & Event Sourcing

For persistence I choose PostgreSQL to store my data. The application stores into the database:

The users instances (user)
The connections between users (user relationship)
The bills (transaction)

The DB access is provided by the blazing fast reactive-pg-client library

The application stores the transactions between users (events). You can use it as a log of various bills, but you also want to look at a summary of various credits/debits between connected users. To build it, I aggregate the various transactions into one single structure that I call status. Every user has a status and is represented as a map with users as keys and total credits/debits as values. This map is built incrementally every time a user adds/modifies/removes a transaction and is stored in a Redis cache.

Web API

The application exposes a Web REST API that you can interact with. It is documented with an OpenAPI 3 file and exposes most of CRUD endpoints for users, user connections and transactions (some are missing to keep things simple). It also has an endpoint to access status of users. The endpoints are protected with JWT tokens, so to use the application you must complete a login request and you get a token to use for the following requests. The Web API is implemented using vertx-web, vertx-web-api-contract and vertx-web-api-service.

Testing

Okay I admit it, I'm lazy 😄 I tested only the minimum features! I built these tests primarly to show you how I faced and solved common async test problems. I used Junit5 together with vertx-junit5 and testcontainers to spin up Redis and PostgreSQL.

Tutorial parts

Contract design: Design the OpenAPI 3 contract
Vert.x Web API Contract & Service: Setup Vert.x project and bind Vert.x Event Bus services
Persistence: Design and implement persistence
Event Sourcing: Develop the read model and CQRS
Implement services logic: Implement the services
Testing: Spin up test containers and write clean assertions
BONUS: Deploy to OpenShift
BONUS: Refactor to microservices using Vert.x Event Bus

Stay tuned for next chapter! And give me feedback about this tutorial!

Blog Title

Designing APIs for vibe coding

Before vibe coding

What changed with vibe coding

Why is the vibe-ability so low?

How do I increase vibe-ability?

Rust macros are not just about DRY

The argument against macros

Macros are not just about DRY

Use with moderation

JUnit 5, Parallel tests, Extensions and ThreadLocal

Our test codebase

Goals

Parallel tests

Integration tests in parallel

Porting the test suite to JUnit 5

Fixing the MiniCluster extension first

Back on @ParametrizedTest

Results and conclusion

Vert.x container images with jlink

Fat-jar

Jdeps

Zero days since it was DNS

Create the JDK

Complete script and Dockerfile

Conclusions

Kubernetes controllers - The Empire Strikes Back

The kube-watch-abi ABI

Controller

The host

Rust module with async-await

Callbacks are bad!

How async/await works

The controller lifecycle

Our custom Future/Stream

Generalizing on_event to wakeup_future/wakeup_stream

The complete flow

More async ABI methods!

Redesigning the host as an event-driven application

Compiling kube-runtime to WASM

Show me the code!

Next steps

Kubernetes Controllers - A new hope

Introduction

Kubernetes controllers today

Extending Kubernetes APIs in process

Isolated, multi-language, lightweight plugin system? That sounds hard...

Show me your ABI!

WASI

Designing the ABI

Our prototype

The ABI

The controller

Next steps

Wait, where is Golang?

There is still a long road ahead

Reproducibility of benchmarks in Kubernetes

Reproducibility: What and Why

System under test: Knative Eventing

Test & Cluster

Step 0: How to determine reproducibility?

Step 1: Configure the test to don't blow up the system

Step 2: Pin containers to nodes

Step 3: Restart the system after each run

Step 4: Configure the resource limits

Results

Debts Manager Tutorial Part 3: Vert.x Web API Contract & Service

Vert.x Web, API Contract and API Services

Define the API Services

Specify the services inside the spec

Bootstrap the project

Configure the project

vertx-maven-plugin and application properties

MainVerticle

Configure JWT AuthN/Z

Conclusion

Debts Manager Tutorial Part 2: Contract Design

Analysis

Models

Endpoints

Fixing the `MiniCluster` extension first

Back on `@ParametrizedTest`

The `kube-watch-abi` ABI

How `async`/`await` works

Our custom `Future`/`Stream`

Generalizing `on_event` to `wakeup_future`/`wakeup_stream`

`vertx-maven-plugin` and application properties

`MainVerticle`