How to create a Spring Cloud Stream Binder from scratch

Spring Cloud Stream is a framework built on top of Spring Boot and Spring Integration, that is designed to build event-driven microservices communicating via one or more shared messaging systems.

The core Spring Cloud Stream component is called Binder”, a crucial abstraction that’s already been implemented for the most common messaging systems (eg. Apache Kafka, Kafka Streams, Google PubSub, RabbitMQ, Azure EventHub, Azure ServiceBus…).

In this article, we’ll see in detail how to develop a custom Spring Cloud Stream binder from scratch.

Introduction

The official Spring Cloud Stream documentation already provides a very basic explanation of how to implement your own Spring Cloud Stream binder.

Here’s a brief excerpt from it about the Binder Service Provider Interface that must be implemented in order to create a custom binder:

The Binder SPI consists of a number of interfaces, out-of-the box utility classes, and discovery strategies that provide a pluggable mechanism for connecting to external middleware.

The key point of the SPI is the Binder interface, which is a strategy for connecting inputs and outputs to external middleware. The following listing shows the definition of the Binder interface:

public interface Binder<T, C extends ConsumerProperties, P extends ProducerProperties> {
Binding<T> bindConsumer(String name, String group, T inboundBindTarget, C consumerProperties);
Binding<T> bindProducer(String name, T outboundBindTarget, P producerProperties);
}

And here’s one more documentation snippet that’s basically a micro-tutorial about developing Spring Cloud Stream binders:

A typical binder implementation consists of the following:

A class that implements the Binder interface;

A Spring @Configuration class that creates a bean of type Binder along with the middleware connection infrastructure;

A META-INF/spring.binders file found on the classpath containing one or more binder definitions, as shown in the following example:

kafka:\ org.springframework.cloud.stream.binder.kafka.config.KafkaBinderConfiguration

Even though the above documentation is quite helpful to get started, it would definitely help to have a more thoroughly detailed guide and a practical example go along with it. Hint: that’s where we’re heading!

TL;DR (aka I got no time for this, just gimme the code)

If you got no time or will to go through the detailed step by step tutorial, you can jump to my demo on GitHub, which includes a custom file-based Spring Cloud Stream binder like the one shown below and a sample application that depends on it.

Developing the Custom Binder

Let’s get our hands dirty by developing a custom Spring Cloud Stream binder that consumes events by reading files and produces events by writing to files!

Create a new Maven project with a pom.xml file similar to the following, which includes the dependency to Spring Cloud Stream:

Technically, we just need to provide our org.springframework.cloud.stream.binder.Binder implementation, but practically the binder depends on 2 more components that we need to provide first: the “ProvisioningProvider” and the “MessageProducer”.

The ProvisioningProvider is responsible for the provisioning of consumer and producer destinations, and it is particularly useful to convert the logical destinations included in the application.yml or application.properties file in physical destination references (you could look Spring beans up by destination name, or simply trim the destination name as we do in the following snippet):

The MessageProducer — unlike the name suggests — is responsible for consuming the events and handling them as messages to the client application that is configured to consume such events.

Here is an example of MessageProducer implementation that polls on a file that matches the trimmed destination name and is located in the project path, while also archiving read messages and discarding consequent identical messages:

Finally, with all the required components in place, we can implement our own Binder by extending the AbstractMessageChannelBinder class, providing the required constructors and overriding the inherited abstract methods:

As you can see the binder implementation relies on: our ProvisioningProvider, injected at construction time; our MessageProducer that is used when defining the consumer endpoints; and finally on a MessageHandler that provides the logic required to produce the message (in this case, the message production is equivalent to writing the payload on a file with the same name as the producer destination).

Last but not least, we need to provide the Spring Configuration for our binder as follows:

and the related src/main/resources/META-INF/spring.binders file with the binder name followed by the qualified name of the binder’s Spring Configuration:

myFileBinder:\ com.example.springcloudstreamcustombinder.config.FileMessageBinderConfiguration

Congratulations! Your custom binder implementation is now complete and can be installed in your local Maven repository by running mvn clean install

Testing the Custom Binder

Head over to start.spring.io and generate a Spring Boot 2.2.0 project with Cloud Stream as the only required dependency (or just click on this link instead, and generate the project from there).

Add your custom binder dependency to the pom.xml file, in the dependencies section:

<dependency>
<groupId>com.example</groupId>
<artifactId>spring-cloud-stream-custom-binder</artifactId>
<version>0.0.1-SNAPSHOT</version>
</dependency>

Replace the src/main/resources/application.properties file with the following application.yml file, which enables logging for all events that are managed by Spring Cloud Stream:

Replace the src/main/java/SpringCloudStreamCustomBinderDemo.java file contents with the following:

Finally, add a file named “input” to the main project directory and write something to it.

With this configuration in place, you can now launch the application by running mvn spring-boot:run, relying on the custom Spring Cloud Stream binder we just implemented to: keep consuming events from the “input” file you just created; write the processing results to an output file named “output”; keep track of all previously read messages in the “archive.txt” file.

Conclusion

The official Spring Cloud Stream reference documentation is quite helpful when it comes down to implementing your own binder but it’s definitely not enough.

That being said, creating your own binder implementation is almost effortless even though it might appear like a hard task at first; also, knowing how to build a custom Spring Cloud Stream binder makes for a quite niche and useful skill in the development field!

Moreover, knowing how Spring Cloud Stream binders work makes it easy to customize already existing ones, and not just building new binders (which, in all honesty, might be a less common scenario given most use cases).

References & Useful Resources

If you liked what you just read, please remember to follow me on Medium and Twitter for more articles like this. I’d really appreciate it ;-)

Software Developer @ https://www.linkedin.com/in/dsibilio/ — If my articles helped you, consider buying me a coffee :) https://www.buymeacoffee.com/dsibilio

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store