Kafka Streams – Transformations Examples

Kafka Streams Transformations provide the ability to perform actions on Kafka Streams such as filtering and updating values in the stream.  Kafka Stream’s transformations contain operations such as filter, map, flatMap, etc. and have similarities to functional combinators found in languages such as Scala.  And, if you are coming from Spark, you will also notice […]

2 days ago More
 

Kafka Producer

Kafka Producers are one of the options to publish data events (messages) to Kafka topics.  Kafka Producers are custom coded in a variety of languages through the use of Kafka client libraries.  The Kafka Producer API allows messages to be sent to Kafka topics asynchronously, so they are built for speed, but also Kafka Producers have the ability […]

18 days ago More
 

Kafka Consumer

In this Kafka Consumer tutorial, we’re going to demonstrate how to develop and run an example of Kafka Consumer, so you can gain confidence to develop and deploy your own Kafka Consumer applications.  At the end of this Kafka Consumer tutorial, you’ll have both the source code and screencast of how to run and customize […]

19 days ago More
 

Kafka Streams – Why Should You Care?

Kafka Streams is another entry into the stream processing framework category with options to leverage from either Java or Scala.  In this post, we’ll describe what is Kafka Streams, features and benefits, when to consider, how-to Kafka Stream tutorials, and external references.  Ultimately, the goal of this post is to answer the question, why should you […]

23 days ago More
 

Kafka Streams Joins Examples

Performing Kafka Streams Joins presents interesting design options when implementing streaming architecture patterns. There are numerous applicable scenarios, but let’s consider an application might need to access multiple database tables or REST APIs in order to enrich a topic’s event record with context information. For example, perhaps we could augment records in a topic with sensor event […]

24 days ago More
 

Kafka Streams Testing with Scala Part 1

After experimenting with Kafka Streams with Scala in the first post [1], I started to wonder how one goes about Kafka Streams testing in Java or Scala.  How does one create and run automated tests for Kafka Streams applications?  In this post, I’ll describe what I’ve learned so far.  Also, if you have any questions or ideas […]

1 month ago More
 

Kafka Streams Tutorial with Scala for Beginners Example

If you’re new to Kafka Streams, here’s a Kafka Streams Tutorial with Scala tutorial which may help jumpstart your efforts.  My plan is to keep updating the sample project, so let me know if you would like to see anything in particular with Kafka Streams with Scala.  In this example, the intention is to 1) provide an SBT project you […]

1 month ago More