You can exercise Kafka Streams topology logic inside a plain test process, with no broker, no cluster, and no deployment. Kafka’s TopologyTestDriver feeds records through your topology, captures the output, and lets you inspect state stores. It answers “does this logic produce the right results for these inputs?” It does not answer “how does this behave on a real cluster?” Keeping those two questions separate is the difference between a fast, reliable test suite and a false sense of safety.
What the driver does and does not replace
Apache Kafka’s TopologyTestDriver API documentation describes the class this way: “Best of all, the class works without a real Kafka broker, so the tests execute very quickly with very little overhead.” The driver can test a topology assembled either with the low-level Topology class or with the DSL’s StreamsBuilder. Internally it simulates Kafka consumers and producers, and its test input and output helpers convert ordinary Java objects to and from serialized bytes.
As an Amazon Associate I earn from qualifying purchases.
That makes it a strong tool for unit-level and topology-level verification: filters, joins, aggregations, custom Processor API logic, and state-dependent branching. It is a weak tool for anything that depends on how Kafka actually moves data. Use a broker-backed integration test when the question involves real cluster behavior, multiple partitions, deployment configuration, or interaction with brokers.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitchesChoosing the right test type
| Question you need answered | TopologyTestDriver | Broker-backed integration test |
|---|---|---|
| Does the topology transform and route records correctly? | Yes, quickly and deterministically | Yes, but slower to run |
| Does a real Kafka broker need to be running? | No | Yes |
| Is partition-sensitive or multi-partition behavior being tested? | No. Input topics are simulated as single-partitioned | Yes, when the test creates multi-partition topics |
| Are deployment settings and broker configuration under test? | Not really. Only the configuration you pass to the driver applies | Yes |
| Can state stores be read and pre-populated directly? | Yes | Possible, but usually indirect |
| Can event-time and wall-clock punctuation be controlled? | Yes, through record timestamps and an explicit mock wall clock | Only with real or approximated timing |
The practical rule: run driver tests on every build, and keep a smaller set of broker-backed tests for the behaviors the driver cannot model.
#1 Best Overall
Dependency setup
Add the kafka-streams-test-utils artifact as a test-scoped dependency. The Apache Kafka 3.8 documentation for Streams testing shows this pattern. Match the artifact version to the Kafka Streams version your application already uses. Do not copy an example version from a guide without checking it against your build.
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-streams-test-utils</artifactId>
<version>${kafka.version}</version>
<scope>test</scope>
</dependency>Define kafka.version once in your parent POM or version catalog, so the production Streams client and the test utilities cannot drift apart.
Writing a driver test, step by step
- Build the topology. With the DSL, construct it with
StreamsBuilderand callbuild(). With the Processor API, assemble theTopologydirectly. Either form works with the driver. - Create the driver with representative configuration. Pass a
Propertiesobject that includes at least an application ID, a bootstrap servers value (it is not contacted), and the default key and value serdes your topology relies on. If your logic reads timestamps, set the timestamp extractor too, usingStreamsConfig.DEFAULT_TIMESTAMP_EXTRACTOR_CLASS_CONFIG. A driver built with missing serde settings can fail in ways that look like topology bugs. - Create input and output helpers. Call
createInputTopicwith the topic name and serializers, andcreateOutputTopicwith the topic name and deserializers. The helper types areTestInputTopic<K, V>andTestOutputTopic<K, V>. - Pipe input and assert output. Call
pipeInputon the input helper, then read results withreadValue,readKeyValue, orreadRecordsToListon the output helper. - Close the driver. Release it in a
finallyblock or test teardown so resources are freed even when an assertion fails.
Topology topology = builder.build();
Properties props = new Properties();
props.put(StreamsConfig.APPLICATION_ID_CONFIG, "orders-test");
props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "unused:9092");
props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass());
props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.String().getClass());
TopologyTestDriver driver = new TopologyTestDriver(topology, props);
try {
TestInputTopic<String, String> in = driver.createInputTopic(
"orders", new StringSerializer(), new StringSerializer());
TestOutputTopic<String, String> out = driver.createOutputTopic(
"enriched", new StringDeserializer(), new StringDeserializer());
in.pipeInput("order-1", "widget");
assertEquals("widget-enriched", out.readValue());
} finally {
driver.close();
}The bootstrap address is never contacted. Any well-formed value works, but writing an obviously fake one makes it clear to future readers that no broker is involved.
Recommended Free Tools
Testing time-based logic
Time handling is where driver tests most often go wrong, because there are two different clocks.
Rank #3
Event-time punctuation
Calling pipeInput can trigger event-time punctuation. The driver advances stream time as records arrive, so you control progress by setting record timestamps. Supply timestamps that cross your punctuation interval, and the scheduled callback fires as it would in a running application. Verify the emitted results after the input that should trigger the callback.
Wall-clock punctuation
Wall-clock punctuation does not follow record timestamps. It requires you to advance the driver’s mocked wall-clock time explicitly, using advanceWallClockTime with a Duration. Until you do, a wall-clock schedule will not fire, which can look like a broken topology when the real cause is a test that never moved the clock.
Rank #4
- Metamorphosis: Franz Kafka (Little Clothbound Classics)
Inspecting and pre-populating state stores
The driver exposes the topology’s state stores, so you can check what a processor wrote and seed state before input is piped. Use getKeyValueStore or getWindowStore with the store name your topology declared. Seeding is useful when a test needs an existing aggregate or lookup row to exist before the first record arrives, and it avoids building a long warm-up sequence of inputs just to reach that state.
Free tools Windows power users keep installed
One-click scans. No signup required.
Limits that change what your tests prove
Input is processed synchronously
The current API documentation states that input processing is synchronous. The configuration settings commit.interval.ms and cache.max.bytes.buffering have no effect in the driver. Behavior is as if each input is committed and flushed immediately. A test cannot reproduce the timing of batched commits or cache-driven flushes, so a bug that appears only under those conditions will not appear in a driver test.
Best Value
Input topics are single-partitioned
Every input topic is simulated as a single partition. Any logic that depends on key distribution across partitions, co-partitioning of joined topics, or ordering behavior among multiple partitions needs a broker-backed test. Do not present a passing driver test as evidence that those behaviors are correct.
Driver tests assume your configuration is representative
The driver uses only the properties you give it. If production sets timestamp extractors, serdes, or state directory options differently, the test passes against a configuration that does not exist in production. Keep the test properties drawn from the same source as the application’s real configuration wherever practical.
Keeping the test and the application on the same version
Apache Kafka’s API reference is published for the current release line, and at the time of this review its reference was for Kafka 4.3.1. The Streams testing guide in the Kafka 3.8 documentation points readers to the latest documentation for current details. Treat both as references, and confirm class names and method signatures against the exact version your project compiles against. If your application runs on an older Kafka Streams release, the method names above should be checked in that version’s Javadoc before you rely on them.
A quick verification before writing a large suite: create a one-operator topology, run one record through the driver, and confirm the output. That single test exposes most setup problems, including missing serdes and a mismatched test-utils version, before they multiply across the suite.
Quick Recap
“
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
