Free tools Windows power users keep installed
One-click scans. No signup required.
For a supported Apache SeaTunnel Kafka source running in streaming mode, set partition-discovery.interval-millis to a positive polling interval. SeaTunnel can then periodically discover partitions added while the job is running, without requiring a job restart for that discovery behavior. The documentation example uses 5000 milliseconds; it is an example, not a universal production recommendation. A newly discovered partition starts from start_mode unless checkpointed offsets already exist for it.
Enable periodic discovery on the Kafka source
Configure the option in the Kafka source block of the streaming job. Preserve your existing connection, topic, format, and security settings; the values below are illustrative, based on the Apache SeaTunnel Kafka source documentation.
As an Amazon Associate I earn from qualifying purchases.
env {
job.mode = "STREAMING"
checkpoint.interval = 5000
}
source {
Kafka {
topic = "seatunnel_topic"
bootstrap.servers = "localhost:9092"
consumer.group = "seatunnel_group"
start_mode = latest
partition-discovery.interval-millis = 5000
format = json
}
}The discovery interval controls how often the source checks for newly added partitions. The documented 5000 value is an example interval, not a recommendation for every workload. Choose an interval that fits your operational needs and verify the setting against the documentation for your deployed SeaTunnel version.
The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →What happens when a new partition is found
After discovery, the source’s starting offset for that partition depends on its offset state: it starts from the configured start_mode unless checkpointed offsets already exist for that partition. Discovery and offset selection are separate concerns. Configuring an interval does not itself create checkpointed offsets or guarantee a particular starting offset.
#1 Best Overall
Check version and job scope
The documented behavior applies to the Kafka source connector in long-running streaming mode. SeaTunnel’s connector history identifies dynamic discovery of topics and partitions in streaming mode as a change introduced in version 2.3.0. Versioned 2.3.8 and 2.3.13 documentation surfaced for this setting describes it as optional and shows a default of -1, while the current documentation example uses 5000. These references do not establish one default that applies to every release, so check the documentation matching your deployment before relying on a default or implementation detail.
Quick Recap
Rank #3
- This setting concerns source-side partition discovery; it does not configure Kafka sink behavior.
- The cited evidence does not establish identical behavior across every SeaTunnel release or execution engine.
- The dynamic-discovery guidance is for streaming jobs; do not assume it applies to bounded jobs.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
