LLM Streaming with Embabel: How to Stream Text, Objects, and Tool Calls

Embabel can stream LLM output as it is generated, including raw text, thinking events, and generated objects. For tool-enabled interactions, its streaming tool loop also coordinates successive model calls and tool execution, so a streamed response can cover more than one inference turn.

What Embabel streaming can return

The Embabel Agent Framework User Guide 1.5.1 describes streaming as delivering LLM output gradually to an application. Depending on the approach, an application can receive:

#	Preview	Product	Price
1		The Agentic AI Bible: The Complete and Up-to-Date Guide to Design, Develop, and Scale Goal-Driven,...	$24.95	Buy on Amazon
2		System Design for the LLM Era: Patterns and principles for production-grade AI architecture	$33.24	Buy on Amazon
3		Hands-On LLM Serving and Optimization: Hosting LLMs at Scale	$60.29	Buy on Amazon
4		Building LLMs for Production: Enhancing LLM Abilities and Reliability with Prompting, Fine-Tuning,...	$47.49	Buy on Amazon
5		LLM Inference Engineering Handbook: Crush API Costs, Cut Latency and Build Reliable Production...	$37.99	Buy on Amazon

As an Amazon Associate I earn from qualifying purchases.

• Raw text: incremental text chunks, exposed as a Flux<String>.
• Thinking events: events representing the model’s thinking content.
• Generated objects: parsed, typed results emitted as events.

The guide identifies StreamingEvent, StreamingPromptRunnerBuilder, LlmMessageStreamer, StreamingToolLoop, and DefaultStreamingToolLoop as part of the streaming APIs. Reactive handlers such as doOnNext, doOnError, and doOnComplete let an application respond to events or chunks, handle failures, and observe completion.

How to create a raw-text stream

The current guide’s basic pattern builds a streaming prompt runner, supplies a prompt, and calls generateStream(). The resulting stream can be processed with Reactor callbacks:

#1 Best Overall

Sale

The Agentic AI Bible: The Complete and Up-to-Date Guide to Design, Develop, and Scale Goal-Driven, LLM-Powered Agents that Think, Execute and Evolve

$24.95

Buy on Amazon

Flux<String> stream = promptRunnerBuilder
    .streaming()
    .withPrompt(prompt)
    .generateStream();

stream
    .doOnNext(chunk -> handleChunk(chunk))
    .doOnError(error -> handleError(error))
    .doOnComplete(() -> handleComplete());

This illustrates the documented builder sequence and reactive handling pattern; it is not a complete application setup. Consult the Embabel Agent Framework User Guide for the full API and adapt the example to the dependency version in your project. Older documentation uses different method names, so do not mix snippets from different releases without checking compatibility.

How to stream generated objects

For typed output, the stream can contain events for both thinking content and parsed objects. Handle the event types separately so the application does not treat every event as the same kind of payload.

Rank #2

Sale

System Design for the LLM Era: Patterns and principles for production-grade AI architecture

$33.24

Buy on Amazon

Scalar strings need special care. The current guide warns that using String.class directly can cause a bare JSON string to be interpreted as thinking content by the structured streaming parser. Instead, wrap the scalar in a result type such as StringResult, which gives the model an object schema with a value property. The parser can then emit it as a structured object event.

This advice concerns the documented object-stream parsing path; it is separate from streaming ordinary raw text as Flux<String>.

Rank #3

Sale

Hands-On LLM Serving and Optimization: Hosting LLMs at Scale

$60.29

Buy on Amazon

How streaming works with tool calls

LlmMessageStreamer.streamInference streams one inference while advertising the tools available to the model. It does not execute requested tools by itself. Embabel’s streaming tool loop handles the broader cycle:

1. Stream an inference and assemble the assistant response.
2. Execute tools requested by the assistant.
3. Add tool results to the conversation history.
4. Start the next inference and continue streaming.

The returned stream can include content from each inference turn, including thinking content produced before or between tool calls. Available tools can also change between turns; the guide names ToolInjectionStrategy and UnfoldingToolInjectionStrategy as examples of strategies for updating them.

Rank #4

Sale

Building LLMs for Production: Enhancing LLM Abilities and Reliability with Prompting, Fine-Tuning, and RAG

$47.49

Buy on Amazon

The practical distinction is that streaming exposes intermediate output to the application while Embabel coordinates the tool-enabled interaction around it. A single streamed inference and a complete agent interaction are not the same thing.

What structured-output support does—and does not—mean

The current guide states that Spring AI does not currently support native structured output for streaming. That limitation applies to the native structured-output path; it does not mean Embabel lacks raw-text streaming or APIs for streaming generated objects. Check the behavior of the provider and versions in your application rather than assuming all combinations work alike.

Best Value

Sale

LLM Inference Engineering Handbook: Crush API Costs, Cut Latency and Build Reliable Production Systems — Real Benchmarks, Python Code and Complete Code Repository for Engineers at Scale

$37.99

Buy on Amazon

Which guide and API version should you follow?

The version 1.5.1 guide documents streaming as a developed capability. The feature’s support history is useful context, but not a compatibility guarantee: a project discussion recorded streaming support in a 0.3.1-SNAPSHOT build on December 18, 2025, and was closed with the feature marked implemented on August 10, 2026. The discussion also pointed to OpenAI, Anthropic, and Ollama integration-test examples; provider behavior still depends on the actual versions and setup.

An earlier 0.3.1 guide uses the same broad concepts but older spellings, including .withStreaming(). For implementation, use the documentation matching your selected Embabel release; the older guide is available at Embabel Agent Framework User Guide 0.3.1. The historical status discussion is at Embabel’s streaming support discussion.

When Embabel is useful compared with direct Spring AI use

Embabel builds on Spring AI and adds a higher-level layer for agent workflows, composable actions, orchestration, and testing. If an application only needs a straightforward model stream, direct Spring AI use may be sufficient. If it needs a coordinated tool loop or broader agent workflow structure, Embabel’s abstractions may be relevant. The choice is about orchestration needs; the available documentation does not establish blanket differences in speed, cost, or output quality. See the Embabel project repository for the framework overview.

Quick Recap

SaleBestseller No. 1

The Agentic AI Bible: The Complete and Up-to-Date Guide to Design, Develop, and Scale Goal-Driven, LLM-Powered Agents that Think, Execute and Evolve

$24.95

SaleBestseller No. 2

System Design for the LLM Era: Patterns and principles for production-grade AI architecture

$33.24

SaleBestseller No. 3

Hands-On LLM Serving and Optimization: Hosting LLMs at Scale

$60.29

SaleBestseller No. 4

Building LLMs for Production: Enhancing LLM Abilities and Reliability with Prompting, Fine-Tuning, and RAG

$47.49

SaleBestseller No. 5

LLM Inference Engineering Handbook: Crush API Costs, Cut Latency and Build Reliable Production Systems — Real Benchmarks, Python Code and Complete Code Repository for Engineers at Scale

$37.99

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.