Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Scan for outdated or missing drivers - takes under a minute3Repair Windows errors before they cause bigger problemsSome links on this page are affiliate links: if you buy through them we may earn a commission, at no extra cost to you.
To stream an LLM response from Spring AI to a browser, use Spring WebFlux, call ChatClient‘s reactive stream().content() API, and expose the chunks as an endpoint producing text/event-stream. A typical controller returns Flux<ServerSentEvent<String>>, so the browser can render each piece as it arrives instead of waiting for the complete answer. Spring MVC applications can use SseEmitter instead, but WebFlux is the more direct fit for a reactive model stream.
This guide shows the complete WebFlux implementation, the richer chatResponse() option, browser clients, an MVC alternative, and the reliability and security details that determine whether token streaming works in production.
As an Amazon Associate I earn from qualifying purchases.
Recommended Free Tools
What does streaming an LLM response with Spring AI actually do?
Streaming changes the response from one completed HTTP body into a sequence of small events. Spring AI receives partial output from the configured model provider and exposes those pieces through a Reactor publisher. Your controller maps each piece to an SSE event, and the client appends the data as events arrive.
The essential reactive call is:
Flux<String> output = chatClient.prompt()
.user("Tell me a joke")
.stream()
.content();content() emits text chunks. If the client needs response metadata or generation information, use stream().chatResponse() and map each ChatResponse to a DTO or an SSE event. The abstraction is provider-portable: changing the model-provider configuration does not require rewriting the SSE controller.
#1 Best Overall
What do I need before implementing the endpoint?
The streaming example targets the Spring AI API documented for version 2.0.1. An older 1.1.8 reference also exists, so use the API that matches the Spring AI version declared by your project rather than mixing snippets from different releases.
Use Spring WebFlux for the reactive path
Add the WebFlux starter to a Maven application:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
</dependency>You also need a configured Spring AI model provider and a ChatClient.Builder bean. The provider-specific credentials and model properties belong in your normal Spring configuration; the controller can remain unchanged when the provider changes.
Choose the server programming model
- WebFlux: returns a reactive publisher directly and models backpressure and cancellation naturally.
- Spring MVC: fits an existing servlet application and uses
SseEmitterfor asynchronous sends.
Do not add WebFlux solely to an application that must remain servlet-only without checking its runtime and dependency choices. Conversely, do not collect a reactive stream into a single string when the goal is low-latency output.
How do I build a Spring AI SSE endpoint with WebFlux?
Declare an endpoint that produces MediaType.TEXT_EVENT_STREAM_VALUE, obtain the prompt, call the reactive ChatClient API, and map every text chunk to a named SSE event.
1. Create the controller
@RestController
@RequestMapping("/api/llm")
class LlmController {
private final ChatClient chatClient;
LlmController(ChatClient.Builder builder) {
this.chatClient = builder.build();
}
@GetMapping(value = "/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
Flux<ServerSentEvent<String>> stream(@RequestParam String prompt) {
return chatClient.prompt()
.user(prompt)
.stream()
.content()
.map(text -> ServerSentEvent.<String>builder()
.data(text)
.event("token")
.build());
}
}The Flux stays lazy: model work begins as the request is subscribed to, and each emitted value is written as an SSE event. The event name is optional, but naming it token gives browser clients an explicit event type instead of making them handle every message through the generic listener.
2. Start the application and call it
With the application running on its configured port, request the endpoint from a terminal with buffering disabled:
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchPC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11curl -N "http://localhost:8080/api/llm/stream?prompt=Tell%20me%20a%20short%20joke"The -N option asks curl not to buffer output, making each event visible as it arrives. The exact number and size of chunks depend on the model provider and its transport; a chunk is not guaranteed to equal one word, sentence, or model token.
3. Keep the response as a stream
Do not call operations that aggregate the publisher, such as collecting all values into a list or joining them into one string, before returning from the controller. Aggregation removes the incremental behavior the browser is expecting.
Should I use content() or chatResponse()?
Use content() when the browser only needs text. Use chatResponse() when your application must transmit metadata or generation information alongside the text.
Text-only events
The content() variant produces Flux<String>, which is the simplest shape for a chat window. The controller can send each value as an SSE event whose data is a text fragment.
Structured response events
The richer form is:
return chatClient.prompt()
.user(prompt)
.stream()
.chatResponse()
.map(response -> ServerSentEvent.<ChatResponse>builder()
.event("response")
.data(response)
.build());In a production API, expose a deliberately designed DTO rather than coupling browser code to every internal response field. Keep the event names and JSON shape stable so clients can evolve independently of the model provider. If the provider adds or omits metadata, the controller contract should still remain predictable.
How does the browser consume Spring AI SSE output?
For a GET endpoint, the browser’s native EventSource API is enough. It maintains the connection and invokes a callback whenever an event arrives.
const source = new EventSource(
'/api/llm/stream?prompt=' + encodeURIComponent(question)
);
source.addEventListener('token', event => {
appendText(event.data);
});
source.onerror = () => {
source.close();
};Use encodeURIComponent for the prompt because spaces, punctuation, and non-ASCII characters must be encoded in a query string. Append each received fragment to the current assistant message; replacing the entire message on every event can produce flicker and unnecessary rendering work.
When EventSource is not the right client
Native EventSource is GET-oriented and does not provide a general way to attach arbitrary authentication headers or send a request body. For a POST prompt, header-based authentication, or a larger structured request, use fetch() with a streaming reader, or submit the prompt to a job endpoint and have the client subscribe to a separate stream.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Whichever client you choose, define what a normal completion, transport failure, and model failure look like. A client should not assume that a closed TCP connection always means successful generation.
How do I stream from a Spring MVC application?
A servlet-based application can use Spring MVC’s SseEmitter. The controller returns the emitter immediately, sends chunks as they become available, and calls complete() when generation finishes.
@RestController
@RequestMapping("/api/llm")
class MvcLlmController {
private final ChatClient chatClient;
MvcLlmController(ChatClient.Builder builder) {
this.chatClient = builder.build();
}
@GetMapping(value = "/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
SseEmitter stream(@RequestParam String prompt) {
SseEmitter emitter = new SseEmitter();
chatClient.prompt()
.user(prompt)
.stream()
.content()
.subscribe(
text -> {
try {
emitter.send(SseEmitter.event()
.name("token")
.data(text));
} catch (IOException ex) {
emitter.completeWithError(ex);
}
},
emitter::completeWithError,
emitter::complete);
return emitter;
}
}The emitter approach is useful when the rest of the application already depends on servlet request handling. It requires careful lifecycle handling: send failures, upstream errors, and normal completion must each terminate the emitter. In a fully reactive application, returning the Flux directly avoids adapting a publisher into callback-style code.
What is the difference between WebFlux and MVC for LLM SSE?
Both can deliver incremental SSE events, but they differ in how the stream is represented and managed.
| Concern | WebFlux | Spring MVC |
|---|---|---|
| Returned type | Flux<ServerSentEvent<T>> or another reactive publisher |
SseEmitter from a servlet controller |
| Backpressure and cancellation | Modeled by the reactive chain; cancellation can propagate upstream when supported | Handled through emitter lifecycle and asynchronous servlet processing |
| Integration | Best when the application already uses the reactive stack | Convenient for an existing servlet application |
| Event serialization | Declarative mapping from publisher values to SSE events | Explicit send calls for each event |
| Operational behavior | Still depends on proxy buffering and idle timeouts | Has the same intermediary concerns, plus emitter timeout and lifecycle settings |
Neither model bypasses network infrastructure. A reverse proxy or load balancer that buffers the response can make a correctly implemented stream appear to arrive all at once.
How should cancellation and errors work?
Streaming does not eliminate failure; it moves failure into the middle of a response that may already contain bytes. Design both sides of the connection for that case.
Propagate browser disconnects
When a browser tab closes or the user cancels generation, stop unnecessary upstream model work where the provider and Reactor chain support cancellation. Returning the original reactive stream gives WebFlux a chance to propagate cancellation; wrapping it in an unbounded background task can continue consuming provider output after the client is gone.
Handle errors before and after commitment
If authentication, validation, or model setup fails before headers are committed, return an appropriate HTTP error. Once SSE bytes have been sent, changing the HTTP status is generally no longer possible. In that case, define an event-level error convention, send a final error event when possible, and have the browser stop rendering the partial answer and show a retry or failure state.
Free tools Windows power users keep installed
One-click scans. No signup required.
Complete normally
On successful generation, let the publisher complete or, if your client contract requires an explicit marker, emit a documented completion event before completion. Keep that marker distinct from text events so a client never appends it to the assistant’s message.
How do heartbeats keep long-lived SSE connections healthy?
A quiet model stream can look idle to a proxy, load balancer, or network device. Periodic heartbeat or no-op events provide traffic that helps detect a disconnected client and prevents an otherwise healthy connection from being treated as abandoned.
In WebFlux, merge or otherwise schedule a periodic heartbeat source with the model publisher, taking care not to let heartbeat values become visible assistant text. Another option is an SSE comment or a dedicated event type that the browser ignores. The interval must be shorter than the strictest idle timeout in your path, and the heartbeat source must be cancelled when the request ends.
Spring WebFlux documentation specifically describes periodic sends as a way to detect disconnected clients. Heartbeats are not a substitute for configuring sensible server, proxy, and client timeouts; they work together with those settings.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Which production settings and safeguards matter?
Most streaming failures occur outside the three-line ChatClient call. Use this checklist before exposing the endpoint publicly:
- Proxy buffering: disable or bypass response buffering for the SSE route so intermediaries forward events promptly instead of coalescing them.
- Idle timeouts: align application, proxy, load-balancer, and client timeouts with the maximum expected generation duration; use heartbeats for quiet periods.
- Cancellation: verify that disconnecting the browser stops avoidable provider work.
- Validation: enforce prompt-size limits before invoking the model. Large prompts increase cost, latency, and the chance of rejected requests.
- Authentication and authorization: protect the endpoint like any other model-invoking API. Check the caller before opening a long-lived connection.
- Rate limiting: limit concurrent streams and request frequency. A stream occupies resources for its entire lifetime, not just for the initial request.
- Logging: do not indiscriminately log prompts or generated text. They may contain confidential, personal, or regulated information.
- Event contracts: document event names, data types, completion behavior, and error behavior so browser and server releases remain compatible.
- Observability: record timing and outcome metrics without storing sensitive prompt or response contents by default.
How do I troubleshoot a stream that does not arrive incrementally?
Use the symptom to narrow the fault rather than changing the ChatClient call at random.
The response arrives only at the end
- Confirm the controller declares
produces = MediaType.TEXT_EVENT_STREAM_VALUE. - Confirm the return value is a live
Fluxor activeSseEmitter, not a collected string. - Test with
curl -Nto separate browser rendering from server behavior. - Inspect reverse-proxy and load-balancer buffering settings and idle timeouts.
The browser receives events but displays nothing
- Check that the listener uses the same event name the server sends, such as
token. - Verify the prompt is URL-encoded and that the request is not failing authentication or CORS checks.
- Log the raw SSE stream in a development environment to confirm that data fields are present.
The stream stops partway through
- Look for upstream model errors, client disconnects, emitter timeouts, and intermediary idle timeouts.
- Ensure the error callback completes the emitter or terminates the reactive request rather than leaving a half-open connection.
- Add a documented event-level error message for failures that happen after the first bytes have been sent.
Chunks are unexpectedly large or irregular
Chunk boundaries are transport and provider behavior, not a promise of one event per model token. Render fragments incrementally but treat them as arbitrary text pieces; do not build parsing logic that depends on a particular chunk size.
How do I keep the implementation portable across model providers?
Keep provider-specific settings in Spring configuration and keep the controller focused on the Spring AI ChatClient contract. The stream().content() and stream().chatResponse() methods provide the same application-facing shape while the configured provider changes underneath.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
If your browser contract uses a stable DTO, provider-specific metadata can be normalized at the mapping boundary. That prevents a provider switch from forcing changes to JavaScript event handling, authentication flow, or persistence code.
What should I test before shipping?
Test the behavior of the whole connection, not only whether the model eventually returns text.
- Start a request with a short prompt and verify that the first event arrives before generation completes.
- Confirm each event has the expected name and data encoding.
- Close the browser connection during generation and verify that cancellation reaches the reactive chain or emitter lifecycle.
- Force an authentication, validation, and provider error, checking both pre-commit HTTP errors and post-commit event-level errors.
- Leave a connection quiet long enough to exercise heartbeat behavior and intermediary idle limits.
- Run the route through the same proxy or load balancer used in production and verify that events are not buffered.
- Test prompts containing spaces, punctuation, Unicode, and the maximum accepted size.
Which Spring AI streaming design should I choose?
For a new reactive service, return Flux<ServerSentEvent<String>> from WebFlux and use content() unless the client needs metadata. For an established servlet application, use SseEmitter with explicit send, error, and completion handling. In both cases, keep the SSE event schema stable, preserve cancellation, and configure the network path so incremental writes remain incremental.
Frequently Asked Questions
Why does my SSE endpoint work locally but buffer in production?
A reverse proxy or load balancer may be buffering the response even though the application emits events correctly. Compare a direct application request with a request through each intermediary, then disable response buffering for the SSE route and verify its idle timeout is longer than the generation period or supported by heartbeats.
Can I send JSON objects instead of plain text chunks?
Yes. Use the chatResponse() stream or map your own response object to ServerSentEvent<YourDto>. Define a stable JSON envelope and event name so clients do not depend on provider-specific fields.
How should a client distinguish a failed generation from a normal disconnect?
Define an explicit error event for failures that occur after the response starts, and reserve normal stream completion for successful termination. The client can then show a retry state for an error event instead of silently treating every closed connection as success.
Is one SSE connection suitable for every chat interaction?
A connection is appropriate for one in-progress generation, but concurrent generations should be limited and tracked per user or session. For workflows that outlive a browser tab or need durable status, a POST-to-job design with a separate subscription stream is safer than keeping one request open indefinitely.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
