A passing test only supports a fix when the “before” build can still expose the bug. In a September 30, 2026 DEV Community post, ROSH Company Labs describes reviewing a fix for an AG-UI stream-completion bug and then discovering that the two installable artifacts being compared carried package versions 0.0.58 and 1.0.1—a major-release gap, not merely the small source change under review. Those artifact details and test results are the author’s account, not an independent reproduction of the builds.
The bug was about mistaking a truncated stream for a completed run
AG-UI is an event-based protocol for connecting agents to user-facing applications, according to its project repository. In issue #2300, the reported failure occurs when a stream ends without a terminal RUN_FINISHED or RUN_ERROR event. Instead of treating the missing terminal event as an error, a client could resolve the run as successful and leave partial assistant output looking complete.
The issue page says terminal events are mandatory and includes a reproduction. That establishes the motivating behavior; it does not independently verify the later comparison of pull-request artifacts described in the DEV post.
Why the first passing test did not establish that the fix worked
ROSH Company Labs says it initially tried the reproduction against the published client. That client did not contain the assertion added in the open pull request. Both cases passed, but that outcome could not show the assertion was effective: the tested client was not the relevant “before” control for that change.
#1 Best Overall
The useful control question is: Can the side that is supposed to fail actually fail? If the control lacks the behavior or check whose effect is being assessed, a green result is uninformative. As the post puts it, “A before that cannot fail tells you nothing about an after that passes.”
The installed artifacts reportedly differed by more than the patch
The author then tested artifacts associated with two pull-request commits. In the post’s account, the older artifact failed the resend-during-teardown case, while the newer artifact passed. On examining what was installed, the author reported a much broader difference than the source change being reviewed:
| Reported check | Older artifact | Newer artifact |
|---|---|---|
| Package version | 0.0.58 |
1.0.1 |
dist/index.js size |
65,523 B | 82,982 B |
| Commit spacing | 21 days between the compared commits | |
These are measurements and outcomes reported by ROSH Company Labs in 2026, not independently reproduced measurements. The post also reports differences in dependencies and bundled output. The project’s release history shows dated releases and package versions, so version comparisons need to be read in their release context. A version label or bundle-size difference alone does not identify which source changes caused the difference.
The central distinction is between a source commit and a built, installable artifact. A commit diff describes source changes; it does not prove that two CI artifacts were built from only those changes. The installed package, its dependencies, and its generated output are what the test actually exercised.
The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →How to make a before-and-after test meaningful
- Confirm the control can reproduce the defect. Run the failing scenario against an artifact that contains the relevant pre-fix behavior. If it passes, investigate the reproduction or the chosen artifact before interpreting the “after.”
- Check both artifacts contain the intended test mechanism. Verify that the assertion or other relevant change exists where expected; do not assume a published package or pull-request build includes it.
- Identify what was actually installed. Record the artifact source and package version, and inspect dependency and built-output differences when the comparison is meant to isolate a small patch.
- Explain the result, not just its direction. Ask whether the observed behavior arrived for the reason the test was designed to check. A result that matches expectations can still come from a setup mismatch.
ROSH Company Labs summarizes the last check as, “Did the result arrive the way you expected?” The post’s practical warning is to slow down and verify setup even when the output looks like the anticipated pass or failure.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What this case does—and does not—show
This episode is a specific warning about test validity and artifact identity, not evidence that CI comparisons generally cross major versions or that a particular build system is unreliable. The post attributes the version, dependency, bundle, and test-result details to its own investigation; the available issue and repository pages establish the bug context and project, but do not independently corroborate that artifact comparison.
The lesson is narrow but consequential: before treating a green “after” as evidence for a fix, establish that the control could fail and that the two installed artifacts represent the states you intended to compare.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
