A green test can be correct about its test environment and still say nothing reliable about a browser behavior that environment cannot model. That is what happened when a transcript-scrolling test passed in jsdom even though the desktop app failed to scroll to its newest message: the test needed layout measurements, but its environment did not provide layout.
How the scrolling test passed while the app was broken
Scott Mallinson describes a desktop-app transcript that was supposed to scroll to its newest message. The test suite already covered that behavior, but the test ran in jsdom, which supplies a DOM without browser layout. In Mallinson’s account, the relevant element heights were zero. The height comparison therefore made the content appear to fit, and the assertion that the transcript was at the bottom passed.
As an Amazon Associate I earn from qualifying purchases.
The test had not omitted scrolling coverage. It asked an environment without layout to answer a layout question, then treated the resulting value as meaningful. As Mallinson puts it, “The defect was mine: I asked it a layout question and believed the answer.” Read Mallinson’s account.
What a passing assertion actually establishes
An assertion is evidence about behavior as represented by the environment running the test. If that environment does not implement a property the behavior depends on, a plausible-looking return value can conceal the gap. Here, zero heights did not make the test fail; they made the faulty scroll calculation look consistent with the assertion.
#1 Best Overall
Adding more assertions to the same environment would not have supplied layout. The useful question is: “The better question is which of the properties you care about your test environment actually implements, and what it returns for the ones it doesn’t.”
Choose an environment that can answer the question
For the scrolling case, Mallinson moved the test to Playwright in a real browser. He stubbed the app’s IPC layer, so the backend did not need to run. That separated the behavior needing browser fidelity—layout and scrolling—from unrelated backend work.
| Test environment | What it can establish here | Backend requirement in Mallinson’s example | Trade-off |
|---|---|---|---|
| DOM environment such as jsdom | DOM-level behavior, but not layout-dependent scrolling; the reported element heights were zero. | No backend was needed for the DOM test. | Lightweight for DOM checks, but cannot validate behavior dependent on layout. |
| Playwright in a real browser | Browser layout and scrolling behavior. | Mallinson stubbed IPC, so the backend did not need to run. | More CI/runtime weight than a DOM-only test. |
| Shipping WebView | Would more closely match the app’s actual browser engine. | Not stated in Mallinson’s account. | Mallinson deferred this in favor of Chromium, while leaving open a change if another layout issue escaped. |
This is a fidelity-versus-cost decision, not a guarantee that Chromium will catch every WebView-specific issue. Use the least expensive environment that implements the properties the test needs; reserve closer-to-shipping checks for behaviors where engine differences matter.
Keep the tested operation narrow
Mallinson chose to set the transcript’s scrollTop directly rather than call scrollIntoView. His reason was that scrollIntoView can move scrollable ancestors, including the document, while setting the transcript’s scroll position targeted the container. This is an implementation choice for his case, not a rule that every interface should use direct scrollTop.
Rank #3
Check how styles interact with hidden state
A separate dashboard bug involved a tile marked hidden that remained visible. The component rule .tile { display: flex } overrode the browser’s built-in [hidden] { display: none } behavior because of CSS specificity. Mallinson fixed that combination with:
.tile[hidden] { display: none }When a component sets its own display value, test the state it is meant to suppress as well. A hidden attribute alone may not win against the component’s stylesheet.
Rank #4
Audit the properties your environment does not model
Layout is one possible gap, not the only one. For a test that depends on a particular browser or platform property, establish whether the environment implements it and what value it returns when it does not. Mallinson identifies these areas as worth auditing:
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →- Layout and element measurements
- Font rendering
- Timing and animation
- Real network behavior
- Pixel rendering
A missing capability is especially risky when it produces a normal-looking default rather than an explicit failure. Make the environment choice from the behavior’s dependencies, not from the fact that the test is labeled a UI test.
Decide whether the shipping WebView belongs in the test loop
Mallinson used Chromium through Playwright instead of testing against the app’s shipping WebView, weighing closer fidelity against CI cost. He described that as a judgment call and would reconsider if another layout issue escaped. For a team, the relevant trigger is whether a browser-engine difference could plausibly change the behavior being tested and whether the additional fidelity justifies the runtime and maintenance cost.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
