A developer’s offline-message feature passed a reported 216 tests, yet failed in real use because its asynchronous, multi-device delivery flow broke across component boundaries. In a first-person DEV Community post, author lucifer911 describes four problems: messages arriving before keys were restored, premature acknowledgements, a cleanup path that missed live delivery, and conversation keys surviving chat deletion. The account is a useful case study in what tests can miss—not evidence that any one test layer is useless.
What the test suite covered—and what happened in use
The author reports that the suite included storage unit tests, integration tests against a real PostgreSQL database, and end-to-end tests over real WebSocket connections. The reported total was 216 passing tests. The author says a check using the deployed build and a second browser exposed the problems; the account does not publish the suite or implementation, and the count has not been independently audited. Read the author’s account on DEV Community.
As an Amazon Associate I earn from qualifying purchases.
The common thread is not that the individual parts necessarily failed their tests. It is that the full feature depended on timing, message state, persistence, and cleanup across a client and server. A test that verifies a component in isolation may not exercise the sequence in which those components interact.
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitches1. The socket opened before the decryption keys were ready
When the client opened its socket, the server began sending messages it had held for the recipient. Meanwhile, the client was restoring the conversation’s decryption keys asynchronously from browser storage. Messages could therefore arrive before the client was ready to process them. In the author’s tests, key loading was effectively instant, masking the timing window.
The reported fix was to restore saved state before connecting. More generally, a connection being open and an application being ready to handle incoming work are separate states. If setup must finish first, the client should not begin accepting that work until it does.
2. The client acknowledged messages before handling them
The client confirmed a message as soon as it arrived, and the server deleted its held copy after receiving that confirmation. But arrival did not guarantee that the client had successfully decrypted, stored, or displayed the message. If later handling failed, the server could already have discarded the only remaining copy.
The author changed the order so acknowledgement followed successful handling, while making an exception for messages the device could never read because its conversation keys were gone. As the author put it, “Arrival is not delivery.” The practical design question is what event makes deletion safe: a socket receiving bytes, or the application reaching a state from which the message can be recovered and used?
3. Live-delivered messages escaped the cleanup path
The server stored messages generally but relied on confirmation to remove them. That confirmation path did not run for live delivery, so copies of messages delivered while the recipient was online accumulated in server storage. A weekly sweep had been clearing the lingering copies.
The reported fix was to store a message only when its recipient was absent. The author’s observation—“A delete that only runs on one code path is not a delete”—points to a useful check for any retention design: trace both live and offline delivery all the way through storage and removal, then inspect what remains after each sequence.
4. Deleting a chat left its encryption keys behind
Removing a chat deleted its messages but not its keys. When the contact was added again, the client could use stale keys even though the other side had discarded the former conversation state. The resulting mismatch meant messages could not be decrypted.
Rank #4
The author’s fix was to remove the keys along with the deleted chat state. This is a dependency problem: deleting an owner object should also account for data that is meaningful only while that object exists. For a conversation, that includes its associated cryptographic state.
Free tools Windows power users keep installed
One-click scans. No signup required.
What this case suggests about testing asynchronous features
The author says the deployed-build check with a second browser took about ten minutes. That is the author’s reported figure, not an independently verified timing benchmark. The value of the check was the realistic sequence: another device, persistent browser state, a network connection, and the actual deployed build interacting together.
Best Value
That does not establish that all four bugs could only have been found after deployment. The startup race depended on realistic I/O timing in the author’s account. The other mechanisms—acknowledgement outcomes, residual database records, and orphaned keys—could potentially be caught by tests that assert state after those sequences. That is an inference from the reported failure modes, not an independent assessment of the project.
For similar features, useful verification includes:
- Model setup and readiness separately; test delayed storage restoration while incoming work becomes available.
- Assert that a message is retained until the chosen successful-handling condition is reached, including failure cases.
- Exercise both online and offline delivery, then inspect persisted records after each path.
- Delete a conversation and verify that its messages and dependent key state are removed together.
- Run a user-like, multi-device flow against the deployed build as an additional layer—not a substitute for unit, integration, or end-to-end tests.
The author’s closing formulation is apt for this particular account: “Tests tell you the parts work. They are much worse at telling you the whole thing does.”
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
