A test that fails intermittently is not automatically flaky. Check what happened in the failing run: an assertion or test-harness error, an unhandled language exception, or a native fault that terminated the app process. A later passing run does not erase crash evidence from the failed one.
What failed: the test, an exception, or the process?
Start with the failure mechanism, not how often it happens. Android Developers defines a crash as an unexpected exit caused by an unhandled exception or signal. That can include an unhandled Java or Kotlin exception, or a native signal such as SIGSEGV. The operating system terminates the app process. Android Developers describes crashes in those terms.
As an Amazon Associate I earn from qualifying purchases.
| Failure type | What may have happened | Evidence to look for |
|---|---|---|
| Assertion or test-harness failure | A check failed, or the test runner could not complete its work; the app process may still be alive. | Assertion output, test-runner error, or harness logs. |
| Unhandled language exception | An exception reached a point where the relevant runtime could not handle it. | Exception type and message, plus the stack trace or crash report. |
| Native signal or fatal runtime failure | A low-level fault terminated the app process, potentially outside a managed-language handler’s reach. | Signal or termination information and the native crash stack. |
This distinction matters because “intermittent” describes the observed pattern, not the cause. Timing, inputs, device state, or thread interactions can affect whether a fault appears, but a crash report from one run remains evidence of a process crash.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minuteWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallHow to classify a failure before calling it flaky
- Read the original test output. Look for an assertion or runner failure, and determine whether the test process reported that the app stopped unexpectedly.
- Check for crash artifacts from that same run. On Android, inspect the exception or signal and the crash stack trace. Android’s crash diagnosis guidance describes stack traces as the nested calls leading to a failure and notes that causes can range from invalid API arguments to multithreaded interactions.
- On Apple platforms, collect the appropriate diagnostic material. Xcode can obtain reports from App Store, TestFlight, and devices. Use a crash report to examine termination and thread activity; for an issue that is not a crash, consult operating system console logs. Apple explains how crash reports and device logs differ and how to acquire them.
- Locate the handler boundary. Decide whether the failure is a catchable exception in the relevant scope, an unhandled exception, or a fatal native signal. Identify which process and runtime produced it before assuming a handler could see it.
- Then investigate repeatability. Re-run or vary conditions to learn when the issue reproduces. Treat a subsequent pass as another observation—not as a reason to discard a crash report from the failing run.
Why try/catch may not catch the failure
A try/catch handler is not a universal recovery mechanism. It can handle exceptions only when they travel through a supported exception path and reach that handler. It cannot restore an app process after a fatal native signal has terminated it.
Managed exceptions can be catchable—or unhandled
On Android, an unhandled Java or Kotlin exception can end the app process. A handler can help only when the exception reaches it through the language and runtime’s normal handling path; code outside the handler’s scope or an exception that remains unhandled can still lead to a crash. The relevant evidence is the exception and its stack, not merely the presence of a try/catch somewhere in the code.
Native faults cross a different boundary
A native signal such as SIGSEGV is not an ordinary managed-language exception for a JavaScript or Kotlin handler to catch. Once the operating system terminates the process, application-level code in that process cannot run a recovery handler. Expo’s error-reporting documentation describes this boundary for its supported reporting setup: a native crash terminates the app before JavaScript can react, and a report is written on-device and dispatched at the next launch. That behavior should not be generalized to every JavaScript environment or reporting configuration.
Exception unwinding has platform-specific limits
Apple documents a separate boundary: if a thrown exception reaches a stack frame without unwind data, exception handling cannot proceed and the process halts. Its guidance discusses compiler unwind-table settings in that context. This is a build- and platform-specific limitation, not evidence that ordinary try/catch is generally ineffective on Apple platforms. See Apple’s guidance on language exception crashes.
Recommended Free Tools
Choose the diagnostic artifact that matches the symptom
Android: inspect the exception or signal and stack trace
Android crash stack traces show the nested function calls leading up to the failure. Use them to find the crash path and examine the relevant exception or signal. A crash may also originate in a background component while the app is not in the foreground, so absence of a visible crash dialog during a test does not by itself rule out a process crash. Android’s crash documentation covers both crash behavior and diagnosis.
Apple platforms: distinguish crash reports from logs
Apple crash reports record how an app terminated and the code running on each thread at the time. They are not interchangeable with Jetsam event reports or device console logs, which provide different kinds of system and app context. Use a crash report when the app terminated; use console logs to investigate issues that do not constitute crashes. Apple’s pages on diagnosing with reports and logs and acquiring diagnostic material explain these distinctions.
Separate test failures, crashes, and non-fatal events
A failed test, a process-ending crash, and a reported non-fatal exception are different records. For example, Firebase Crashlytics documents collection and grouping of crashes, non-fatal exceptions, and other events; it also describes a separate path for recording non-fatal exceptions. Such reporting can help collect and triage evidence, but the cited documentation does not establish that reporting prevents crashes or proves their root cause. See Firebase Crashlytics and its Android non-fatal exception guidance.
Rank #4
When is it fair to call the test flaky?
Use “flaky test” for a test whose result changes under conditions that should produce the same outcome, after checking whether the test or harness itself is responsible. Do not use the label to explain away a recorded app crash. A crash can be intermittent and still be a crash; classify the process outcome and its artifacts first, then investigate why the test sometimes reaches the fault and sometimes does not.
Free tools Windows power users keep installed
One-click scans. No signup required.
No universal retry count establishes flakiness. The useful question is whether repeated runs and changed conditions reveal a consistent failure mechanism—not whether one retry happened to pass.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
