A URL-to-Markdown API can return HTTP 200 and valid Markdown while missing the article entirely. If the URL is a JavaScript-rendered single-page app (SPA), the initial HTML may contain only an app shell; an extractor can then mistake navigation and footer text for the page’s main content. Detect that low-quality result, render the page in a browser when needed, wait for meaningful content, and validate the extraction before calling it successful.
Why does an API return a menu instead of the article?
Many SPAs send an initial HTML document with the scaffolding for the app, then use JavaScript to fetch or construct content for the requested route. Google Search Central describes this app-shell pattern: “Some JavaScript sites may use the app shell model where the initial HTML does not contain the actual content and Google needs to execute JavaScript before being able to see the actual page content that JavaScript generates.” Google’s JavaScript SEO guidance also notes that not all bots execute JavaScript.
As an Amazon Associate I earn from qualifying purchases.
A static HTTP fetch can therefore succeed at the transport level without retrieving the article. The HTML may still contain a site header, navigation, footer, and an empty mount point. An extractor that chooses the largest or most article-like text block can return that repeated site chrome as Markdown. The result is syntactically plausible but semantically wrong.
HTTP status and Markdown validity are not content checks. Google also documents cases where client-side errors can leave a page returning HTTP 200, and recommends inspecting rendered HTML with tools such as its Rich Results Test or URL Inspection. Those checks help diagnose what a search crawler sees; they do not replace an extraction API’s own quality checks. Google’s JavaScript troubleshooting guidance
#1 Best Overall
How can you detect an SPA shell response?
Use multiple signals rather than a single word-count cutoff. Sparse text, an empty likely main-content region, familiar app mount points such as #root, #__next, or #app, and repeated navigation or footer phrases can all suggest that the route content has not arrived. These are heuristics, not universal indicators: frameworks and site structures vary, and a short legitimate page can also have little text.
One public implementation uses fewer than 25 words and mount-point markers as signals for deciding whether to escalate from static fetching. Treat that threshold as an implementation choice, not a standard. The yomi project README describes a static-first approach that can fall back to headless Chrome when a response appears JavaScript-gated.
Check the extracted candidate, too. Word count alone can miss a long menu that looks substantial. Consider whether the text has plausible article structure, whether navigation labels recur, and whether the candidate contains a meaningful main-content region rather than site-wide boilerplate.
Should you render every page in a headless browser?
Rendering pages in a browser executes client-side JavaScript and can expose content absent from the initial response, but it requires browser resources and adds operational work. A static-first strategy with conditional browser fallback avoids rendering every URL, while always rendering may be simpler when broad JavaScript coverage matters more than resource use. The available sources describe these trade-offs and implementation patterns, but do not establish a controlled comparison of cost, latency, or extraction accuracy.
| Approach | Coverage and control | Operational considerations | Failure handling |
|---|---|---|---|
| Static fetch only | Does not execute page JavaScript; can miss content generated after the initial HTML. | Does not require browser infrastructure. | Needs shell and low-content checks, with an explicit failure response when content is missing. |
| Static first, browser fallback | Can handle likely client-rendered pages while retaining control over when and how long to render. | Requires fallback browser infrastructure or a rendering service, but reserves it for flagged pages. | Make the escalation reason, readiness timeout, and final result observable. |
| Browser rendering for every URL | Executes JavaScript broadly; readiness still needs to be defined for each page or site pattern. | Uses browser resources on every request and entails infrastructure or service dependency. | Validate extracted content even after rendering; browser execution alone does not guarantee an article was retrieved. |
A hosted browser-rendering or URL-to-Markdown API is another way to avoid operating browser infrastructure yourself. Evaluate its actual JavaScript execution, readiness behavior, timeout handling, and output validation rather than assuming that “rendered” means “ready.” Cloudflare Browser Run documentation warns that default page-load behavior can return empty or incomplete results for JavaScript-heavy pages. Vendor documentation for URLpipe’s extraction API describes extraction from a browser-rendered page; that establishes the service pattern, not independent performance or program terms.
How do you know the rendered page is ready?
A generic page-load event is not proof that an SPA has finished rendering or fetching route data. Cloudflare’s Browser Run documentation specifically warns that page-load completion may precede JavaScript rendering completion. Instead, define a meaningful readiness condition tied to the content you need, and give it a bounded wait with a clear timeout path.
- Wait for a relevant main-content element or another site-specific signal, not merely the browser’s initial load event.
- Set a maximum wait so a stalled route cannot hold the request indefinitely.
- If the expected content condition is not met before timeout, mark the render as incomplete rather than extracting the shell as an article.
- After the condition is met, run extraction against the rendered DOM and validate the extracted result.
What should the recovery flow do?
- Fetch and retain diagnostics. Make an ordinary HTTP request and keep the response status, final URL, and raw HTML so you can distinguish redirects, server responses, and a likely app shell.
- Inspect the candidate content. Look for sparse visible text, an empty main region, mount-point markers, or repeated site chrome. Use these as combined warning signals, not rigid pass/fail rules.
- Escalate likely shells. Open the page in a JavaScript-capable browser, then wait for a meaningful content condition with a bounded timeout. Do not equate a load event with completed hydration or route-data fetching.
- Extract from the rendered DOM. Run the same article extraction logic against the rendered page, then inspect its output for plausible article structure and boilerplate repetition.
- Return an honest failure when needed. If content remains absent or implausibly thin, return an explicit low-content or render-failed status with useful diagnostics. Depending on the site, a print view or RSS representation may provide an alternate source, but availability is site-specific.
During development, Google’s Rich Results Test or URL Inspection can help you inspect rendered HTML. They are search-rendering diagnostics, not substitutes for your own checks that an extracted result is actually article content. Google’s guidance on fixing JavaScript search issues
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Repair Windows errors before they cause bigger problems3Scan for outdated or missing drivers - takes under a minuteHow common is this failure?
A case article published October 1, 2026, reports that about 1 in 6 URLs in its author’s own API traffic came back under 60 words. That is a limited observation about that API’s requests, not an industry-wide rate, and it does not establish that every short result was a navigation-only page. The figure is a useful reminder to measure low-content responses, not a prevalence estimate for URL-to-Markdown tools generally.
Quick Recap
Best Value
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
