Some links on this page are affiliate links: if you buy through them we may earn a commission, at no extra cost to you.
If PhantomJS finishes page.open() but a jQuery-populated area is empty, the usual problem is timing: navigation completion and $(document).ready() do not mean a later AJAX request has finished rendering. Check the navigation status, make sure jQuery is available before using it, wait for a page-specific signal that the data has arrived, and only then read the DOM with page.evaluate().
Why PhantomJS can reach document.ready before the content appears
PhantomJS has several distinct milestones that are easy to conflate. Its page.open(url, callback) callback reports whether navigation succeeded or failed; it does not promise that application code subsequently fetching data has completed. jQuery’s $(document).ready() indicates that the initial document is ready for DOM operations, not that an AJAX call started by the page has returned or updated the DOM. The PhantomJS API documents the navigation callback status as success or fail (PhantomJS WebPage API); PhantomJsCloud’s guidance likewise distinguishes DOMContentLoaded from later page completion and discusses AJAX-driven waits (PhantomJsCloud documentation).
The fix is to synchronize against the result you need, rather than assuming a lifecycle event or an arbitrary delay implies success. A result element appearing, a loading marker disappearing, an expected item count being reached, or an application flag becoming true are better signals than a fixed sleep.
Recommended Free Tools
Use this sequence to wait for AJAX-rendered content
- Open the page and check its status. If it is not
success, diagnose navigation or network failure before treating the empty result as a timing issue. - Ensure jQuery is loaded. If the target page already provides jQuery, use that copy. Otherwise inject it with
page.includeJs(), and put any jQuery-dependent work in that function’s callback. - Wait for a completion signal. Poll for a selector, flag, or other state that represents the data you need. Set a deadline so the script cannot wait forever.
- Read a serializable value. Use
page.evaluate()to return text or a plain object, not a DOM node or page-context function. - Exit only when the asynchronous work is done. Do not call
phantom.exit()before the include callback or your wait has completed.
Runnable PhantomJS example
This pattern checks for #results-loaded and then reads text from #results. Replace both selectors and the timeout with values appropriate to the site. The completion selector should indicate that the expected data has actually been rendered, not merely that an empty container exists.
#1 Best Overall
var page = require('webpage').create();
page.onError = function (msg, trace) {
console.log('page error: ' + msg);
};
page.onResourceError = function (resourceError) {
console.log('resource error: ' + resourceError.url + ' :: ' + resourceError.errorString);
};
var url = 'https://example.test';
page.open(url, function (status) {
if (status !== 'success') {
console.log('open failed for ' + url + ': ' + status);
phantom.exit(1);
return;
}
// Skip includeJs if the page already loads jQuery.
page.includeJs('https://ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js', function () {
var deadline = Date.now() + 10000;
function poll() {
var ready = page.evaluate(function () {
return !!document.querySelector('#results-loaded');
});
if (ready) {
var result = page.evaluate(function () {
var node = document.querySelector('#results');
return node ? node.textContent : '';
});
console.log(result);
phantom.exit();
return;
}
if (Date.now() >= deadline) {
console.log('Timed out waiting for #results-loaded');
phantom.exit(1);
return;
}
setTimeout(poll, 100);
}
poll();
});
});The jQuery URL shown is the one used in the documented example; for real deployments, use a compatible source available to the PhantomJS process, or rely on the version already included by the site. The critical ordering is that the injected library’s callback encloses the dependent work. PhantomJS’s automation guide warns that phantom.exit() belongs inside the includeJs callback or the process may exit before the library is included (PhantomJS quick start).
Prefer a condition over a fixed sleep
A fixed delay such as setTimeout(readResult, 3000) is simple, but it is unreliable in both directions: it wastes time on fast responses and still fails when a slow response takes longer. Polling an application-specific signal lets the script proceed as soon as the relevant state exists while retaining a timeout for failure handling.
Choose a condition tied to successful data rendering. For example, query a known result row, check that a loading indicator is gone, compare a result count with an expected value, or have the page set a flag in its AJAX success handler. If the application can return zero results, use a separate loaded-state signal; testing only whether result rows exist would mistake a valid empty response for a timeout.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Fix the driver behind crashes, sound loss and screen glitches3Clear out junk files and repair common Windows errorsUsing jQuery explicitly
If you need jQuery methods in the evaluated page, load the library first and place the dependent work inside page.includeJs(url, callback). If the site already includes jQuery, injecting another version may create compatibility problems, so verify availability before adding a second copy. The loading fix does not require jQuery for the wait itself: browser-native selectors in page.evaluate() can check the signal and extract text.
Rank #2
- HTML CSS Design and Build Web Sites
- Comes with secure packaging
- It can be a gift option
What page.evaluate can return
page.evaluate() runs code in the page context, but its results cross a serialization boundary. Return simple values such as strings, numbers, booleans, arrays, or plain objects. Do not return a DOM element, closure, or function; PhantomJS explicitly notes that “Closures, functions, DOM nodes, etc. will not work!” in its evaluate API documentation.
For example, return node.textContent rather than node. If several values are needed, return a plain object:
var data = page.evaluate(function () {
var node = document.querySelector('#results');
return {
found: !!node,
text: node ? node.textContent : '',
count: document.querySelectorAll('#results .item').length
};
});Keep the function passed to evaluate() self-contained: variables and closures from the PhantomJS script context are not page-context values. Pass or return only supported serializable data.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Diagnose an empty result systematically
Confirm navigation really succeeded
Log the URL being opened and the callback’s status. A fail is not an AJAX wait problem. Check that the URL is reachable from the machine running PhantomJS, that redirects lead where expected, and that required network access is available. The callback’s documented status values are success and fail (WebPage API).
Rank #3
Expose page JavaScript errors
Install page.onError before opening the URL. A script exception may prevent the page’s success handler from inserting content, making a timing wait appear to be the cause. Log the message and stack trace when available, then fix the page error or account for it before waiting longer.
Log resource failures
Use page.onResourceError to see failed resource URLs and error strings. A failed API request, script, certificate negotiation, or network transfer cannot be repaired by polling longer. For deeper diagnosis, also log resource request and response events supported by the PhantomJS version in use; inspect whether the expected API request was made and whether it returned successfully.
Check whether the selector can reach the content
Verify the selector in the actual rendered document. Content inside an iframe belongs to a separate document context; content inside a shadow DOM may require querying through the relevant shadow root. PhantomJS uses an older browser engine, so modern page constructs or APIs may not behave as they do in a current browser. If the selector is absent, first determine whether the page created the content at all, rather than increasing the wait interval.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Inspect load progress as a diagnostic, not a content guarantee
PhantomJS exposes page.loading and page.loadingProgress; its guide describes 100 as fully loaded (loadingProgress API). These properties can help explain current load state, but an application may still perform asynchronous work after a load milestone. Use them alongside, not instead of, the application-specific readiness condition.
Rank #4
- Brand: Wiley
- Set of 2 Volumes
- A handy two-book set that uniquely combines related technologies Highly visual format and accessible language makes these books highly effective learning tools Perfect for beginning web designers and front-end developers
Common causes and fixes
| Symptom | Likely cause | Fix |
|---|---|---|
| jQuery is undefined | The script runs before the library is available, or the page does not load it. | Use the page’s existing jQuery after confirming availability, or call page.includeJs() and wait for its callback before dependent work. |
| Navigation succeeds but results are empty | The page’s AJAX request or rendering has not finished. | Wait for a selector, loaded flag, marker removal, or expected count that reflects completion. |
| The process exits before the wait or injected script completes | phantom.exit() runs too early. |
Call it only on completion or timeout, after asynchronous callbacks have run. |
| Wait expires although the page appears loaded | The signal selector is wrong, data is validly empty, an exception occurred, or content is in an iframe or shadow DOM. | Inspect the rendered structure, distinguish loaded-empty from not-loaded, and log page errors. |
| The result is missing or unusable in the PhantomJS script | evaluate() returned a DOM node, function, or closure. |
Return text or a plain JSON-serializable value instead. |
| The API-backed section never appears | The API or another required resource failed, including possible network or certificate issues. | Log resource errors and responses, then fix the underlying request or environment problem. |
Performance and reliability considerations
Polling every 100 milliseconds, as in the example, is a practical diagnostic pattern, not a universal optimal interval. A shorter interval checks more often; a longer one adds latency between the page becoming ready and your script noticing. Keep a finite deadline and choose it based on the target application’s normal behavior and the consequences of a slow page. The sample’s 10-second deadline is illustrative, not a guarantee that every site will finish within that period.
For repeatable automation, prefer a page-owned loaded flag or a selector that unambiguously represents completed data. Record whether a run ended through navigation failure, page error, resource failure, or a readiness timeout; those outcomes call for different fixes. Avoid treating a timeout as success by silently returning an empty string, because that makes incomplete captures indistinguishable from valid empty content.
Or skip the browser setup
If the goal is a clean screenshot rather than debugging PhantomJS internals, ScreenshotNeo is a website screenshot API and MCP server. Its one-request API returns a PNG, JPEG, WebP, or PDF. For example, request a screenshot of the page like this (see the ScreenshotNeo API documentation):
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://example.test -o shot.webpScreenshotNeo accepts cookie or consent banners like a visitor and removes more than 60 known consent platforms, newsletter popups, and chat widgets before capture; each of those steps can be turned off. Bot checks or CAPTCHAs, blank pages, timeouts, failed loads, and cache hits are not billed, with response headers identifying the page verdict and billing status. Its MCP server provides take_screenshot, get_page_info, and capture_pdf tools for Claude, Cursor, and other MCP clients. The free plan includes 1,000 screenshots per month without a card; paid plans start at $5 for 3,000 shots.
Sign up for ScreenshotNeo’s free 1,000 screenshots a month, with no card required.
Frequently Asked Questions
Does document.ready wait for jQuery AJAX calls?
No. It marks initial DOM readiness; asynchronous requests may still be running or waiting to render.
Should I use page.onLoadFinished or page.open’s callback?
For the example here, use the callback supplied to page.open() to check its navigation status. Neither navigation completion callback establishes that later AJAX-rendered content is ready.
Free tools Windows power users keep installed
One-click scans. No signup required.
Can I return an element from page.evaluate()?
No. Return serializable data such as the element’s text or a plain object containing values.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
