What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Short answer: do not assume you may automate searches just because a results page is publicly visible. First identify the search engine and the exact data you need, then check that provider’s current access rules and use an API or another route it authorizes. For Google specifically, its published policy says automated queries to Google Search—including scraping results for rank checks or other automated access without express permission—violate its spam policies and Terms of Service. That is Google’s stated policy position, not a universal legal ruling about every search engine or every use of search data.
Decide what you mean by “scrape search engine results”
A search results page is not the same thing as an ordinary website you crawl. A crawler visits pages on a site and follows links or other instructions to collect content. A search-results collector asks an engine to run queries and gathers information the engine returns, such as links, snippets, ads, or local results. The two activities involve different providers, rules, interfaces, and data.
Before building anything, write down the engine and the fields you actually need. “Search results” may mean organic result URLs, titles and snippets, sponsored placements, local listings, or a particular ordering for a specified location and language. Do not assume a provider’s API contains every feature shown in its web interface. The specific coverage of a route must be confirmed in that route’s current documentation.
- Target: Which search engine or engines will be queried?
- Fields: Which result types and attributes are essential, and which are optional?
- Context: Does the use case require a particular country, language, or query location?
- Use: Will the data be stored, displayed to others, used for rank tracking, or reused in another product?
These answers shape both the access decision and the technical design. A feed that returns organic links but not local results, for example, may not serve a project that depends on local listings.
Check the engine’s rules before choosing a collection method
Read the target engine’s current terms and any specific published policy on automated access. Google’s Search spam policy explicitly addresses machine-generated traffic: it includes automated queries to Google Search, such as scraping results for rank checking or other automated access without express permission. Google says, “Such activities violate our spam policies and the Google Terms of Service.” That statement is about Google Search and Google’s published rules; it should not be broadened into a claim about every provider.
#1 Best Overall
Google’s general Terms of Service also address automated access that violates machine-readable instructions and scraping content that does not belong to the user. A provider’s terms or policies are not, by themselves, a definitive legal judgment. This guide does not determine whether a particular collection method or use is lawful. Where the consequences matter, get advice for the relevant jurisdiction and use case rather than treating an API’s existence, a robots.txt file, or a court headline as blanket permission.
Do not treat robots.txt as search permission
Google describes robots.txt as a mechanism for managing crawler access and traffic. It is not a way to ensure that a URL is absent from Google Search, and it does not grant permission to automate queries to Google Search. For ordinary sites your crawler visits, check the site’s crawler guidance separately. Those instructions address crawling that site; they do not replace the search engine’s own terms or answer every legal question.
Do not read a lawsuit as a universal yes or no
Search-related litigation is time-sensitive and does not supply a universal answer for all engines, users, or methods. In the reporting reviewed for this article, a July 2026 dismissal in Google LLC v. SerpApi was followed by an amended complaint and a renewed motion to dismiss reported by SerpApi on August 25, 2026. The current docket status and the order’s exact legal implications were not established here. The reported dismissal should not be presented as blanket permission to scrape search results.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Prefer an official or authorized data route
Once you have checked the provider’s rules, look for a documented API or another route the provider authorizes for your purpose. Confirm eligibility before writing an integration: an API can exist while being unavailable to new applicants, limited to certain result types, or governed by terms that do not fit your intended storage or display.
Google Custom Search JSON API: important availability limits
Google documents the Custom Search JSON API as a way to receive results in JSON from a configured Programmable Search Engine. Using it requires a configured engine and an API key. It should not be assumed to be a general-purpose replacement for every Google Search results page: verify that its configured search scope and returned fields meet your need.
As stated in Google’s documentation version surfaced in 2026, the API is closed to new customers. Google says existing customers have until January 1, 2027 to transition. The same overview states an allowance of 100 free queries per day, with additional queries available for a fee. These are changeable product details; check Google’s current overview and applicable terms before relying on eligibility, quota, cost, or the transition date.
Google’s overview lists Vertex AI Search as an alternative for searching up to 50 domains and says it is gathering interest for a full-web-search solution. Do not assume either option is equivalent to a general Google Search results API, or that you can access it on the terms you need. Confirm current availability, scope, pricing, and conditions directly with Google.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Evaluate an authorized route against your requirements
If you are considering a third-party SERP API, assess the provider and its terms rather than presuming that an API makes every use permitted. The information available here does not validate a particular vendor, coverage claim, or authorization program. Compare the points below before adopting one:
| What to check | Question to answer |
|---|---|
| Authorization and eligibility | Does the provider permit your intended collection and downstream use, and can your account use the service? |
| Coverage and fields | Does it return the exact result types and fields you need, including any required ads, local results, or snippets? |
| Geography and language | Can you request the country, language, or location context your project needs, and how is that context defined? |
| Quota and total cost | What are the rate limits and charges at your expected volume, including retries and failed requests? |
| Storage and reuse | Can you retain, display, or redistribute the returned data in the way your application requires? |
| Reliability and maintenance | How are errors reported, and what work will be needed when the provider changes its API or terms? |
Use the provider’s own current documentation and agreement to answer these questions. Do not infer quality, coverage, or permission from a marketing description alone.
Design the collector around the authorized interface
After confirming a route is both available to you and appropriate for the intended use, implement against its documented interface—not by reverse-engineering a search page or substituting browser automation for a provider’s stated access rules. Exact API parameters, endpoints, and response fields depend on the selected provider and are not specified here; use that provider’s current documentation rather than copying an example for a different service.
- Define one query request. Specify only the query, location, language, and result options supported by the documented interface. Avoid assuming that a parameter name has the same meaning across providers.
- Keep credentials out of source control. Store API keys in an appropriate secret store or environment configuration, and follow the provider’s instructions for protecting and rotating them.
- Parse the documented response shape. Treat optional fields as optional. Preserve the source and retrieval time internally when needed to interpret or refresh stored data, subject to the provider’s retention terms.
- Handle failures explicitly. Distinguish authentication or quota errors from temporary network failures and responses that contain no results. Retry only when the provider’s guidance permits it.
- Set an operational limit and stop path. Keep request volume conservative, monitor error responses, and pause collection when access is blocked or challenged. Do not respond to a block by rotating identities or trying to evade it.
- Review the use of stored data. Apply the provider’s limits on retention, display, and reuse, and remove data when your obligations or project needs require it.
This approach is intentionally provider-neutral: it makes a collector easier to maintain without implying that a particular request pattern overrides a search engine’s restrictions.
How to collect from ordinary websites without confusing the rules
If the actual goal is to crawl pages on a site—not to automate queries against a search engine—start with that site’s crawler guidance and the content you are allowed to access. Respect applicable machine-readable instructions, keep request volume modest, and build a clear way to stop. A site’s robots.txt can communicate crawler preferences and manage traffic, but it does not guarantee that a URL is excluded from a search index and is not proof of permission for some separate activity.
Best Value
Also separate the page-fetching step from any later need to inspect or archive a visual rendering. A screenshot represents a rendered page; it is not a structured source of search-result data and does not resolve the access rules for the page or service being captured.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Or skip the browser setup: capture a page with ScreenshotNeo
ScreenshotNeo is a website screenshot API and MCP server, not a search-results API. It will not run a search query or return structured SERP fields. If you already have a URL you are authorized to capture and need a rendered image or PDF, one GET request can return a screenshot. See the ScreenshotNeo documentation for API details.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpReplace the example URL with the page you are allowed to capture and provide your API key. ScreenshotNeo accepts cookie or consent banners as a visitor and removes more than 60 known consent platforms, newsletter popups, and chat widgets before capture; each of those steps can be turned off. Bot checks or CAPTCHAs, blank pages, timeouts, failed loads, and cache hits are not billed, and the response identifies the page verdict and billing status in headers. Its MCP server offers take_screenshot, get_page_info, and capture_pdf for Claude, Cursor, and other MCP clients.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Fix the driver behind crashes, sound loss and screen glitches3Repair Windows errors before they cause bigger problemsThe free plan includes 1,000 screenshots per month with no card. Paid plans start at $5 for 3,000 screenshots; yearly billing gives two months free, and every feature is available on every plan. Sign up for ScreenshotNeo’s free plan to try it with 1,000 screenshots a month and no card.
Reliability, maintenance, and cost: what to plan for
An authorized API can reduce the engineering burden of extracting structured results, but it does not eliminate the work of checking access, handling errors, or controlling spend. Estimate expected query volume before selecting a plan or relying on a daily allowance. Include retries in that estimate; a retry loop can multiply requests when the underlying problem is persistent.
- Quota: Check whether limits are daily, monthly, or rate-based and how overages are handled. Google’s 100-free-query daily allowance is the amount stated in the 2026 documentation version surfaced here, not a promise that the limit will remain unchanged.
- Response quality: Validate that the returned fields are present and usable for your particular queries. Do not treat missing fields as evidence that the result did not exist.
- Change management: Watch the provider’s API and terms notices, especially if the integration depends on a product with a transition deadline.
- Data lifecycle: Store only what you need and ensure your retention, display, and reuse practices fit the applicable terms.
- Failure behavior: Log enough to diagnose errors without exposing API keys or retaining more user or query data than necessary.
For the Google Custom Search JSON API in particular, existing customers should plan around Google’s stated January 1, 2027 transition deadline and verify the current notice. A new project should not be designed around access that Google says is closed to new customers unless its current eligibility information establishes otherwise.
Troubleshooting common collection problems
| Symptom | Likely cause | What to do |
|---|---|---|
| You cannot enable or obtain access to an API | The service may be closed to new customers, require a configured search engine, or require credentials. | Check current eligibility and setup requirements. For Google Custom Search JSON API, the documentation surfaced in 2026 says it is closed to new customers and requires a configured Programmable Search Engine and API key. |
| The API returns fewer result types than the search page | The API’s scope or response fields may not include the feature you expect. | Compare the documented output with your required fields before building around it; choose a route whose stated coverage fits the task. |
| Requests fail after a volume increase | You may have reached a quota or rate limit, or a transient service/network issue may be occurring. | Inspect the provider’s documented error response and current limits. Reduce request volume and use only the retry behavior the provider permits. |
| A page appears in search despite a robots.txt restriction | robots.txt is not a guarantee that a URL will be absent from Google Search. | Do not use robots.txt as an indexing-removal guarantee or as a substitute for access rules. Follow the search provider’s documentation for the specific indexing issue. |
| A search page blocks or challenges automated access | The provider may be restricting the automated request. | Stop the automated access and review the provider’s policy or use an authorized route. Do not attempt to evade the challenge. |
| A legal headline seems to say scraping is allowed | A procedural ruling may be limited, appealed, amended, or otherwise unresolved. | Check reliable current case records and obtain jurisdiction-specific advice before relying on a litigation outcome. The reported Google LLC v. SerpApi developments described above do not establish blanket permission. |
Frequently Asked Questions
Does a screenshot API give me structured search-result data?
No. ScreenshotNeo captures a rendered page as an image or PDF; it is not a SERP API and does not return structured search-result fields.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
