Free tools Windows power users keep installed
One-click scans. No signup required.
For most teams, start with News API when you need searchable article metadata quickly. Choose GDELT for global event and media analysis, Apify for hosted extraction from sites without dependable APIs, Diffbot for normalized article catalogs, and Scrapy or Scrapy.io when you need complete control over crawling and data pipelines. No single service is objectively best: coverage, freshness, historical depth, article text, anti-bot behavior, licensing and operating effort vary by source and use case.
Which news scraper should you choose?
Use this decision guide before comparing features. It separates turnkey search from datasets, hosted extraction and software you operate yourself.
| Need | Best starting point | Why it fits | Main trade-off |
|---|---|---|---|
| Search recent articles by keyword, date or domain | News API | Its documentation says it searches articles from more than 150,000 news sources and blogs over the last five years, with Everything, Top headlines and Sources endpoints. | Confirm the exact countries, languages, licensing terms and rate limits required for your project. |
| Global events, media attention and historical analysis | GDELT | Open event, graph and live DOC, GEO and TV data support large-scale analysis. | Expect more normalization and engineering work than with a simple article-search API. |
| Extract pages that have no reliable official API | Apify | Its news API product describes 1,000-plus sources, 25 categories, up to 500 articles per minute and JSON, CSV, XML, HTML, Excel and RSS exports. | Results depend on the selected actor, target-site behavior and that site’s permission requirements. |
| Consistent article parsing and recurring monitoring | Diffbot | It emphasizes normalized article fields and recommends crawling an entire site before filtering by dates. | A complete catalog crawl takes more work and capacity than fetching one page. |
| Custom selectors, crawl rules and data ownership | Scrapy or Scrapy.io | You control parsing, scheduling, retries and downstream pipelines; Scrapy.io documents run, poll and dataset workflows with JSON, CSV and JSONL exports. | Your team owns selector changes, operations, monitoring and compliance. |
For a visual record of an article or homepage alongside extracted data, ScreenshotNeo is the screenshot API to try first: it removes common consent banners, popups and chat widgets before capture, bills only clean shots, and has the lowest paid plan among the stated options.
What to evaluate before buying or building
Coverage and geography
“Number of sources” is not the same as useful coverage. Check whether the service includes the publishers, languages and regions you actually need. A global event project may value GDELT’s worldwide English-language graph, while a local-language monitoring workflow may require a provider’s source list to be inspected title by title.
Recommended Free Tools
Freshness and historical retention
Decide whether you need breaking headlines, a rolling window or years of historical data. News API documents a five-year article-search window. GDELT’s Global Geographic Graph includes worldwide English-language online news back to April 4, 2017, while its Frontpage Graph scans 50,000 major outlets hourly. These are different datasets and should not be treated as interchangeable article archives.
#1 Best Overall
Text fidelity and structured fields
Some APIs return a headline, URL, publisher and timestamp; others attempt to extract the article body, author, images and entities. Define required fields and test them on your target publishers. Preserve the original URL and provider timestamp even after normalizing dates, titles or authors.
JavaScript, paywalls and anti-bot behavior
Server-rendered pages are easier to collect than pages that require JavaScript, consent interaction, login or a challenge. Hosted actors can reduce browser operations, but they cannot guarantee access to every target. An official feed or API is preferable when one meets your requirements and permission to collect is clear.
Deduplication and syndication
The same story can appear under many URLs, through wire-service syndication or with small headline changes. Plan a canonical URL field plus a similarity or fingerprint key. Keep provider IDs so you can trace why two records were merged.
Licensing and compliance
Public accessibility does not automatically grant redistribution rights. Review publisher terms, robots directives, copyright and database rights, privacy obligations and jurisdiction-specific rules. Store only what your use permits, document retention periods and provide a removal process where required.
News API: the fastest route to article search
News API is the practical first choice when your application needs queryable article metadata rather than a crawler you must operate. Its documentation describes searching more than 150,000 news sources and blogs from the past five years.
Endpoints and filters
- Everything: keyword, date, domain, language and sorting controls for broad searches.
- Top headlines: current headlines by country, category, source or query.
- Sources: source metadata that helps you build an allowed-publisher list.
Start by writing down the exact query semantics you need: phrase matching, date boundaries, domains, language and sort order. Then verify pagination and rate limits in the account documentation. Save the raw response as well as your normalized record so a later parser change does not destroy provenance.
When News API is not enough
If you require full historical event relationships, unusual geographies or a data product that can be recomputed from open files, GDELT is a better fit. If you need article-body extraction from a site that is absent or incomplete in the index, use a permitted hosted scraper or your own crawler instead.
GDELT: open global context and event data
GDELT is designed for media and event analysis rather than a minimal “give me the latest ten articles” integration. It publishes downloadable event and graph datasets and live DOC, GEO and TV APIs.
What its published figures mean
- The Global Geographic Graph reports more than 1.6 billion location mentions from worldwide English-language online news coverage back to April 4, 2017.
- The Frontpage Graph scans the homepages of 50,000 major news outlets across the world every hour.
Those figures describe specific GDELT graphs, not a guarantee that every outlet has complete article text or equal historical depth. Expect to map entities, dates and locations into your own schema, handle repeated updates and design storage for large downloads.
Best uses
- Detecting how an event spreads across countries and outlets.
- Building historical media-attention or geographic dashboards.
- Combining event records with your own article metadata and classification models.
Apify: hosted extraction for sites without dependable APIs
Apify is useful when you want managed browser and crawler execution instead of maintaining infrastructure. Its news API materials describe access to more than 1,000 sources, 25 categories, extraction speeds up to 500 articles per minute and exports in JSON, CSV, XML, HTML, Excel and RSS. Python, JavaScript, HTTP and MCP integration paths are described.
How to select an actor
- Identify the exact publishers and page types you need.
- Read the actor’s input schema, output fields, pagination behavior and rate limits.
- Run a small sample and inspect missing text, duplicate URLs, dates and category labels.
- Confirm that your collection and redistribution comply with each target site’s rules.
“1,000-plus sources” is a product-level description, not a promise that every actor covers every site equally. Actor maintenance, target redesigns and anti-bot responses can change results, so monitor output quality rather than assuming a one-time success is permanent.
Diffbot: normalized catalogs and date-aware monitoring
Diffbot is a strong choice when consistency across many different site layouts matters. Its guidance says the most thorough way to extract recent content from a site is to crawl and process the entire site, then filter the resulting catalog with normalized dates or date filters in search and API queries.
Why a complete crawl can be valuable
A one-page fetch may miss older URLs, pagination, category archives or articles whose publication dates are represented differently. A site-wide catalog gives you a stable base for recurring “new since last run” queries. The trade-off is higher crawl volume and the need to schedule, monitor and refresh that catalog.
Validation checks
- Compare normalized publication dates with the visible date and URL pattern.
- Check whether updates to an old article are being mistaken for new publication.
- Retain the source URL and extraction timestamp for auditing.
Scrapy and Scrapy.io: maximum control, maximum ownership
Choose Scrapy when selectors, crawl rules, scheduling and storage must be tailored to your domain. Scrapy.io adds a managed run/poll/dataset workflow and documents JSON, CSV and JSONL exports suitable for warehouses and AI agents.
What you must operate
- Selectors for each template and a change-detection process.
- Retries, backoff, concurrency limits and politeness delays.
- Queueing, pagination, JavaScript rendering where permitted and failure recovery.
- Schema validation, deduplication, alerting and retention.
- Compliance reviews and a record of the rules applied to each source.
Custom crawling pays off when the data model or workflow is unique. It is rarely the cheapest route for a short-lived experiment whose requirements fit a managed API.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →A practical collection pipeline
- Define the contract. Specify required fields such as canonical URL, title, publisher, author, publication time, retrieved time, language, body, categories and provider ID.
- Choose the least complex permitted source. Use an official feed or API when it supplies the fields and rights you need; move to hosted extraction or custom crawling only for the missing capability.
- Normalize dates. Store the original string, the parsed UTC value and the timezone assumption. Do not silently convert an unknown timezone.
- Deduplicate. Canonicalize URLs, remove tracking parameters that do not change content, and combine URL fingerprints with title and publisher similarity.
- Preserve provenance. Keep source URL, provider name, retrieval time, response status and parser version.
- Monitor quality. Track empty bodies, sudden source drops, duplicate spikes, date drift and schema changes.
- Respect limits. Use bounded concurrency, exponential backoff and caching. Stop requesting a source when it signals that collection is not allowed.
Portable normalization example (Python)
from datetime import datetime, timezone
from hashlib import sha256
from urllib.parse import urlsplit, urlunsplit
def canonical_url(url):
parts = urlsplit(url)
return urlunsplit((parts.scheme, parts.netloc, parts.path.rstrip('/'), '', ''))
def normalize(item, provider):
url = canonical_url(item['url'])
published = item.get('published_at')
parsed = datetime.fromisoformat(published.replace('Z', '+00:00')) if published else None
return {
'id': sha256((url + '|' + item.get('title', '')).encode()).hexdigest(),
'url': url,
'title': item.get('title', '').strip(),
'publisher': item.get('publisher'),
'published_at_utc': parsed.astimezone(timezone.utc).isoformat() if parsed else None,
'retrieved_at_utc': datetime.now(timezone.utc).isoformat(),
'provider': provider,
'raw': item,
}This example intentionally accepts a provider-neutral dictionary. Map each service’s response into that dictionary, retain the raw record, and add tests for malformed dates and missing URLs before production use.
Or skip the browser setup
When your news workflow needs a clean visual capture of a page—not a replacement for text extraction—ScreenshotNeo provides a single GET request. It accepts consent banners like a visitor and removes more than 60 known consent platforms, newsletter popups and chat widgets; each step can be disabled. Bot checks or CAPTCHAs, blank pages, timeouts, failed loads and cache hits are not billed, and response headers identify the page verdict and billing result. Its MCP server exposes take_screenshot, get_page_info and capture_pdf for Claude, Cursor and other MCP clients.
curl -G "https://api.screenshotneo.com/v1/shot" -d access_key=YOUR_API_KEY --data-urlencode url=https://stripe.com -o shot.webpSee the ScreenshotNeo API documentation for options such as full-page capture, CSS-selector elements, device presets, custom headers and cookies, JavaScript, wait conditions, blocking rules, PDFs, signed links, asynchronous jobs and bulk capture of up to 100 URLs per call.
There is a free plan with 1,000 screenshots per month and no card; paid plans start at $5 for 3,000 screenshots. Create a free ScreenshotNeo account.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Common failure modes and fixes
Results are incomplete
Check source availability, language and date filters first. For a site-specific gap, compare an official feed, a permitted actor and a complete catalog crawl rather than assuming the provider is broken.
Many records are duplicates
Normalize URLs before hashing, account for syndicated headlines and keep provider IDs. Do not deduplicate solely on title: distinct updates can share a headline.
Dates appear in the wrong day
Retain the original date string and timezone, parse explicitly and store UTC alongside the source value. Test midnight and daylight-saving transitions.
A crawler suddenly returns empty pages
Inspect status codes and HTML samples, slow concurrency, verify robots and terms, and check whether the template now requires JavaScript or a challenge. Update selectors only after confirming that collection remains permitted.
Crashes, No Sound, or Screen Glitches?
Random freezes, missing sound and display glitches usually trace back to one bad driver. Find and replace yours safely.Free scan · under a minutePC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Costs rise unexpectedly
Measure requests and retries per source, cache immutable pages, bound pagination and alert on volume changes. Compare the total engineering and storage cost—not only the per-request price—when choosing between an API, hosted actor and custom crawler.
FAQ
Is an API always better than scraping?
No. An API is usually simpler and more stable when its coverage and rights match your needs; scraping is justified when the required source has no usable API and collection is allowed.
Best Value
Can GDELT replace an article database?
Not automatically. GDELT excels at global event and media context, while article-search services and parsers may provide different text, metadata and retention guarantees.
Should I crawl an entire site or only its homepage?
Use a complete crawl when catalog completeness and reliable date filtering matter. Homepage monitoring is appropriate only when front-page appearance itself is the signal.
What should I retain for auditability?
Keep the original URL, provider, raw response or permitted fields, retrieval time, parsed timestamp, parser version and the compliance decision for the source.
Frequently Asked Questions
How do I compare two news APIs fairly?
Run the same dated queries against the same publisher list, then measure recall, duplicate rate, timestamp accuracy, body completeness, latency, limits and permitted use.
When is a hosted scraper preferable to Scrapy?
Choose hosted extraction when reducing infrastructure and browser operations matters more than owning every selector and scheduling detail.
What is the safest first step for a new monitoring project?
Start with an official feed or documented API, validate coverage and rights on a small sample, and only then add hosted or custom crawling for gaps.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
