You cannot make a WordPress blog impossible to scrape. The practical goal is to reduce how much content automated bots can collect, make abusive traffic expensive, preserve legitimate readers and search access, and respond quickly when someone republishes your work. Start with excerpt-only feeds and a reviewed robots.txt, then add WAF/CDN rate limits, hotlink protection for bandwidth, monitoring and a documented takedown process.
What “stopping” a scraper really means
Any page sent to a visitor can be copied by a browser or bot. WordPress.com Support’s guidance on preventing content theft cautions that “there is no way to fully guarantee the complete protection of your work.” A determined scraper can ignore instructions, imitate a browser, or copy text manually.
That limitation does not make controls pointless. Layered defenses can limit feed exposure, block abusive request patterns before they reach WordPress, protect image bandwidth and give you evidence for removal requests. Judge each measure by what it covers, how easily it can be bypassed, its effect on your server and legitimate visitors, and the work required to maintain it.
How do I prevent RSS scraping in WordPress?
Change feeds to summaries or excerpts
- Sign in to WordPress and open Settings → Reading.
- Find For each article in a feed, show.
- Select Summary (or the excerpt option shown by your version) and save the change.
- Open the site’s feed in a feed reader or browser and confirm that it contains enough context for legitimate subscribers without publishing every full article.
A full feed gives a basic RSS scraper the complete post in one request. A summary feed narrows that exposure, but it also makes the feed less convenient for readers who prefer full-text subscriptions. This setting does not protect the HTML page, REST responses, images or content that has already been copied.
#1 Best Overall
Does robots.txt stop scrapers?
No. WordPress describes robots.txt as a set of directions telling search engines what they should and should not check. Cooperative crawlers may follow those directions; hostile bots can ignore them. Treat the file as a traffic preference, not authentication or an access-control barrier.
Use robots.txt narrowly
- Review the generated file and disallow only low-value, duplicate or genuinely sensitive paths that should not be crawled.
- Keep XML sitemap locations discoverable so compliant search engines can find your canonical pages.
- Do not disallow your entire site to fight scraping; that can remove your pages from search while an abusive bot continues to fetch them.
- Developers can modify WordPress’s generated output with the
robots_txtfilter. Test changes after theme, plugin or hosting updates.
Never put secrets, private documents or an access-control rule in robots.txt. If a path must be private, protect it with authentication or remove public access at the server or application layer.
Block abusive requests at the edge
WordPress security guidance recommends rate limiting at a web server or at the edge through a managed WAF/CDN. Blocking a request before PHP and WordPress run preserves origin CPU, memory and bandwidth.
Start with conservative rules
- Place the domain behind a reputable WAF/CDN and verify that normal page views, publishing, administration and APIs still work.
- Challenge or rate-limit repeated requests to post archives, feeds, search, REST endpoints and large downloads. Cloudflare’s scraping examples cover limits based on query strings, request bodies and resource downloads.
- Begin with thresholds that observe suspicious bursts rather than immediately banning broad user-agent categories.
- Review firewall events, origin logs and support reports for false positives; then tighten rules in small steps.
- Allow trusted integrations, uptime checks and approved partners through explicit, documented exceptions where needed.
Edge controls have broad coverage across HTML, feeds, APIs and downloads, but they require tuning. A rule that is too aggressive can block readers behind shared networks, accessibility tools, search crawlers or legitimate feed services.
Use hotlink protection for stolen image bandwidth
Hotlink protection checks the HTTP referrer on image requests and can stop other sites from embedding files directly from your origin. That can reduce bandwidth consumed by your server. Cloudflare explicitly notes that “Hotlink protection has no impact on crawling,” so it does not stop someone from copying an article or downloading an image directly.
Configure exceptions before enabling it
- Allow your own domains and any image URLs intentionally used in RSS feeds, newsletters, social cards or approved partner pages.
- Test images from a normal browser, feed reader and social-sharing preview after activation.
- Use it when bandwidth theft is the problem, not as a substitute for WAF rules or copyright enforcement.
Compare the main anti-scraping controls
| Control | Coverage | Bypass resistance | Origin-resource effect | False-positive risk | Setup and cost | SEO/feed effect |
|---|---|---|---|---|---|---|
| Summary RSS feed | Feeds | Low; pages remain public | Reduces feed payloads | Low | Easy; built in | Less useful for full-text subscribers |
robots.txt |
Compliant crawlers | Low; voluntary | None against bots that ignore it | High if important paths are blocked | Easy; built in, with optional development changes | Can harm discovery if misconfigured |
| WAF/CDN rate limiting | HTML, feeds, APIs and downloads | Medium to high, depending on tuning | Protects the origin by filtering at the edge | Medium; monitoring is essential | Moderate; service or plan costs vary | Usually neutral when trusted traffic is allowed |
| Hotlink protection | Embedded images | Low to medium; targets referrer-based use | Reduces image bandwidth | Medium if feed or social exceptions are missed | Moderate; CDN setting | Can break intentional image embeds |
| Monitoring and takedowns | Copies already published | Not preventive | No traffic reduction | None for visitors | Manual or paid monitoring; legal effort varies | Does not change distribution |
Detect copied posts and establish ownership
Create an evidence trail
- Keep dated drafts, publication records, source files and backups that show when you created and published the work.
- Place a clear copyright notice on the site. It communicates ownership even though it is not a technical barrier.
- For important images, consider a visible watermark as an attribution deterrent. A watermark can itself be cropped or removed, so it does not prevent copying.
Search for copies
- Choose distinctive sentences from a post and search them in quotation marks.
- Create a Google Alert for your site name, author name and distinctive brand phrases.
- For a larger archive, evaluate Copyscape’s search or paid monitoring. Compare the service’s coverage and cost with the value of the content you need to protect.
When you find a match, save the copied URL, screenshots or downloaded pages, the original URL and proof of your earlier publication date before contacting anyone.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What to do when someone republishes your work
- Document first. Record the original and copied URLs, dates, screenshots, page source where useful, and the specific text or images taken.
- Check authorization. Confirm that the use is not licensed, quoted under an applicable exception, submitted by a partner or already permitted by your terms.
- Ask for attribution or removal. Send a factual request to the site owner with the original link, the copied material and a deadline.
- Contact the host or platform. Use its abuse or copyright procedure if the operator does not respond.
- Consider a DMCA notice where applicable. WordPress.com describes the DMCA as a US federal framework for removing unauthorized online uses. Procedures and legal rights vary by country, provider and type of use; use the relevant provider’s form and obtain legal advice for contested or high-value matters.
Do not publish accusations based only on a similar topic or common phrase. Preserve evidence and identify the exact expression, image or file that was copied.
Quick Recap
A practical WordPress anti-scraping sequence
- Set Settings → Reading → For each article in a feed, show → Summary, then test a legitimate feed reader.
- Review
robots.txt; block only low-value or sensitive crawl paths and leave XML sitemaps discoverable. - Deploy a WAF/CDN. Add conservative limits for repeated archive, feed, REST, search and large-download requests; monitor before tightening.
- Enable hotlink protection if image bandwidth is being consumed, with explicit feed, social and partner exceptions.
- Update WordPress core, themes and plugins, remove unused plugins, and inspect logs for request bursts, unusual user agents and sequential URL access.
- Add a copyright notice, retain dated originals and backups, and configure quoted-text searches or alerts. Adopt paid monitoring when the archive justifies it.
- For each infringement, capture evidence, request attribution or removal, and escalate through the host’s abuse process or an appropriate DMCA workflow.
Common mistakes to avoid
- Relying on
robots.txtas if it were a password. - Blocking every unfamiliar user agent; legitimate crawlers and accessibility tools may be affected.
- Enabling hotlink protection without testing RSS, social previews and approved embeds.
- Assuming a summary feed protects the full HTML article or API output.
- Waiting to collect evidence until after a copied page disappears or changes.
- Claiming a specific percentage reduction in scraping; these controls have no universal benchmark and results depend on the bot, configuration and site.
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Recommended Free Tools
