What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Major publishers and platforms have told Apple not to use their public web content for AI training. The practical distinction is crucial: blocking Applebot-Extended can restrict foundation-model training while leaving Apple’s ordinary search crawling available. Blocking Applebot is broader and can reduce visibility in Apple search experiences.
The widely circulated news story dates to September 1, 2024, when Futurism reported that organizations including The New York Times, The Atlantic, the Financial Times, Gannett, Vox Media, Condé Nast, Facebook, Instagram, Tumblr and Craigslist had blocked or said they were blocking Apple’s AI-training access. That list should be read as reported behavior at the time, not as a continuously audited list through 2026.
As an Amazon Associate I earn from qualifying purchases.
What websites were actually blocking
The commonly cited rule is placed in the site’s root-level robots.txt file:
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →User-agent: Applebot-Extended
Disallow: /Apple says Applebot-Extended does not crawl webpages. It is a secondary user-agent that communicates whether data collected by Applebot may be used to train Apple’s generative foundation models. Calling it a separate “AI crawler” is therefore a useful shorthand, but technically imprecise.
#1 Best Overall
- BUILT FOR COLLEGE. AND BEYOND — MacBook Air with the M5 chip packs blazing speed and powerful AI capabilities into an incredibly portable design. And with up to 18 hours of battery life,* this thin and light powerhouse is ready to take on almost any major, just about anywhere.
- TEAR THROUGH TOUGH ASSIGNMENTS — With its faster CPU and unified memory, the M5 chip delivers even more performance and fluidity across apps, making multitasking and creative workflows smooth and responsive. A powerful Neural Engine and next-generation GPU with Neural Accelerators give you a powerful platform for AI.
- MAKE QUICK WORK OF YOUR TO-DO LIST — Apple Intelligence helps you write, express yourself, and get things done effortlessly — whether it’s for school or everyday life. With groundbreaking privacy protections, it gives you peace of mind that no one else can access your data — not even Apple.*
- UP TO 18 HOURS OF BATTERY LIFE — MacBook Air delivers incredible battery life with amazing performance, so you can power through a full day of classes without worrying about plugging in.
- A BRILLIANT 13.6-INCH DISPLAY* — The gorgeous Liquid Retina display on MacBook Air supports 1 billion colors, making photos and videos pop with rich contrast and sharp detail, and text appears supercrisp. So everything — from class presentations to movies to games — looks truly stunning.
A publisher can target a narrower path instead:
User-agent: Applebot-Extended
Disallow: /private/Path-level policies require careful testing because URL structures, redirects, caches and later site changes can make a supposedly narrow rule incomplete.
Applebot versus Applebot-Extended
| Tool or control | Purpose | What blocking or using it changes |
|---|---|---|
| Applebot | Apple’s general web crawler for search-related discovery, including functions associated with Spotlight, Siri and Safari | A broad block can affect crawling, indexing, snippets and Apple discovery |
| Applebot-Extended | A preference about whether Applebot-collected data may be used to train foundation models | Can restrict training use while ordinary Applebot crawling remains allowed |
isAccessibleForFree: false |
Schema.org markup identifying non-free or paywalled content | Apple says the page may remain searchable but will not be used as additional context for generated output |
| Privacy inquiry | Individual objection concerning specified URLs containing personal data | Case-specific review, not a universal site-wide block |
Apple’s documentation for Applebot and Applebot-Extended is the basis for these distinctions.
Does blocking Applebot-Extended remove a site from Apple search?
No, not necessarily. Apple says a site that disallows Applebot-Extended can still appear in Apple search results if ordinary Applebot access is permitted. In practical terms, a publisher can express “index us, but do not use our crawled content to train foundation models.”
Do these 3 things before closing this tab:
1Scan for outdated or missing drivers - takes under a minute2Repair Windows errors before they cause bigger problems3Fix the driver behind crashes, sound loss and screen glitchesThat does not mean search access and AI use are identical. Apple separately documents that searchable pages may provide additional context for generated answers, depending on accessibility signals and the product feature involved. Search indexing, answer context and model training are different uses.
Rank #2
- BUILT FOR COLLEGE. AND BEYOND — MacBook Air with the M5 chip packs blazing speed and powerful AI capabilities into an incredibly portable design. And with up to 18 hours of battery life,* this thin and light powerhouse is ready to take on almost any major, just about anywhere.
- TEAR THROUGH TOUGH ASSIGNMENTS — With its faster CPU and unified memory, the M5 chip delivers even more performance and fluidity across apps, making multitasking and creative workflows smooth and responsive. A powerful Neural Engine and next-generation GPU with Neural Accelerators give you a powerful platform for AI.
- MAKE QUICK WORK OF YOUR TO-DO LIST — Apple Intelligence helps you write, express yourself, and get things done effortlessly — whether it’s for school or everyday life. With groundbreaking privacy protections, it gives you peace of mind that no one else can access your data — not even Apple.*
- UP TO 18 HOURS OF BATTERY LIFE — MacBook Air delivers incredible battery life with amazing performance, so you can power through a full day of classes without worrying about plugging in.
- A BRILLIANT 13.6-INCH DISPLAY* — The gorgeous Liquid Retina display on MacBook Air supports 1 billion colors, making photos and videos pop with rich contrast and sharp detail, and text appears supercrisp. So everything — from class presentations to movies to games — looks truly stunning.
What if a site blocks ordinary Applebot?
A broad rule looks like this:
User-agent: Applebot
Disallow: /Apple says Applebot follows standard robots.txt directives. This rule can stop Apple from crawling the site for search-related purposes, which may reduce visibility in Spotlight, Siri suggestions, Safari-related discovery and other Apple surfaces. It should not be used casually by a publisher that depends on Apple traffic.
A mixed policy makes the intended separation explicit:
User-agent: Applebot
Allow: /
User-agent: Applebot-Extended
Disallow: /The result still depends on correct root-level file delivery, rule syntax, crawler compliance, server and CDN behavior, and Apple’s processing of the directive.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallOutdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWhy publishers object to AI-training use
Public availability is not the same as consent to every commercial use. Reported objections vary, but commonly include:
Rank #3
- BUILT FOR COLLEGE. AND BEYOND — MacBook Air with the M5 chip packs blazing speed and powerful AI capabilities into an incredibly portable design. And with up to 18 hours of battery life,* this thin and light powerhouse is ready to take on almost any major, just about anywhere.
- TEAR THROUGH TOUGH ASSIGNMENTS — With its faster CPU and unified memory, the M5 chip delivers even more performance and fluidity across apps, making multitasking and creative workflows smooth and responsive. A powerful Neural Engine and next-generation GPU with Neural Accelerators give you a powerful platform for AI.
- MAKE QUICK WORK OF YOUR TO-DO LIST — Apple Intelligence helps you write, express yourself, and get things done effortlessly — whether it’s for school or everyday life. With groundbreaking privacy protections, it gives you peace of mind that no one else can access your data — not even Apple.*
- UP TO 18 HOURS OF BATTERY LIFE — MacBook Air delivers incredible battery life with amazing performance, so you can power through a full day of classes without worrying about plugging in.
- A BRILLIANT 15.3-INCH DISPLAY* — The gorgeous Liquid Retina display on MacBook Air supports 1 billion colors, making photos and videos pop with rich contrast and sharp detail, and text appears supercrisp. So everything — from class presentations to movies to games — looks truly stunning.
- Compensation: A publisher may not want valuable reporting used to improve a commercial model without payment.
- Traffic and attribution: AI-generated answers can satisfy a question without sending a reader to the original site.
- Copyright uncertainty: Publishers may dispute whether copying and processing articles for model training is lawful.
- Competition: A company may view an AI platform as a competitor, even while licensing content to another company.
- Consent and control: A machine-readable rule is a quick way to state a preference across a large archive.
- Data quality and brand risk: Journalism, forums and social posts are useful training material but can contain sensitive information or context a publisher never intended to repurpose.
These are strategic motivations, not proof that every organization named in the 2024 coverage acted for the same reason.
Why allowing one AI company does not mean approving all AI training
A licensing deal with OpenAI, for example, can coexist with a block on Apple. A publisher may authorize a defined archive, negotiate attribution and payment, require security or deletion terms, or simply prefer one commercial relationship over another. “Pro-AI” and “anti-AI” are too crude to describe these choices; the issue is usually which use is authorized, by whom and under what terms.
Apple’s current explanation of its data practices
In its June 8, 2026 support information, Apple says it uses publicly available web information, including material crawled by Applebot, to help train the foundation models behind Apple Intelligence, Services and Developer Tools. Apple also describes other sources: licensed or purchased third-party datasets, open-source data, user studies and synthetic data. See Apple’s support page and its training-data disclosure.
Apple says Applebot does not crawl sites requiring login credentials or protected by a paywall. It also says it applies filters intended to remove certain personal information and low-quality or profane material, and that private user data and user interactions are not used to train its foundation models. Those are Apple’s published representations, not an independently verified inventory of every training source.
Rank #4
- BUILT FOR COLLEGE. AND BEYOND — MacBook Air with the M5 chip packs blazing speed and powerful AI capabilities into an incredibly portable design. And with up to 18 hours of battery life,* this thin and light powerhouse is ready to take on almost any major, just about anywhere.
- TEAR THROUGH TOUGH ASSIGNMENTS — With its faster CPU and unified memory, the M5 chip delivers even more performance and fluidity across apps, making multitasking and creative workflows smooth and responsive. A powerful Neural Engine and next-generation GPU with Neural Accelerators give you a powerful platform for AI.
- MAKE QUICK WORK OF YOUR TO-DO LIST — Apple Intelligence helps you write, express yourself, and get things done effortlessly — whether it’s for school or everyday life. With groundbreaking privacy protections, it gives you peace of mind that no one else can access your data — not even Apple.*
- UP TO 18 HOURS OF BATTERY LIFE — MacBook Air delivers incredible battery life with amazing performance, so you can power through a full day of classes without worrying about plugging in.
- A BRILLIANT 13.6-INCH DISPLAY* — The gorgeous Liquid Retina display on MacBook Air supports 1 billion colors, making photos and videos pop with rich contrast and sharp detail, and text appears supercrisp. So everything — from class presentations to movies to games — looks truly stunning.
Paywalls and AI-generated answers are a separate control
Apple’s UK documentation says publishers can mark non-free pages with Schema.org’s isAccessibleForFree: false:
<script type="application/ld+json">
{
"@context": "https://schema.org",
"isAccessibleForFree": false
}
</script>Apple says such pages may remain eligible for search results but will not be used as additional context when Apple AI models generate output in Apple products and services. This markup addresses answer context; it is not a substitute for an Applebot-Extended training preference or an Applebot crawl block. Details are in Apple’s paywall guidance.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Can an individual ask Apple to remove a page?
Apple provides an Apple Intelligence Privacy Inquiries process for URLs containing an individual’s personal data. The requester must identify specific URLs. Apple says it may decline in some circumstances, including when another person’s rights are implicated or a request is impractical or unreasonable.
The Tool Desk
Outbyte Driver Updater FREEScan for outdated or missing drivers - takes under a minuteDriver Scan →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →This is different from a publisher’s robots.txt policy. A robots rule is a site-level crawler instruction; the privacy form is a URL-specific request. Neither should be treated as a guaranteed deletion of every historical copy or dataset.
Best Value
- BUILT FOR COLLEGE. AND BEYOND — MacBook Air with the M5 chip packs blazing speed and powerful AI capabilities into an incredibly portable design. And with up to 18 hours of battery life,* this thin and light powerhouse is ready to take on almost any major, just about anywhere.
- TEAR THROUGH TOUGH ASSIGNMENTS — With its faster CPU and unified memory, the M5 chip delivers even more performance and fluidity across apps, making multitasking and creative workflows smooth and responsive. A powerful Neural Engine and next-generation GPU with Neural Accelerators give you a powerful platform for AI.
- MAKE QUICK WORK OF YOUR TO-DO LIST — Apple Intelligence helps you write, express yourself, and get things done effortlessly — whether it’s for school or everyday life. With groundbreaking privacy protections, it gives you peace of mind that no one else can access your data — not even Apple.*
- UP TO 18 HOURS OF BATTERY LIFE — MacBook Air delivers incredible battery life with amazing performance, so you can power through a full day of classes without worrying about plugging in.
- A BRILLIANT 13.6-INCH DISPLAY* — The gorgeous Liquid Retina display on MacBook Air supports 1 billion colors, making photos and videos pop with rich contrast and sharp detail, and text appears supercrisp. So everything — from class presentations to movies to games — looks truly stunning.
Choosing a policy as a site owner
Use Applebot-Extended only when
- You want to preserve Apple search visibility.
- Your principal objection is foundation-model training.
- You prefer the least disruptive technical change.
Block Applebot when
- You do not want Apple crawling the site at all.
- Apple search traffic is immaterial to your business.
- You have accepted the loss of Apple discovery and snippet visibility.
Use contracts or licensing when
- You require payment, attribution, audit rights, usage limits or deletion obligations.
- You want to authorize selected archives, products or territories rather than everything.
Use access controls when
- Material must not be publicly available.
- You need authentication, subscriptions or a technical barrier stronger than a crawler preference.
Implementation checks and failure modes
- Place a file named
robots.txtat the site root and verify it is publicly served. - Use the exact user-agent spelling shown above; a rule for Applebot-Extended does not govern other AI crawlers.
- Check server logs and a crawler-testing workflow after publishing the change.
- Account for CDN caches, redirects, regional domains and staging sites.
- Remember that disallowing a crawler does not erase material already collected or stop another crawler from obtaining the same public page.
robots.txt is a technical instruction for compliant crawlers, not authentication, encryption or a universal legal prohibition. Apple says it respects the directives, but the file alone does not resolve copyright, contract or privacy disputes.
What this means for the open web
The dispute is moving toward a layered access model: discovery crawling, snippets, AI-answer context, foundation-model training, licensed archives and outright blocking are no longer one decision. Publishers are weighing reach against leverage, compensation, privacy and control. Allowing Applebot may preserve distribution while still exposing content to some search or answer features; blocking Applebot-Extended is narrower, but it does not govern other companies’ crawlers or undo past collection.
Frequently Asked Questions
Is Applebot-Extended a crawler that downloads pages?
No. Apple says Applebot-Extended does not crawl webpages; it communicates how Apple may use data collected by Applebot.
Does a robots.txt rule make scraping illegal?
No. It expresses a preference to compliant crawlers. It does not by itself settle copyright, contract, privacy or other legal questions.
The Bottom Line
For most publishers seeking a targeted opt-out, User-agent: Applebot-Extended with Disallow: / is the narrowest documented choice: it can restrict Apple foundation-model training without necessarily sacrificing Apple search visibility. Blocking ordinary Applebot is a much bigger distribution decision, and neither rule replaces licensing negotiations, access controls or legal advice.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
