Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Repair Windows errors before they cause bigger problemsFix Now →Not automatically in every case. In the United States, a website’s no-scraping notice is not a universal statute or court order. Whether an AI company can lawfully continue collecting material depends on the access method, any applicable contract, the type of material and its use, the legal claim, and the jurisdiction. A Ninth Circuit ruling says public access alone does not make scraping unauthorized under the federal Computer Fraud and Abuse Act (CFAA); it does not grant blanket permission to copy or use public content.
What a no-scraping notice does—and does not—mean
A website can communicate an objection in several ways: a machine-readable instruction such as robots.txt, terms of service, a click-through agreement, or a cease-and-desist letter. Those signals are not interchangeable. Their legal effect depends on what the company agreed to, what the crawler accessed, and which legal theory is being considered.
| # | Preview | Product | Price | |
|---|---|---|---|---|
| 1 |
|
The Proxy Playbook: The Complete Guide to Proxy Servers: How to Source, Test, and Scale Residential,... | $29.95 | Buy on Amazon |
| 2 |
|
How to Host your own Web Server | $15.60 | Buy on Amazon |
As an Amazon Associate I earn from qualifying purchases.
A notice may be relevant evidence of the site operator’s objection, but it does not by itself settle whether scraping is lawful. Nor does the fact that a page is publicly viewable settle whether copying it, retaining it, or using it to train or operate an AI system infringes rights or breaches an agreement.
Free tools Windows power users keep installed
One-click scans. No signup required.
What the Ninth Circuit decided about public pages and the CFAA
In its April 18, 2022 opinion in hiQ Labs, Inc. v. LinkedIn Corp., the U.S. Court of Appeals for the Ninth Circuit considered LinkedIn’s argument that hiQ’s continued scraping of public profiles after a cease-and-desist letter violated the CFAA. The court wrote that “the concept of ‘without authorization’ does not apply to public websites.” That statement concerns the CFAA and access to public websites; it is not a general ruling that anyone may scrape any public page for any purpose.
The distinction is whether information is available to the general public or sits behind an authorization gate. A public page is not equivalent to a login-protected page or a restricted endpoint. The opinion also recognized that website operators may have other legal claims, including copyright infringement, misappropriation, unjust enrichment, conversion, breach of contract, breach of privacy, and state-law trespass to chattels. It noted that operators can use technological self-help against harmful intrusions or attacks.
How the notice and access method affect different claims
| Issue | What matters | What the cited decisions establish |
|---|---|---|
| CFAA authorization | Whether the material was public or behind an access restriction, and the jurisdiction. | The Ninth Circuit’s 2022 hiQ opinion says public-web access does not become “without authorization” under the CFAA merely because the site objects. It does not resolve other claims. |
| Contract | Whether terms applied and formed an agreement, who accepted them, and what they prohibit. | The hiQ opinion records that LinkedIn’s User Agreement restricted scraping and explains that contract claims can remain relevant apart from the CFAA issue. |
| DMCA Section 1201 | Whether a technological measure effectively controls access to copyrighted work and was circumvented. | In a December 18, 2025 order, the Southern District of New York (SDNY) held, on the allegations before it, that robots.txt directives did not effectively control access for the proposed Section 1201 claim. |
| Copyright and downstream use | What was copied, how much, how it was retained or used, and the facts relevant to the particular claim. | The court rulings described here do not establish a categorical answer about whether AI training is fair use or infringement. The U.S. Copyright Office’s AI-training analysis remains an important source of evolving guidance. |
| Privacy and other state-law claims | The information collected, the parties and conduct involved, and applicable state law. | The hiQ opinion identifies possible claims beyond the CFAA; it is not a fifty-state survey or a decision resolving every such claim. |
Why robots.txt is different from a technical access barrier
A robots.txt file tells compliant crawlers what a site operator permits or requests. It is not necessarily a technical mechanism that prevents access. In its December 18, 2025 order denying Ziff Davis leave to file a proposed amended complaint, an SDNY court concluded that the alleged robots.txt directives did not effectively control access to the publishers’ copyrighted works for purposes of the DMCA’s Section 1201 claim. The court reasoned that a bot could access the pages without using credentials or defeating a technical gate simply by disregarding the instruction. It compared the instruction to a request that visitors “keep off the grass.”
That was a district-court pleading ruling in a particular case, addressing a particular claim. It does not mean that ignoring robots.txt is always lawful, or decide whether the same conduct violates a contract, copyright law, privacy law, or state law. A written instruction, an agreed contractual term, and a technical control such as a login requirement are different facts and may matter in different ways.
Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Repair Windows errors before they cause bigger problems3Scan for outdated or missing drivers - takes under a minuteCopyright and AI training are separate questions
Accessing a page and copying its contents are not the same legal question. Even if a CFAA claim based solely on access to public pages does not apply under the Ninth Circuit’s hiQ analysis, that does not determine whether making or keeping copies infringes copyright. The content, amount copied, purpose, retention, commercial context, and use of the material may be relevant to particular claims; no single factor resolves every case.
Rank #2
The U.S. Copyright Office launched an initiative examining copyrighted material used in AI training. On May 9, 2025, it released a pre-publication Part 3 on generative AI training and said a final version would follow without expected substantive changes to its analysis or conclusions. That status reflects ongoing legal and policy analysis, not a blanket determination that all AI training is fair use or that all training copies infringe.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.What a cease-and-desist letter changes
A letter makes the operator’s objection explicit and can affect the facts relevant to claims such as contract or state-law trespass, even where the Ninth Circuit’s public-page CFAA reasoning applies. Its effect still depends on matters such as whether a contract exists, what the crawler accessed, and what conduct followed. The letter does not, by itself, turn a public page into a login-protected resource or establish that every legal claim will succeed.
What AI companies and site operators should check
For a company collecting website material
- Identify whether the target is a public page, a login-protected page, a restricted endpoint, or content behind another technical access control.
- Review the site’s terms and determine whether the company accepted them or otherwise became bound by them; do not treat
robots.txtas a substitute for that contract analysis. - Assess the material and each intended use separately, including copying, retention, model training, and resulting outputs.
- Consider the relevant jurisdiction and legal theory. A Ninth Circuit CFAA decision and an SDNY Section 1201 pleading order do not create one nationwide rule for every kind of scraping.
- Take a cease-and-desist letter seriously as a notice of objection and have counsel assess the specific access, agreement, and use facts before continuing.
For a website operator
- Use clear terms and, where appropriate, a process that records assent if the goal is to form an agreement restricting access or reuse.
- Distinguish a crawler instruction from technical access controls; the SDNY ruling shows that a robots.txt directive alone was not treated as an effective Section 1201 access measure on the allegations in that case.
- Document the content at issue, the access method, the notice given, and any observed impact. The applicable claim and available remedies depend on those facts.
What the FTC says about AI companies’ promises to their own customers
The FTC’s January 2024 guidance says model-as-a-service companies must honor commitments made to customers, including commitments in website terms. It also warns that retaining or using consumer data for other purposes without clear notice and affirmative express consent can create legal risk. This guidance concerns an AI service’s own customer promises and data practices; it does not decide whether a third-party website’s no-scraping notice binds an outside crawler.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Where the answer remains uncertain
The decisions discussed here are limited in geography, claim, and posture: hiQ is a Ninth Circuit CFAA opinion, while the robots.txt ruling is an SDNY order on a proposed pleading under DMCA Section 1201. Neither is a nationwide ruling that resolves every scraping dispute. Copyright questions involving AI training also remain under active analysis. For a real dispute, the relevant facts include where the parties are, whether terms formed a contract, how access occurred, what was copied, and how the material was used. A company facing a demand or planning continued collection should get advice from qualified technology or intellectual-property counsel.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
