Recommended Free Tools
The change is a shift toward classifying automated requests by what they do—search, train AI systems, or act as agents—and giving site owners more purpose-specific controls. It does not mean every crawler has a single purpose, or that robots.txt has become a technical block. Cloudflare’s September 2026 controls illustrate the distinction: a site can publish a preference about AI training, or separately block crawlers, with different consequences for search access.
What changed in how bots are described?
Cloudflare now groups automated activity into three behavior categories: Search, Training, and Agent. The important shift is from treating a crawler as if its name alone tells you what it does to considering the purpose of its requests. A single crawler can have more than one behavior, so these categories are not necessarily mutually exclusive labels for separate bots.
As an Amazon Associate I earn from qualifying purchases.
This is Cloudflare’s classification and control approach, not a universal web standard. Other services may use different categories, controls, or definitions.
| Behavior | What it means here | What a site owner should consider |
|---|---|---|
| Search | Crawling associated with search | Whether blocking a crawler could affect search access or discoverability |
| Training | Requests associated with AI training | Whether to express a preference against training use while allowing other activity |
| Agent | Automated activity classified as agent behavior | Whether to allow or control that behavior under the site’s policy |
Can you block AI training bots without blocking search?
Sometimes, but it depends on the crawler and the control you choose. In Cloudflare’s September 2026 controls, a full block can apply to mixed-use crawlers such as Applebot, Bingbot, and Googlebot. Because those crawlers also serve search functions, a full block can affect search access as well as training-related access.
#1 Best Overall
- Server-Class Home Server Built for 24/7 Workloads - Designed as a purpose-built home server rather than general-purpose SBCs, Mini PCs, entry NAS systems, or routing-only devices. As a compact, pocket-sized single board server platform, ZimaBoard 2 1664 combines x86 architecture, quad-core performance up to 3.6GHz, 16GB DDR5 memory, and 64GB eMMC storage for reliable always-on home servers, homelabs, and self-hosted workloads.
- PCIe 3.0 x4 Expansion for Real Server Builds - Built as a server-class platform with native PCIe expansion, ZimaBoard 2 features a full PCIe 3.0 x4 slot for high-speed, low-latency upgrades beyond USB-based limitations. Supports 10GbE NICs, NVMe adapters, GPUs, and AI accelerators to build scalable home servers, homelabs, and advanced self-hosted systems—offering greater expansion flexibility than typical SBCs, Mini PCs, and entry-level NAS devices.
- Native Dual SATA & Dual 2.5GbE Networking - Built with server-class storage and networking I/O, ZimaBoard 2 integrates dual SATA ports for direct HDD/SSD connectivity and dual 2.5GbE Ethernet for high-throughput, low-latency networking. This architecture enables reliable DIY NAS, fast storage, routing, and multi-service home server deployments—while avoiding USB-based performance constraints common in ARM SBCs, Raspberry Pi–based setups, Mini PCs, and entry-level NAS devices.
- ZimaOS Preinstalled + Wide OS Compatibility - Comes preinstalled with ZimaOS for a clean, ad-free private cloud experience—centralized file dashboard, automatic backups, P2P downloads, private photo/video sharing, 500+ plug-ins, and secure on-device AI that keeps your data at home. Also supports TrueNAS, Proxmox, Debian, Ubuntu Server, pfSense, OpenWrt, and Linux containers, making it perfect for Plex media servers, Pi-hole, firewalls, backups, Docker labs, home-cloud services, and multi-service deployments.
- All-in-One NAS, Router, Docker & Homelab Server - Replace multiple devices with one low-power. ZimaBoard 2 can serve as a NAS, router, Docker host, firewall, media server, or homelab node—delivering a flexible, open alternative to ARM SBCs, Mini PCs, and entry-level NAS systems.
Cloudflare describes its Disallow AI Training preference as the option for keeping accountable mixed-use crawlers available for search while expressing a preference against AI training. That preference is not the same as a technical block. Cloudflare’s blocking controls operate separately; robots.txt communicates a published preference rather than enforcing access restrictions by itself.
What if a crawler does more than one thing?
A mixed-use crawler makes a simple allow-or-block decision harder. Blocking the crawler by name may stop more than the behavior a site owner wants to prevent. A purpose-based preference can be more selective where the platform supports it, but the actual effect depends on the crawler’s behavior and the control being applied.
Rank #2
- 【Advanced Home Data & Media Hub】For advanced home users who need phone backup, file storage, and centralized data management. Centralize family photos, 4K videos, movies, computer backups, and personal files in one place while running multiple apps for home entertainment and everyday data management. Suitable for households with growing digital libraries and multiple NAS use cases.
- 【Built for Creators, Media Servers & Advanced Apps】Powered by the Intel N100 Quad-Core CPU, 8GB DDR5 RAM, 2.5GbE networking, and dual M.2 NVMe slots, DXP2800 handles large files and heavier workloads with ease. Run Docker, virtual machines, and media server applications compatible with Plex—ideal for content creators, tech enthusiasts, and advanced home users managing 4K videos, RAW photos, personal media libraries, and multiple NAS apps.
- 【Up to 80TB for Growing Digital Libraries】 Supports up to 80TB of storage using two HDD bays and two M.2 NVMe SSD slots for family photos, movies, RAW photos, 4K videos, work files, and device backups. AI photo management supports recognition of people, objects, scenes, and locations, album organization, and duplicate photo detection. HDDs and SSDs are not included.
- 【AI-powered Home Surveillance】Turn DXP2800 into a centralized home surveillance hub by connecting compatible network cameras and storing recordings locally on your NAS. AI-powered features include Face Recognition, People Detection, and Pet Detection, helping advanced home users review important events more efficiently while managing home surveillance and personal data in one place.
- 【One data Center Across Your Devices】Keep files from desktops, laptops, phones, tablets, and other devices together instead of scattered across cloud accounts and external drives. Access, back up, organize, and share data across Windows, macOS, Android, iOS, web browsers, and compatible smart TVs—ideal for creators and advanced home users working across multiple devices.
Search controls may also apply at a broader level than a site owner expects. Google documents that Googlebot Smartphone and Googlebot Desktop use the same robots.txt product token. That means a robots.txt rule using that token does not distinguish those two crawler subtypes.
Do these 3 things before closing this tab:
1Repair Windows errors before they cause bigger problems2Scan for outdated or missing drivers - takes under a minute3Clear out junk files and repair common Windows errorsHow much crawler activity is involved?
Cloudflare’s worldwide verified-bot data for 2025 gives a sense of activity on its own network. These percentages are shares of Cloudflare-verified bot traffic, not a census of all web traffic or all crawlers.
Rank #3
- Server-Class Home Server Built for 24/7 Workloads - Designed as a purpose-built home server rather than general-purpose SBCs, Mini PCs, entry NAS systems, or routing-only devices. As a compact, pocket-sized single board server platform, ZimaBoard 2 832 combines x86 architecture, quad-core performance up to 3.6GHz, 8GB DDR5 memory, and 32GB eMMC storage for reliable always-on home servers, homelabs, and self-hosted workloads.
- PCIe 3.0 x4 Expansion for Real Server Builds - Built as a server-class platform with native PCIe expansion, ZimaBoard 2 features a full PCIe 3.0 x4 slot for high-speed, low-latency upgrades beyond USB-based limitations. Supports 10GbE NICs, NVMe adapters, GPUs, and AI accelerators to build scalable home servers, homelabs, and advanced self-hosted systems—offering greater expansion flexibility than typical SBCs, Mini PCs, and entry-level NAS devices.
- Native Dual SATA & Dual 2.5GbE Networking - Built with server-class storage and networking I/O, ZimaBoard 2 integrates dual SATA ports for direct HDD/SSD connectivity and dual 2.5GbE Ethernet for high-throughput, low-latency networking. This architecture enables reliable DIY NAS, fast storage, routing, and multi-service home server deployments—while avoiding USB-based performance constraints common in ARM SBCs, Raspberry Pi–based setups, Mini PCs, and entry-level NAS devices.
- ZimaOS Preinstalled + Wide OS Compatibility - Comes preinstalled with ZimaOS for a clean, ad-free private cloud experience—centralized file dashboard, automatic backups, P2P downloads, private photo/video sharing, 500+ plug-ins, and secure on-device AI that keeps your data at home. Also supports TrueNAS, Proxmox, Debian, Ubuntu Server, pfSense, OpenWrt, and Linux containers, making it perfect for Plex media servers, Pi-hole, firewalls, backups, Docker labs, home-cloud services, and multi-service deployments.
- All-in-One NAS, Router, Docker & Homelab Server - Replace multiple devices with one low-power, fanless system. ZimaBoard 2 can serve as a NAS, router, Docker host, firewall, media server, or homelab node—delivering a flexible, open alternative to ARM SBCs, Mini PCs, and entry-level NAS systems.
| Category or crawler | Share or change reported | Scope |
|---|---|---|
| Search crawlers | 40% | Cloudflare verified-bot traffic worldwide in 2025 |
| AI crawlers | 20% | Cloudflare verified-bot traffic worldwide in 2025 |
| Googlebot | Over 28% | Share of Cloudflare verified-bot traffic in 2025 |
| GPTBot | About 7.5% | Share of Cloudflare verified-bot traffic in 2025 |
| Bingbot | 6% | Share of Cloudflare verified-bot traffic in 2025 |
| ChatGPT-User requests | Peak reached as much as 16 times the beginning-of-year level | Cloudflare-measured request volume in 2025, not a count of users |
| OAI-SearchBot requests | Late-October peak reached about 5 times the beginning-of-year level | Cloudflare-measured request volume in 2025, not a count of users |
What should a site owner check before changing bot controls?
- Identify the goal. Decide whether the concern is AI training, search access, agent activity, or a combination.
- Check for mixed use. A crawler’s name may cover more than one behavior, so a full block may have broader effects than intended.
- Separate preference from enforcement. Treat robots.txt as a published preference; use a blocking control when the goal is to restrict access.
- Review the actual rule and its scope. A rule keyed to a shared token may cover multiple crawler subtypes.
Cloudflare’s September 2026 recommendations for new domains differ according to whether a site is ad-supported. They are vendor recommendations, not mandatory settings, and customers can change them. The choice should reflect a site’s goals rather than treating a default as a universal policy.
Cloudflare co-founder and CEO Matthew Prince described the rationale this way: “This is how we make the Internet better: preserving the openness that makes search valuable while giving the people and businesses behind the web meaningful control over how their work is used.”
Quick Recap
Rank #4
- SDI Video Inputs: 1
- SDI Video Outputs: 1 x loop out, 1 x monitor out.
- SDI Rates: 1.5G, 3G, 6G, 12G
- HDMI Video Outputs: 1 x monitor out
- Webcam Output: 1 x Type USB-C
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Quick wins for a faster PC:
Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Clear out junk files and repair common Windows errorsFree Scan →
