cl0q says its crawler node is now open source, allowing operators to run software that discovers websites, fetches their homepages and sends what it finds to cl0q’s index. The project’s October 6, 2026 announcement outlines a Docker-based setup and describes limits intended to control how the crawler visits sites. Those details are claims in the announcement; the linked repository was not independently reviewed here.
What cl0q’s crawler node does
In its October 6, 2026 announcement, cl0q describes the node as a way to discover websites, fetch their homepages and send the resulting information to cl0q’s index. That is a narrower scope than recursively crawling every page on a site: the post describes homepage fetching, not a general-purpose site scraper.
As an Amazon Associate I earn from qualifying purchases.
cl0q says the software does not scrape personal data or build profiles. This describes the project’s stated purpose and behavior, not an independent audit of its code. The announcement says cl0q had mapped 38.5 million domains in 2026 and was working toward 100 million; those are figures reported by the publisher, not independently verified measurements.
What the announcement says about crawl safeguards
cl0q says the safeguards below are hardcoded into the node. The announcement does not independently demonstrate how they are implemented, and the repository was not available for review here.
#1 Best Overall
- Includes Raspberry Pi 5 with 2.4Ghz 64-bit quad-core CPU (8GB RAM)
- Includes 128GB Micro SD Card pre-loaded with 64-bit Raspberry Pi OS, USB MicroSD Card Reader
- CanaKit Turbine Black Case for the Raspberry Pi 5
- CanaKit Low Noise Bearing System Fan
- Mega Heat Sink - Black Anodized
- The crawler identifies the node in its User-Agent.
- It accesses public internet destinations only.
- It checks a site’s robots.txt before each crawl and follows any crawl-delay it specifies.
- It leaves at least one second between requests to the same server.
- It fetches a site’s homepage no more than once every 120 days.
These are the project’s stated operating limits, not independently measured performance or verified guarantees.
What you need before setup
The announcement lists Docker and an online machine as prerequisites. It does not specify an operating system, minimum CPU or memory, storage, bandwidth, or expected hosting cost. It also does not say that cloud hosting is required; an always-on machine capable of running Docker is the stated need.
Rank #2
- Includes Raspberry Pi 4 4GB Model B with 1.5GHz 64-bit quad-core CPU (4GB RAM)
- Includes Pre-Loaded 32GB EVO+ Micro SD Card (Class 10), USB MicroSD Card Reader
- CanaKit Premium High-Gloss Raspberry Pi 4 Case with Integrated Fan Mount, CanaKit Low Noise Bearing System Fan
- CanaKit 3.5A USB-C Raspberry Pi 4 Power Supply (US Plug) with Noise Filter, Set of Heat Sinks, Display Cable - 6 foot (Supports up to 4K60p)
- CanaKit USB-C PiSwitch (On/Off Power Switch for Raspberry Pi 4)
Setup commands from the announcement
The following sequence is transcribed from cl0q’s post and has not been run or checked against the repository’s current files. Confirm the instructions in the project repository before relying on them.
Quick wins for a faster PC:
Clear out junk files and repair common Windows errorsFree Scan →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →git clone https://codeberg.org/cl0qsearch/crawler-node.git
cd crawler-node
cp .env.example .env
docker compose build
docker compose run --rm cl0q-node enroll --contact [email protected]
docker compose run --rm preflight
docker compose up -d- Clone the project and enter its directory. The first two commands download the repository and move into the resulting
crawler-nodefolder. - Create the environment file. Copying
.env.exampleto.envcreates a local configuration file. The announcement does not detail which settings an operator should change. - Build the containers.
docker compose buildprepares the services described by the project. - Enroll the node. Replace
[email protected]with a contact email and run the enrollment command. According to cl0q, enrollment registers the node and returns a token. - Run preflight checks. The
preflightcommand is described as checking that the setup works before crawling begins. - Start the services.
docker compose up -dstarts them in the background.
Because the repository’s current files and commands could not be checked, treat this as the announcement’s setup outline rather than a verified, current installation guide. If a command fails or the project’s README differs, follow the repository’s latest instructions instead.
Rank #3
- Not including the Raspberry Pi 5 (8GB), the Crowpi advanced version comes with the Raspberry Pi 5
- ELECROW Black Case for the Raspberry Pi 5, CrowPi is equipped with a 9-inch HD touchscreen along with a camera; All the regular components used in DIY electronics are packed into the CrowPi development board, such as LCD, LED matrix, buzzer, light sensor, PIR sensor, ultrasonic sensor, IR sensor, etc
- Raspberry Pi Sensors: The Crowpi raspberry pi 5 programming kit is jam-packed with lots of buttons such as 19 different sensors in a tidy easy to use package; You don't have to wait and wire things
- Build Quality: Solid ABS shell and well made components in one place make it strong and convenient to travel
- Programming Lessons: This raspberry pi 5 learning kit ships with step by step instructions and provides 21 lessons to take you through identifying components reading code and running it in the terminal
How the node is organized
cl0q describes the node as five services and one Postgres database, all running locally on the operator’s machine, with zero public ports. The announcement does not name the services or explain their individual roles, so it does not support a more detailed architecture breakdown.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Release status and stated plans
The announcement calls the release v0.1.0 and says the code is licensed under MIT. It describes a GitHub mirror, additional repositories and performance work as plans for the future at the time of publication—not as completed releases. The available information here does not establish their current status.
Quick Recap
Best Value
- 【What you Get】You will get 1*Pi 5 8GB Single Board,1*RasTech Case,1*Active Cooler,1*Screwdriver,1*Installation instructions,12-month free warranty, lifetime service, 24-hour prompt and friendly response.
- 【More Connectors】There are two USB 3.0 ports(5Gbps simultaneously) and two USB 2.0 ports, which triple total bandwidth ,support any combination of up to two cameras or displays. Peak SD card performance is doubled through support for the SDR104 high-speed mode. It provides a smooth desktop experience for you. Offer Gigabit Ethernet and a PCIe interface, along with dual-band Wi-Fi and Bluetooth 5.0/BLE wireless capability. The RasTech Pi 5 Kit use the new 27W 5.1V 5A USB-C power connector.
- 【 Support Dual 4Kp60 Display 】Each of the two microHDMI sockets can control a 4K display at 60 Hertz, now support HDR, offering super HD video for media streaming projects. RPi 5 is the first RPi model that comes with a PCI Express port (PCIe 2.0 x1 with 500 MB/s) to attach SSDs (requires separate M.2 HAT).
- 【 Excellent Chips And Applications】Pi 5 is a full-size Pi computer using silicon built in-house at Pi. The RP1 “southbridge” provides the bulk of the I/O capabilities for Pi 5. Pi 5 is more friendly and convenient in the development of Internet of Things, Web development, machine identification, automatic control and other electronic equipment applications and network.
- 【 Faster CPU, Better GPU 】 Pi 5 features a Broadcom BCM2712 64-bit quad-core Arm Cortex-A76 processor running at 2.4GHz, it delivers a 2–3× increase in CPU performance relative to RaspberryPi 4. The 800MHz VideoCore VII GPU is compatible to OpenGL ES 3.1 and Vulkan 1.2, substantial uplift in graphics performance. Pi 5 Offers lightning-fast CPU speed, a PCI Express interface, a Real Time Clock (RTC) and a power button and runs significantly cooler than Pi 4.
Rank #4
- Fully assembled for plug-and-play operation
- Includes Raspberry Pi 5 with 8GB RAM
- 256 GB PCIe Pi NVMe SSD (Pre-loaded with Pi 64-Bit OS)
- M.2 HAT+
- CanaKit Turbine Black Case for the Pi 5
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
