Log File Bot Traffic Cost Analyzer examines server logs to break down AI bot traffic, estimate associated costs, quantify crawl budget, and produce edge rules. The project describes its processing as client-side, with logs staying on the user's machine. It says it makes no XHR, WebSocket, or sendBeacon calls; the only stated network request is for a demo file that the user chooses to load. The analyzer groups traffic using 127 bot signatures and 18 browser patterns, with bandwidth, IP, and TTFB details by category. Verification draws on vendor IP data, TLS, cloud ASN heuristics, and behavior; heuristic findings are marked low confidence, and optional DoH reverse DNS is listed. Cost estimates use measured bytes and CDN pricing presets, with copyable rules for Cloudflare, Fastly, and AWS. Inputs include JSON, JSONL/NDJSON, Apache Combined, Nginx, W3C Extended, AWS ALB, Cloudflare Logpush, and gzip files. Browser analysis under 500 MB is sampled triage; CLI analysis from 500 MB to 50 GB is described as exact, with BigQuery for 50 GB and above. It is free and MIT-licensed.
Who it is for
It suits users who need private log triage or want to inspect crawler traffic and crawl waste. The project specifically positions it for cases where logs cannot leave the machine.
What is good
- Client-side processing keeps logs on the user's machine.
- Classifies traffic with 127 bot signatures and 18 browser patterns.
- Generates copyable Cloudflare, Fastly, and AWS rules.
- Exports CSV and PDF reports.
What to know first
- Browser analysis under 500 MB is sampled triage.
- CLI analysis is described as exact only from 500 MB to 50 GB.
- BigQuery is specified for 50 GB and above.
Verdict
This free tool covers bot traffic, estimated costs, and several log formats, with an emphasis on local processing. Note the different analysis paths and scale limits for browser, CLI, and BigQuery use.
Get started with Log File Bot Traffic Cost Analyzer
- Open the project website on GitHub.
- Use the browser version deployed on Cloudflare Pages, or its GitHub Pages backup.
- Upload supported log files for browser analysis below 500 MB.
- For 500 MB to 50 GB exact analysis, use the CLI route documented by the project.
- For 50 GB and above, use the listed BigQuery route.
- Use the documented Docker run command if deploying with Docker.
What the free plan stops at
The Free plan describes browser analysis under 500 MB as sampled triage, CLI analysis from 500 MB to 50 GB as exact, and BigQuery for 50 GB and above.
Questions about Log File Bot Traffic Cost Analyzer
How much does Log File Bot Traffic Cost Analyzer cost?
The listed Free plan costs 0.00 USD per free. The project describes the software as open source and MIT-licensed.
What platforms does it support?
The listed platforms are Linux, self-hosted, and web. Cloudflare Pages is identified as the primary deployment and GitHub Pages as a backup.
Does it keep uploaded logs private?
The project describes the tool as client-side and says logs do not leave the user's machine. It says the only network request is for a demo file the user chooses to load.
What log sizes can it analyze?
Browser analysis under 500 MB is sampled triage. CLI analysis from 500 MB to 50 GB is described as exact, with BigQuery for 50 GB and above.
What can it export?
It lists CSV and PDF downloads, as well as generated robots.txt, llms.txt, and crawlers.json policies.
What log formats are supported?
Supported inputs include JSON, JSONL/NDJSON, Apache Combined, Nginx, W3C Extended, AWS ALB, Cloudflare Logpush, and gzip files.
Log File Bot Traffic Cost Analyzer plans and pricing
All plansCompared on SEO log file analysis tools
- Log input methods
- upload
- Crawler segmentation
- Yes
- Crawl waste reports
- Yes
