Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallHyperLogLog can estimate how many distinct values have appeared while keeping a compact sketch instead of every identifier. In Redis, that sketch uses up to 12 KB, plus key overhead, and Redis documents a 0.81% standard error. The title’s 100-billion figure is an illustrative scale, not a Redis benchmark: the result is an estimate, not an exact count, and the sketch cannot tell you whether a particular identifier was seen.
Why counting distinct values can take so much memory
To count unique visitors exactly, a system must distinguish an identifier it has already encountered from a new one. An exact set can do that, but it must retain the distinct values (or equivalent information sufficient to distinguish them), so its memory use grows as the set grows.
As an Amazon Associate I earn from qualifying purchases.
HyperLogLog trades away exactness and membership lookup. It summarizes hashed inputs in a small set of counters rather than retaining the original values. That makes it useful for aggregate questions such as “How many unique visitors did the site have today?” when a compact estimate is more useful than an exact list.
How the sketch turns rare patterns into a count
For intuition, imagine hashing each input into a well-distributed bit string. Part of the hash selects one of many registers; the remaining bits are examined for a leading-zero pattern. A register keeps its largest observed pattern. Long runs of leading zeroes are rare, so seeing one is evidence that many hashes have been processed.
#1 Best Overall
One register is noisy, so HyperLogLog combines information across many registers to reduce variation. The estimator uses a harmonic mean, with corrections for small and large ranges. This is an intuitive overview, not a full derivation of the algorithm.
What “12 kilobytes” means in Redis
The size is specific to Redis’s implementation, not a universal size for every HyperLogLog library. Redis documents a dense representation of 12,288 bytes: 16,384 six-bit counters plus a 16-byte header. Redis can also use a sparse representation that takes less space. The stated limit is up to 12 KB per sketch, plus a few bytes for the key, regardless of how many observations have been processed.
The “100 billion” in the title is an illustrative scale from the cited article, not a documented Redis performance test or benchmark. The memory claim describes the sketch’s bounded representation; it does not establish a measured throughput or accuracy result at that cardinality.
Do these 3 things before closing this tab:
1Clear out junk files and repair common Windows errors2Scan for outdated or missing drivers - takes under a minute3Repair Windows errors before they cause bigger problemsHow accurate is the estimate?
Redis documents a standard error of 0.81% for its HyperLogLog cardinality estimate. Standard error is not a hard maximum deviation for every result, nor a guarantee that each answer falls within 0.81% of the true count. It is Redis’s documented error statistic for its implementation.
Because the sketch does not retain the original identifiers, it cannot return the set, identify which values were counted, or answer whether a specific visitor or event ID has appeared.
Use HyperLogLog with Redis
Redis exposes three core commands for adding observations, estimating cardinality, and combining sketches:
| Task | Redis command | What it does |
|---|---|---|
| Add observations | PFADD key element... |
Adds values to the sketch stored at key. |
| Estimate cardinality | PFCOUNT key |
Returns an approximate count for the sketch. |
| Combine sketches | PFMERGE destination source... |
Merges source sketches into a destination sketch. |
For example, a service can add each observed visitor ID to a daily sketch with PFADD visitors:2026-10-04 visitor-id, then retrieve its approximate distinct count with PFCOUNT visitors:2026-10-04. The identifier here is an input to the sketch; Redis does not preserve it for later membership checks.
To roll up partitions or periods, merge their sketches with PFMERGE visitors:week visitors:day1 visitors:day2, or use multi-key PFCOUNT to estimate the union directly. A union estimate accounts for overlapping inputs approximately through the sketch; it does not require retaining all original IDs. Redis notes that multi-key PFCOUNT does more work and has different performance characteristics from counting one key.
Best Value
When an estimate is the wrong tool
| Decision | Exact hash set | HyperLogLog |
|---|---|---|
| Cardinality | Exact | Approximate |
| Check whether a particular item was seen | Yes | No |
| Memory as distinct values grow | Grows with retained values | Bounded by the implementation and configuration; Redis uses up to 12 KB per sketch |
| Combining partitions | Requires retaining values and applying a set/union strategy | Sketches can be merged or counted together in Redis |
| Good fit | Billing, payment deduplication, or coupon redemption when exactness matters | Aggregate estimates such as unique visitors or distinct search queries |
Choose an exact structure when a wrong count or a missed duplicate can trigger a charge, authorize a payment, or grant a benefit. HyperLogLog answers a different question: approximately how many distinct inputs occurred, without retaining the inputs themselves.
Sources and implementation context
Redis’s HyperLogLog documentation describes the data type and its commands. Its PFCOUNT command documentation explains the approximation, standard error, and memory representation. The illustrative 100-billion framing and algorithm overview appear in Athreya aka Maneshwar’s article on HyperLogLog.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
Free tools Windows power users keep installed
One-click scans. No signup required.
