October DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsWindows FixRecommendedWindows errors stealing your time? Find the fix fastScan stability, cleanup and performance issues.Fix NowOctober DealsAmazon USDeal season is back - check today's better picksAmazon US: current deals, useful picks and tech finds.See Picks×
Skip to content
World desk6 min

Time-Series Storage: How to Evaluate Encoding and Compression for IoT Data

Learn how to benchmark time-series encoding and compression for IoT data without mistaking a high compression ratio for the best storage choice.
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Evaluate time-series encoding and compression by testing the complete storage path against data and queries that resemble your IoT workload—not by comparing compression ratios in isolation. Measure bytes per point, fidelity, CPU and memory use, ingestion and query latency, and operational behavior for each supported encoding-and-codec combination. There is no universal winner: results depend on the data, implementation, version, hardware, and workload.

Encoding and compression are related, but they are not the same

Encoding represents values in a way that can exploit their type or sequence pattern. Examples include run-length encoding (RLE) for repeated values, difference coding for predictable sequences, and dictionary encoding for repeated categories. A general-purpose compression codec can then operate on the encoded byte stream. Some storage engines expose these as separate choices; others bundle or manage parts of the process internally.

As an Amazon Associate I earn from qualifying purchases.

The stages interact. An encoding may already remove much of the redundancy that a codec could otherwise exploit, or it may produce a stream that takes more CPU to compress. Do not multiply ratios from separate algorithm tests to estimate the result. Benchmark the actual combinations available in the storage engine you plan to use.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Apache IoTDB’s documentation, accessed October 7, 2026, describes encoding choices by data type and a separate compression stage. It lists codecs including Snappy, LZ4, Gzip, Zstandard, and LZMA2, and calls LZ4 the default and recommended method for its implementation. That is an IoTDB-specific recommendation, not evidence that LZ4 is best for every IoT workload.

#1 Best Overall
M502 Temperature Data Logger USB Temp/Humidity Recorder with PDF and Excel (Reusable) Refrigerator Recording Thermometer 14400 Points High Accuracy (no delay - 1 Pack)
  • [Accurate Temperature and Humidity Recording]:Our M502 temperature and humidity data logger has a wide measurement range of -22℉~158℉ (-30°C~+70°C) and 0%RH~100%RH with an accuracy of ±0.5℉/0.3°C and ±5%RH. Up to 14,400 temperature and humidity points can be recorded and comes with a calibration certificate to ensure accurate and reliable data recording
  • [Easy to use]: Our M502 is plug and play with a USB port, no software required, connect to windows to easily generate PDF and Excel reports. LCD visual display allows you to easily switch between key information, including current temperature and humidity values, average, max or min values, current date and logging points. And you can mark current important events with temperature + time in up to 5 groups. In addition, in the "STOP" mode, you can reset and reuse it after resetting.
  • [Customizable Cold Chain Management]: You can start the logger by downloading the free software in the manual and presetting the start delay. You can set the maximum or minimum alarm temperature as well as humidity. You can set display Fahrenheit/Celsius degree. You can set and match your local time. Equipped with a low temperature resistant CR2450 battery that can last up to 90 days of recording. The logger has IP 67 waterproof protection.
  • [Wide Application]: M502 is a multi-purpose data logger, ideal for transportation and storage of pharmaceuticals, frozen food, fresh food, vegetables, fish, etc. It can be used in every stage of cold chain (food) logistics, including refrigerated containers/trucks, reefer bags, home refrigerators/freezers, etc.
  • [worry free warranty ]: Factory programmed parameters: log recording interval -10 minutes; One year warranty and lifelong customer service. We also provide 24/7 US technical support through email and phone.

Match candidates to the shape and type of your data

Repeated states, smooth signals, noisy floating-point readings, counters, and irregular timestamps can produce very different results. Begin with the patterns your devices actually generate, then test methods supported by your chosen engine.

Data or pattern Possible fit What to verify
Consecutive repeated values RLE How often runs occur in the real stream and whether the selected implementation preserves the required values exactly.
Monotonic integer sequences TS_2DIFF or another difference-based encoding Whether the sequence remains predictable with gaps, resets, late arrivals, or out-of-order writes.
Close successive time-series values Gorilla-style encoding Encoded size, decode speed, and exact reconstruction for the numeric values and timestamp patterns in use.
Repeated categorical values Dictionary encoding, especially at low cardinality How dictionary size and lookup costs change as the number of distinct values grows.
Noisy floating-point readings Test the engine’s supported floating-point encodings and codecs together Losslessness or explicitly allowed error; do not infer fidelity from a small stored size.

These are candidate relationships, not guarantees. For example, Apache IoTDB documents RLE as useful for consecutive repeated values, TS_2DIFF for monotonic integer sequences, Gorilla for close successive values, and dictionary encoding for low-cardinality data. The fit still depends on the sequence and the database’s implementation.

Rank #2
SparkFun DataLogger IoT - 9DoF IMU for Built-in Logging of a Triple-axis Accelerometer, gyro, and Magnetometer. MicroSD Socket, USB Type C, Board Dimensions: 1.66in. x 2.00in.
  • The SparkFun DataLogger IoT - 9DoF comes preprogrammed to automatically log IMU, GPS, and various pressure, humidity, and distance sensors.
  • Included on every DataLogger IoT is an IMU for built-in logging of a triple-axis accelerometer, gyro, and magnetometer. Whereas the original 9DOF Razor used the old MPU-9250, the DataLogger IoT uses the ISM330DHCX from STMicroelectronics and MMC5983MA from MEMSIC.
  • Datalogger Features: MAX17048 LiPo Fuel Gauge, Ports, 1x USB type C, 1x JST style connector for LiPo battery, 2x Qwiic enabled I2C, 1x microSD socket, Support for 4-bit SDIO and microSD cards formatted to FAT32.
  • The DataLogger IoT is highly configurable over an easy-to-use serial interface. Simply plug in a USB-C cable and open a serial terminal at 115200 baud. The logging output is automatically streamed to both the terminal and the microSD card. Pressing any key in the terminal window will open the configuration menu.
  • It was specifically designed for users who just need to capture a lot of data to a CSV or JSON file and get back to their larger project. Save the data to a microSD card or send it wirelessly to your preferred Internet of Things (IoT) service!

Use product recommendations as test candidates, not universal rules

Implementation examples help identify what to benchmark, but their features and performance cannot establish a cross-product ranking.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Storage implementation Documented behavior How to use the information
Apache IoTDB Its guide recommends RLE for BOOLEAN, TS_2DIFF for integer and timestamp types, Gorilla for FLOAT and DOUBLE, and PLAIN for TEXT and STRING. It documents LZ4 as its default and recommended compression method. Use these as IoTDB-specific starting points. Confirm the exact release, settings, supported types, and fidelity behavior you will deploy.
Prometheus local TSDB Its storage documentation describes two-hour blocks, chunk segments, metadata and index files, and a WAL for current samples. The --storage.tsdb.wal-compression option compresses the WAL. Prometheus says WAL size may be halved depending on the data, with little extra CPU, and notes version-compatibility implications. Treat the size and CPU statements as product documentation estimates, not independent benchmark results or guarantees. Include WAL behavior and compatibility in an evaluation if this option matters to your deployment.
InfluxDB 3 Enterprise Its storage-engine documentation describes .pt columnar files sorted by series key and timestamp, with type-specific compression including delta-delta RLE for timestamps, Gorilla for floats, and dictionary encoding for low-cardinality strings. Test the deployed edition and version with your own series cardinality, data types, and query mix; a documented encoding choice does not predict an overall ranking.

Build a benchmark around the workload you need to serve

A useful comparison holds the environment steady while varying the storage options. Preserve the data and scripts so another run can reproduce the result.

Rank #3
Elitech IOT Temperature Data Logger 4G Single-use, Shadow Data, 3-Times Accidental Touch, Auto Flight Mode, Light/Shock/Location, PDF/CSV Report, 32000 Points 60Days Glog5T
  • Shadow Data: Meeting your data security strategies, real time temperature data recorder Glog5T allows you to forget to turn it on or mistakenly stop, which have still been recording during sudden situations. With Cloud Platform, effectively prevents risks throughout the entire cold chain process.
  • 3-Times Accidental Touch: Compared with other disposable loggers, Glog5T Greatly reduces your risk and cost of use.
  • Auto Flight mode/ Electronic Fence: Glog5T strives for aviation safety, complied with Do160. Custom area auto enable (Manual activation, Timed activation, Electronic fence).
  • Multi-Source Sensing: Standard sensors: temp.(Internal)/light/Shock/Location(LBS). Optional sensors: Humidity/PH value/CO2/-320℉external ultra-low temp. Sensor.
  • Glog5T widely used in Food Cold Chain, Harvest Management, Cold Chain Logistics, Insulation Box Matching and Life Science Market.
  1. Describe the workload. Record data types, number and cardinality of time series, sampling regularity, arrival rate, batch size, device count, missing or late data, retention period, and whether compression happens on a constrained device or after ingestion.
  2. Select representative data. Include smooth signals, noisy sensor values, counters or other monotonic values, repeated states, categorical fields with low and high cardinality, and irregular or delayed samples where relevant. Document scaling or preprocessing; do not silently make data more compressible than the source stream.
  3. Fix the test environment. Keep hardware, software version, configuration, data ordering, and concurrency constant across candidates. Record warm-up and cache conditions, and save benchmark scripts and source files.
  4. Measure the whole path. Capture encoded bytes and total stored bytes per point, compression ratio, encode and decode throughput, CPU and memory, ingestion throughput and tail latency, and representative raw, range, aggregate, and latest-value query latency. Include flush, compaction, or recovery behavior when it affects the intended deployment.
  5. Verify reconstruction. Decode the data and compare it with the input. State whether results are exactly lossless or give the error metric and allowed tolerance for any lossy setting. Check timestamp handling, nulls, special numeric values, and relevant boundary values. IoTDB, for example, documents integer minimum-value restrictions for some Gorilla and Chimp integer encodings.
  6. Repeat and report scope. Run enough repetitions to expose variation. Put the version, configuration, hardware, dataset, and query mix beside each result; report warm-up and cache conditions so readers can interpret it.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Report more than a compression ratio

Use a consistent definition. For example, define compression ratio as original bytes divided by stored bytes, and state exactly which bytes are included. A ratio based only on a value stream is not comparable with one that includes timestamps, indexes, metadata, or other storage overhead. Report total stored bytes per point as well as the encoded payload size when the engine exposes both.

  • Storage: bytes per point and the ratio, with the included components and dataset size specified.
  • Fidelity: exact equality after decoding, or the error metric and permitted tolerance.
  • Compute cost: CPU and memory for encoding and decoding, including the device or server where each runs.
  • Ingestion: sustained throughput and latency, including tail latency and the tested batch and concurrency settings.
  • Queries: latency for representative raw reads, range scans, aggregates, and latest-value lookups—not just a single convenient query.
  • Operations: effects of late or out-of-order data, flushes, compaction, recovery, and compatibility constraints that matter to the deployment.

Compression ratio alone can hide a trade-off: a smaller representation may demand more compute, slow reads, or change write latency. Choose metrics and acceptance limits based on the actual device, retention, and query requirements rather than treating one measure as a proxy for all the others.

Rank #4
Lascars Wireless CO2 and Air Quality Data Logger - EL-IOT-CO2
  • Comprehensive Air Quality Monitoring: Measures carbon dioxide, temperature, and humidity.
  • Customizable Alarms: Set high and low alarms for instant notifications.
  • Dual Power Options: USB power supply with AA battery backup for uninterrupted monitoring.
  • Access Anywhere: Use the EasyLog App for data viewing, analysis, and download on any internet-enabled device
  • Wide Application: Suitable for home, workplace, schools, and horticulture.

Interpret published benchmark figures in context

Published numbers are useful for understanding what a particular paper or product test measured, but they are not substitutes for a workload-matched comparison. The Apache IoTDB paper published in 2020 reports up to 30 million data points per second on a single node, along with raw-query results in the hundreds of milliseconds and aggregation-query results in the tens of milliseconds on billions of data points. Those are paper-era claims; the hardware and evaluation conditions in the paper must be considered before comparing them with another system or a current deployment.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

The 2018 Sprintz paper by Davis Blalock, Samuel Madden, and John Guttag presents a lossless time-series method for IoT settings with tight memory and latency budgets. It reports compression speeds of up to 200 MB/s for 8-bit data at its highest-ratio setting and 600 MB/s at its fastest setting. These rates describe the paper’s prototype, datasets, and tested hardware, not expected performance on arbitrary devices. The work is a research candidate and a useful reminder to account for memory and latency budgets, not a current product recommendation.

There is no neutral, current head-to-head ranking established here for IoTDB, Prometheus, and InfluxDB using the same data, hardware, configuration, and queries. The IoTDB comparison page identifies version 0.11.1 and its own workload setup, so its results should be read as historical and version-specific rather than as a current universal comparison.

Choose the option that meets your constraints

First rule out configurations that fail the fidelity requirement or cannot support required types and sequence patterns. Among the remaining candidates, compare storage savings against CPU, memory, ingest and query latency, operational behavior, and compatibility under the same workload. Keep the complete test conditions with the result: a ratio without its dataset and storage definition, or a throughput number without its hardware and version, is not a sound basis for choosing an IoT storage design.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Leave a Reply

Your email address will not be published. Required fields are marked *

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
Crashes, No Sound, or Screen Glitches?Free driver scan
Windows Errors? Fix Them Before They SpreadFree repair scan

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.