Datalab announced OmniParseBench on October 8, 2026, as an open benchmark for OCR and document parsers. It asks a practical question—which parser is best on documents like yours?—using 16,288 page-level pass/fail tests across 2,937 pages, 2,343 documents and 92 languages, rather than one opaque leaderboard number.
What OmniParseBench measures
Each benchmark test is a yes-or-no question about one page. Examples include whether text is present, whether one item comes before another, whether a value belongs under particular table headings, or whether a line sits inside a specified layout block. A parser earns a pass or fail for each question, making its errors inspectable at page level.
As an Amazon Associate I earn from qualifying purchases.
The headline score is the arithmetic mean of three family scores: text, tables and layout. Because the families are averaged equally, the headline is not simply the percentage of all 16,288 tests passed; a strong result in one family cannot completely hide a weak result in another.
The Tool Desk
Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →The five test types
| Test type | Tests | Family | What it checks |
|---|---|---|---|
present |
7,275 | Text | Whether required content appears |
order |
1,189 | Text | Whether items occur in the correct sequence |
repeat |
35 | Text | Whether repeated content is handled correctly |
table_cell |
5,386 | Tables | Whether a cell is associated with the specified headings and structure |
layout_kind |
2,403 | Layout | Whether content is classified inside the required layout block |
The counts are listed in Datalab’s 2026 dataset card and total 16,288 tests.
#1 Best Overall
- OUR MOST ADVANCED SCANSNAP. Large touchscreen, fast 45ppm double-sided scanning, 100-sheet document feeder, Wi-Fi and USB connectivity, automatic optimizations, and support for cloud services. Upgraded replacement for the discontinued iX1600
- CUSTOMIZABLE. SHARABLE. Select personalized profiles from the touchscreen. Send to PC, Mac, mobile devices, and clouds. QUICK MENU lets you quickly scan-drag-drop to your favorite computer apps
- STABLE WIRELESS OR USB CONNECTION. Built-in Wi-Fi 6 for the fastest and most secure scanning. Connect to smart devices or cloud services without a computer. USB-C connection also available
- PHOTO AND DOCUMENT ORGANIZATION MADE EFFORTLESS. Easily manage, edit, and use scanned data from documents, receipts, photos, and business cards. Automatically optimize, name, and sort files
- AVOIDS PAPER JAMS AND DAMAGE. Features a brake roller system to feed paper smoothly, a multi-feed sensor that detects pages stuck together, and skew detection to prevent paper damage and data loss
How to benchmark documents that resemble yours
OmniParseBench attaches tags to every test along five axes. Engineers can therefore report performance on comparable slices instead of relying on the aggregate score.
- Structure: tables, mathematics, forms, multi-column pages and other document arrangements.
- Rendering: handwriting, tiny text, rotation and degraded scans.
- Role: headings, captions, footnotes, lists, code, figures and headers or footers.
- Script and language: useful for multilingual workflows and specific writing systems.
For example, a team processing handwritten forms can isolate handwriting, form and language tags, then compare text, table and layout results on that slice. A publisher handling dense two-column articles can examine ordering and layout tests without allowing easier single-column pages to dominate the decision.
What an OCR score actually measures
The score measures whether a parser’s output satisfies the benchmark’s defined assertions. It is not a direct measure of character error rate, word error rate, reading speed or extraction cost. A parser can recognize many words yet fail an ordering, table-cell or layout assertion that downstream software needs.
Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Rank #2
- Design and Speed: Work with Windows XP/7/8/10/11 AND macOS 10.13 or later. Not compatible with Android and iOS. Designed for A3&A4(11.69*16.53 & 8.27*11.75 inch) document, any objects smaller than A3 size can be scanned with Ultra-fast scanning speed, about 1 second per page. Perfect device to scan FLAT papers
- USB Document Camera & Scanner: Work as both a document camera for remote teaching&learning compatible with ZOOM; Goole Meet and a document scanner to scan papers and convert/OCR files. OCR supports 180+ languages for text recognition. Please note that Thai, Hebrew, and Arabic are currently not supported. If you need the complete OCR language support list, please feel free to contact us for more details
- Patented Flattening Curved Book Page Technology: Shine Ultra applies CZUR’s patented technology to flatten the curved surface after pixel transformation to flattening of the book page (Only suitable for thinner books, ET series is recommended for thicker books)
- High Resolution & AI Tech: CMOS 13MP (4160*3120, A4≈340 AND A3≈245 DPI) camera. Smart Paging and Auto Cropping; Combine Sides; Stamp Mode; and Multiple Color Modes
- Height Adjustable & Portable: 2-level height adjustable neck. 90 degree foldable and lightweight 4 lbs with foot pedal for convenient operation
Read at least four dimensions together:
- Content family: text, tables and layout scores reveal where a parser succeeds or fails.
- Document conditions: handwriting, tiny or degraded scans, rotation, scripts and languages can change results sharply.
- Structural fidelity: ordering, table relationships and block classification determine whether extracted content remains usable.
- Operations: latency, hardware and infrastructure affect production suitability even when accuracy is similar.
Why Datalab created a new benchmark
Datalab says the earlier olmOCR-bench showed that pass/fail unit tests can be useful, but later improvements increasingly reflected output conventions and benchmark overfitting rather than better reading. It also says ParseBench reports five separate metrics that do not share a common unit, making them difficult to combine or compare cleanly.
OmniParseBench keeps the unit-test approach while adding per-argument tags and one common pass/fail basis. That combination is intended to make both broad comparisons and narrowly targeted audits possible.
Can failures be audited by language or handwriting?
Yes. The scorer can be used with the benchmark’s tags to select tests by language, script, handwriting, rendering condition, document structure or semantic role. The result is a slice-specific view rather than a claim that one parser is universally best.
Rank #3
- FAST SPEEDS - Scans color and black and white documents a blazing speed up to 16ppm (1). Color scanning won’t slow you down as the color scan speed is the same as the black and white scan speed.
- ULTRA COMPACT – At less than 1 foot in length and only about 1. 5lbs in weight you can fit this device virtually anywhere (a bag, a purse, even a pocket).
- READY WHENEVER YOU ARE – The DS-640 mobile scanner is powered via an included micro USB 3. 0 cable allowing you to use it even where there is no outlet available. Plug it into you PC or laptop and you are ready to scan.
- WORKS YOUR WAY – Use the Brother free iPrint&Scan desktop app for scanning to multiple “Scan-to” destinations like PC, Network, cloud services, Email and OCR. (2) Supports Windows, Mac and Linux and TWAIN/WIA for PC/ICA for Mac/SANE drivers. (3)
- OPTIMIZE IMAGES AND TEXT – Automatic color detection/adjustment, image rotation (PC only), bleed through prevention/background removal, text enhancement, color drop to enhance scans. Software suite includes document management and OCR software. (4)
Every test also records how its answer was established. The open scorer does not use a model to judge the parser’s output, which avoids introducing a second, opaque model into the evaluation.
Windows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallOutdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWhat is included and how provenance works
The dataset’s results.parquet contains one row per parser and test. Rows identify the page, document, suite, test type, derivation, tags, result (pass, fail or unsupported) and an explanation of failure. The package also includes page files, gold JSON files, a manifest and source-license records.
This structure lets an engineer move from a low score to the exact pages and assertions responsible. Treat unsupported as a separately reported outcome when analyzing results; it is not the same label as a failed assertion.
Rank #4
- FITS SMALL SPACES AND STAYS OUT OF THE WAY. Innovative space-saving design to free up desk space, even when it's being used
- SCAN DOCUMENTS, PHOTOS, CARDS, AND MORE. Handles most document types, including thick items and plastic cards. Exclusive QUICK MENU lets you quickly scan-drag-drop to your favorite computer apps
- GREAT IMAGES EVERY TIME, NO EXPERIENCE REQUIRED. A single touch starts fast, up to 30ppm duplex scanning with automatic de-skew, color optimization, and blank page removal for outstanding results without driver setup
- SCAN WHERE YOU WANT, WHEN YOU WANT. Connect with USB or Wi-Fi. Send to Mac, PC, mobile devices, and cloud services. Scan to Chromebook using the mobile app. Can be used without a computer
- PHOTO AND DOCUMENT ORGANIZATION MADE EFFORTLESS. ScanSnap Home all-in-one software brings together all your favorite functions. Easily manage, edit, and use scanned data from documents, receipts, business cards, photos, and more
Is the benchmark reproducible offline?
The benchmark is designed for offline scoring. MarkTechPost reports that the harness runs on a Python 3.11-or-newer machine and does not require an API key for scoring; a vendor’s own parser run may require that vendor’s credentials. Verify those operational details against the package documentation before using them in production.
Basic reproduction checklist
- Use Python 3.11 or newer, matching the PyPI metadata for the official
omni-parse-benchpackage. - Install the package in an isolated environment with
python -m pip install omni-parse-bench. - Run the open scorer against the same benchmark pages and record the parser version, settings and environment.
- Preserve the generated results so page-level failures, derivations and explanations can be reviewed later.
- When comparing systems, keep hardware, batch size, page concurrency and preprocessing choices fixed, and report latency per page alongside accuracy.
Version 0.2.0 was released on PyPI on October 14, 2026. The code is Apache 2.0. Tests are CC BY 4.0 with documented exceptions, while page data follows the license of each source. Anyone redistributing page data must inspect the included license files first.
Recommended Free Tools
Published results need their setup attached
Datalab’s Indra OCR report illustrates why benchmark numbers cannot be separated from test conditions. On x86 hardware with four vCPUs, one page processed at a time, the same pages and scorer, and default settings, Indra OCR recorded a 62.1 headline score: 41.3 for text, 56.4 for tables and 88.6 for layout, at 228 milliseconds per page.
Best Value
- FAST DOCUMENT SCANNING — Document scanner with feeder allows you to speed through stacks with a 50-sheet Auto Document Feeder (ADF); Efficient office scanner to help you scan more productively
- INTUITIVE, HIGH-SPEED SOFTWARE — Quickly scan with this desktop document scanner; Epson ScanSmart Software lets you easily preview scans, email files, upload to the cloud, and more; Plus, automatic file naming saves even more time
- SEAMLESS INTEGRATION — Easily incorporate your data into most document management software with the included TWAIN driver; Office document scanner integrates seamlessly with business workflows
- EASY SHARING — Duplex scanner allows you to scan straight to email or popular cloud storage2 services like Dropbox, Evernote, Google Drive, and OneDrive for simple storage and sharing
- SIMPLE FILE MANAGEMENT — Scanner allows the creation of searchable PDFs with Optical Character Recognition (OCR) and convert scans to editable Word or Excel files effortlessly; Designed for home and office document scanning
Under that report’s conditions, PaddleOCR PP-StructureV3 recorded a 58.4 headline score and 40,958 milliseconds per page. These are publisher-reported figures, not an independent reproduction, and they should not be generalized to different hardware, versions, preprocessing or concurrency settings.
| Reported system | Headline | Text | Tables | Layout | Latency | Conditions |
|---|---|---|---|---|---|---|
| Indra OCR | 62.1 | 41.3 | 56.4 | 88.6 | 228 ms/page | x86, four vCPUs, one page at a time, default settings |
| PaddleOCR PP-StructureV3 | 58.4 | not stated in the cited report | not stated in the cited report | not stated in the cited report | 40,958 ms/page | Report conditions cited by Datalab |
What teams should report
A credible internal or published comparison should name the benchmark version, selected suites and tags, parser version, preprocessing, hardware, concurrency, scorer settings and latency method. Include family scores and the slices most relevant to the intended workload, not just the headline mean. This makes it possible for another team to reproduce the result and see whether a claimed advantage applies to its own documents.
As Datalab authors Paul Scemama, Hunter Heidenreich and Vikas Paruchuri put it: “Which parser is best on documents like yours? 16,288 tagged pass/fail tests let you check.”
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
