On one 94-page commercial property insurance policy, LlamaIndex’s SentenceSplitter had the highest reported retrieval recall@5, while Chonkie’s RecursiveChunker split the document fastest. LangChain’s token-aware recursive splitter was close on recall. These are results from one author’s small, task-specific experiment—not a universal ranking of the libraries.
What the 94-page PDF test found
The comparison used a 94-page commercial property insurance policy converted to 214,883 characters with pdftotext -layout. The author wrote 38 questions by hand, paired each with an answer string said to occur on exactly one page, and reported spending 47 minutes labeling them. Retrieval used normalized BAAI/bge-small-en-v1.5 embeddings, cosine-similarity ranking, and a top-five cutoff.
As an Amazon Associate I earn from qualifying purchases.
Recall@5 here means the share of the 38 questions for which at least one of the five highest-ranked chunks contained the paired answer string. It measures retrieval of a known answer string, not whether a language model generated a correct or well-supported answer.
| Library and splitter configuration | Reported recall@5 | Time to chunk the PDF | Chunks produced |
|---|---|---|---|
LangChain: RecursiveCharacterTextSplitter.from_tiktoken_encoder, size 512, overlap 64 |
0.79 | 2.8 seconds | 137 |
LlamaIndex: SentenceSplitter, size 512 tokens, overlap 64 |
0.86 | 4.1 seconds | 129 |
Chonkie: RecursiveChunker, size 512 |
0.82 | 0.38 seconds | 133 |
These figures are reported by the comparison’s author, not independently verified measurements. The original report is dated August 20, 2026: DEV Community comparison.
#1 Best Overall
Why the chunk-size units matter
The first LangChain trial used a size of 512 characters, while the other configurations used 512 tokens. That LangChain run scored 0.44 recall@5. The author then changed it to RecursiveCharacterTextSplitter.from_tiktoken_encoder(chunk_size=512, chunk_overlap=64), which scored 0.79. The corrected run is the LangChain result in the table; the character-based trial is not an equivalent comparison.
Even token-based settings do not guarantee identical token counts across tools. The author notes that tiktoken’s default cl100k_base tokenizer differs from the BERT-family embedding model’s tokenizer. Token-count drift may matter more for code, JSON, and non-Latin text, so record both the size unit and tokenizer when comparing configurations.
How chunk size and overlap changed this test
On the same labeled policy test, the author reported the following recall@5 results for LangChain’s chunk-size and overlap sweeps:
| Setting tested | Reported recall@5 |
|---|---|
| Chunk size: 256 tokens | 0.74 |
| Chunk size: 512 tokens | 0.86 |
| Chunk size: 768 tokens | 0.81 |
| Chunk size: 1,024 tokens | 0.68 |
| Overlap: 0 tokens | 0.79 |
| Overlap: 64 tokens | 0.86 |
| Overlap: 128 tokens | 0.87 |
The 128-token overlap setting also produced 22% more chunks to store and search than the comparison’s baseline. These are results from the same small policy test, not universal settings to copy. They show why it is worth measuring both retrieval performance and the extra index size or search work created by overlap.
Which library should you choose?
The benchmark supports a narrow decision: LlamaIndex led this test on recall@5, and Chonkie led it on single-document splitting time. LangChain’s corrected result was between them on recall and speed. Pick based on what your application needs, then evaluate on your own documents and representative questions.
- If retrieval performance is the priority, use the LlamaIndex result as a candidate to test—not proof it will win on another corpus.
- If ingestion speed is the priority, Chonkie was fastest in this one-document run. Measure throughput on your corpus size and deployment hardware before relying on that advantage.
- If orchestration is central, assess the wider application architecture as well as the splitter. LangChain’s own comparison frames LangChain and LangGraph around agent orchestration, while it frames LlamaIndex around document ingestion, indexing, and retrieval. It describes combining them when an application needs both; this is vendor framing, not an independent evaluation: LangChain’s comparison.
- If lightweight ingestion is the goal, Chonkie’s repository describes it as a library for RAG ingestion. Its project-published chunker benchmarks use other datasets and hardware, and the project cautions that results vary with setup. They are separate context, not validation of the 94-page PDF figures: Chonkie repository and Chonkie benchmark documentation.
What the separate 1,240-file run can—and cannot—tell you
The same author reports processing 1,240 files containing 2.1 GB of extracted text in 47 seconds with Chonkie, 3 minutes 4 seconds with LangChain, and 6 minutes 12 seconds with LlamaIndex. This is a separate author-reported corpus run, and the account does not document enough detail to equate it with the policy test or predict performance on another machine or dataset.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How to make the comparison useful for your project
- Build a representative question set. Include questions users actually ask and define what counts as a retrieved hit—for example, a chunk containing the answer or evidence needed to answer it.
- Keep the retrieval setup fixed. Use the same source text, embedding model, ranking method, and value of
kfor every splitter. Otherwise, the comparison changes more than chunking. - Normalize chunk-size semantics. Record whether size means characters or tokens and which tokenizer is used. Don’t compare a character limit against a token limit as if they were equivalent.
- Measure more than one outcome. Track a retrieval metric such as recall@k and ingestion time; account for the number of chunks because it affects storage and search. If generated answers matter, evaluate answer quality and faithfulness separately.
- Repeat on the intended environment. Library APIs, defaults, package size, and tokenizer behavior can change. Check the versions you plan to deploy and rerun the evaluation on your target corpus and hardware.
Limits of the published result
The benchmark is one author’s report on one policy document and 38 manually labeled questions. The article publishes a scoring script and splitter configurations, but the PDF and the hand-labeled question-and-answer-string dataset are not available in the reviewed material, so the exact numerical results cannot be independently reproduced from those materials. Treat the scores and timings as directional evidence for designing a local test, not as a general framework league table.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
