If your retrieval-augmented generation (RAG) system misses an answer that is plainly in your documentation, inspect the chunks it retrieves. A Markdown chunker that cuts through a fenced code example can separate setup from logic, or code from the heading and explanation that make it understandable. Use structure-aware chunking to preserve those relationships, then test the result on your own documentation and questions.
Why splitting code fences hurts documentation retrieval
A fenced code block is often one coherent example: its language label, setup, implementation, and sometimes output or follow-up explanation work together. A fixed-size splitter that treats Markdown as a flat stream can break that unit at an arbitrary character or token boundary. The resulting fragments may each be too incomplete to answer a question reliably.
As an Amazon Associate I earn from qualifying purchases.
The damage is not limited to the code. If the split also separates a subsection from its heading or nearby prose, retrieval may return a technically accurate fragment without the context that says what it does or when it applies. The RAG Handbook describes this broader issue for code examples and other structured content, including tables and lists: splitting across meaningful boundaries can discard context (RAG Handbook, “Structure-aware chunking”).
What a useful Markdown chunk should preserve
For a code-related question, a retrieved chunk should give the model enough context to interpret the example without requiring it to guess what was omitted. Depending on the document, that can mean keeping together:
#1 Best Overall
- The opening and closing fence, plus its language label.
- The relevant code and any setup or configuration it depends on.
- The heading ancestry that identifies the example’s topic or scope.
- Nearby explanation that defines inputs, expected behavior, or limitations.
Not every chunk must contain an entire page. The goal is to preserve meaningful units and enough context for retrieval, rather than to maximize chunk size or enforce a single size everywhere.
How to stop a Markdown chunker from splitting code blocks
- Inspect the chunks actually stored for retrieval. Check whether fences, code, language labels, explanatory text, and heading context survive ingestion. Looking only at the source Markdown will not reveal what the retriever can see.
- Recognize Markdown structure before applying size limits. Use a Markdown-aware parser or chunking strategy that can identify headings and fenced blocks. Prefer boundaries before or after a code block instead of splitting inside it. The RAG Handbook recommends heading-aware boundaries, intact fenced blocks where practical, and parent-heading context.
- Carry heading context into child chunks. A subsection may be retrieved without its parent section. Prepending relevant parent headings to the chunk text, or storing them as retrievable metadata, helps situate the code. Choose a representation your retrieval and generation stages actually use.
- Define what happens when a block is too large. Keeping a block intact can make a chunk exceed its target size, while an embedding model still has an input limit. Set an explicit policy: preserve the block with appropriate context if it fits the model budget; otherwise consider splitting at logical or syntactic boundaries, or creating a separate representation for oversized examples. These are engineering options, not a universally established best choice.
- Evaluate with representative questions. Include questions that rely on code details, a parent heading, and the relationship between prose and an example. Compare the retrieved chunks and resulting answers before and after the change.
Choose a strategy by its behavior, not its label
“Chunk size” and “Markdown-aware” do not mean the same thing across implementations. Compare a candidate approach against the needs of your corpus:
Rank #2
| What to check | Why it matters |
|---|---|
| Structure fidelity | Does it preserve fenced blocks and recognize headings, lists, or tables as meaningful boundaries? |
| Size unit | Does it count tokens or characters? A character target does not guarantee that a chunk will fit an embedding model’s token limit. |
| Context carried with code | Are parent headings and relevant explanation included in the retrieved text or usable metadata? |
| Oversized-block behavior | Does the implementation split the block, emit an oversized chunk, or apply a documented secondary strategy? |
| Operational complexity | Parsing structure takes more logic than splitting flat text. Account for integration and maintenance as well as the retrieval benefit you observe. |
Rag.NET’s documentation illustrates why implementation details matter: it distinguishes character-based recursive splitting from token-aware and structure-oriented approaches, and documents defaults of 512 characters per chunk with 50 characters of overlap for its recursive splitter when nothing is configured. Those are Rag.NET project defaults, not universal recommendations or token limits (Rag.NET documentation, “Chunking”).
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →Semantic section parsing is another implementation pattern. Extend documents a Markdown section strategy intended to split at semantic boundaries without breaking Markdown elements, and describes page and block metadata for citations. That is a description of Extend’s service, not a guarantee about every parser (Extend, “Parsing for RAG,” version labeled 2026-02-09).
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.How to tell whether the change worked
Use questions drawn from the documentation your system serves, not only a few convenient examples. For each one, inspect the retrieved context and ask:
- Is the relevant code intact, or are required lines in a different chunk?
- Does the chunk identify the heading or section the example belongs to?
- Is the prose needed to interpret the code present or available to retrieval?
- Does the answer become more accurate without introducing irrelevant material?
Compare the old and new retrieval results and answers on the same questions. The reviewed documentation does not establish a universal quality gain or an optimal chunk size, so any improvement should be measured on your own corpus and evaluation set.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
