Hardware FixRecommendedDevice not working? Your driver may be the problemCheck updates for common hardware issues.Fix DriversOctober DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsWindows FixRecommendedWindows errors stealing your time? Find the fix fastScan stability, cleanup and performance issues.Fix Now×
Skip to content
World desk5 min

How to Test README Code Examples and Keep Them Working

A practical guide to choosing a documentation test runner, connecting README examples to tested code, and making example failures visible in CI.
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Test README examples by choosing a runner that understands their language and format, then adding its command to the project’s normal test or documentation build. There is no universal command for every fenced code block: Python’s doctest, Sphinx’s documentation test builder, Rust’s rustdoc, and Markdown-aware tools such as Byexample cover different workflows. The key is to decide what each snippet is meant to prove and make the test cover those snippets deliberately.

First decide which README examples should run

A README may contain executable examples, but also sample output, configuration fragments, shell commands, pseudocode, or code that depends on a live service. Treating every fenced block as runnable can create brittle or unsafe checks; treating none as runnable lets examples drift away from the project.

As an Amazon Associate I earn from qualifying purchases.

  1. Inventory the fenced blocks. Record the language and purpose of each one.
  2. Classify each block. Mark it as an executable example, expected output, configuration, or explanatory sample.
  3. List its prerequisites. Note required packages, environment variables, files, credentials, network access, and state.
  4. Choose a check for each executable example. Be explicit about which blocks the runner discovers; a passing test only verifies what its extraction rules actually find.

Keep credentials out of examples and tests, and avoid running documentation checks against production systems. If a snippet depends on an external service or unpredictable data, either provide a safe, controlled test setup or label the limitation clearly instead of implying that it is verified.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Choose a runner that matches the language and format

Start with the project’s existing language and documentation stack. The right choice depends on whether examples use interactive prompts, marked documentation blocks, language-native doc tests, or ordinary Markdown fences.

Route Best fit What it checks Tradeoff
Python doctest Interactive Python examples in docstrings or text files Executes prompts and compares results with expected output It expects doctest prompt syntax; it does not automatically run every ordinary fenced Python block in arbitrary Markdown.
Sphinx sphinx.ext.doctest Projects already building documentation with Sphinx Runs marked setup and test blocks, grouped by document and group Examples need suitable markup, and the check belongs in a Sphinx documentation workflow.
Rust rustdoc Rust documentation examples Runs language-native documentation tests It is specific to Rust and does not extract arbitrary multi-language README fences.
Byexample Projects seeking a snippet runner for supported languages and formats Its project description includes examples in Markdown fenced blocks Verify current language support, syntax, setup, and CI integration for your specific project before adopting it.
Tested source included in docs Longer examples or examples that are easier to maintain as source files Lets documentation display source that can also be tested independently The include mechanism and test harness still require configuration; including a file alone does not prove the whole README build works.

Python interactive examples with doctest

Python’s standard-library doctest finds interactive examples and can run them from a text file. For a text file that does not end in .py, the command-line tool infers text-file mode:

python -m doctest README.txt

The documented command-line form is python -m doctest [-v] [-o OPTION] [-f] file [file ...]. Use -v when you want verbose test output. Because doctest looks for interactive prompts, ordinary fenced Python such as print(1 + 1) is not automatically a doctest just because it appears in a README. Format examples using the prompt-and-output convention or choose a runner that extracts the Markdown blocks you want to test. Python describes doctest as a way to check examples stay up to date, sometimes called “literate testing” or “executable documentation.” Python doctest documentation

Sphinx examples with the doctest builder

If Sphinx already builds your documentation, sphinx.ext.doctest can collect marked setup and test blocks and execute them with its doctest builder. It supports doctest-style and code-output-style blocks, and runs setup blocks before test blocks. This is a good fit when the documentation already uses Sphinx directives; it is not a general Markdown-fence runner without the appropriate markup and build configuration. Sphinx doctest extension documentation

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Rust examples with rustdoc

For Rust documentation examples, rustdoc provides language-native documentation tests. Use it for Rust’s documented examples rather than assuming it will discover and execute unrelated language fences in a README. The Rustdoc Book: Documentation tests

Markdown fences with a snippet runner

Byexample describes support for finding examples in Markdown fenced code blocks and other formats. Before adopting it, check its current documentation for the languages, syntax, setup, and output behavior your README needs; broad format support does not establish compatibility with every language or project. Byexample project documentation

Choose how the displayed example connects to tested code

For short examples, keep code and expected results together in a testable prompt-based block when that makes the behavior clear. For longer examples, consider maintaining a source file that the documentation includes, so readers see the same code the project tests rather than a second hand-maintained copy.

Ray’s documentation guide distinguishes doctest-style examples, code-output examples, and literalinclude. It recommends doctest style for small examples where intermediate values or object representations matter; code-output style for longer examples or where exact representations do not matter; and literal inclusion for end-to-end examples without outputs. Ray’s guide also says examples dependent on external systems need not be tested in its documentation, and describes skip controls and ellipses for unstable output. Those are choices documented for Ray’s workflow, not a blanket reason to leave other projects’ examples unverified. Ray documentation guide

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Whether you use included source or inline snippets, verify that the documentation build actually consumes the relevant file and that the test command executes the intended example. Displaying tested source is useful, but it does not by itself validate the complete README or documentation build.

Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

Put the check in the normal project workflow

  1. Run the documentation check locally. Use the same command contributors and CI will run, such as python -m doctest README.txt for a text file written with doctest prompts, or the configured Sphinx doctest build for marked Sphinx examples.
  2. Add it to the existing test or documentation job. Keep it alongside the project’s routine checks so a README change can fail visibly in CI.
  3. Check the failure path. Make sure a deliberately incorrect expected result or invalid example causes the command to fail; this confirms the example is being discovered rather than silently skipped.
  4. Review skipped or unstable cases. Explain exclusions and keep skip behavior narrow so an exception does not hide unrelated failures.

Sphinx’s documentation explains why this matters: “it is important to ensure that the documentation stays up-to-date with the code.” Its doctest builder and Ray’s guide describe ways to make selected examples part of documentation checks. Sphinx doctest extension documentation Ray documentation guide

Know what a passing check does—and does not—mean

  • It covers discovered examples, not necessarily every code fence. Prompt-based tools and marked-block builders rely on their configured syntax and extraction rules.
  • It checks the configured behavior. A runner may compare expected output, execute setup and test blocks, or simply run included code; select the form that matches what readers need to learn.
  • It cannot make unstable dependencies deterministic by itself. Control external state where feasible, or state clearly when an example is not exercised.
  • It is only as current as the workflow that invokes it. A local command that nobody runs routinely offers less protection than a check integrated into the project’s documentation or test job.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
PC Slower Than It Used to Be?Free scan - under a minute
Outdated Drivers Are Slowing You DownFree scan - exact matches

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.