Hardware FixRecommendedDevice not working? Your driver may be the problemCheck updates for common hardware issues.Fix DriversOctober DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsPC HealthRecommendedCrashes, freezes, slowdowns? Check your PC nowSpot repairable issues before they interrupt work.Check PC×
Skip to content
World desk5 min

Lexer Design in Koi Editor: How Highlighting and Folding Work

Koi Editor says each supported language has a purpose-built lexer that determines token styling and fold levels. Here is how that design compares with the grammar and query models Koi describes, and how to interpret its latency figures.
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Koi Editor says it uses a small, purpose-built lexer for each supported language. Each lexer identifies tokens, tracks relevant state and decides fold levels, directly connecting language-specific rules to syntax highlighting and code folding. That is a different design from the grammar-and-query approach Koi describes for Sublime Text and Zed; it is not, by itself, proof that one editor is faster.

How Koi says its lexer works

In its July 30, 2026 article, Lexer design in Koi, Michael Sjoberg describes a direct pipeline: a language-specific lexer produces tokens and fold levels. In his words, “Each language has a small purpose-built lexer that directly decides how text should be styled and folded.”

As an Amazon Associate I earn from qualifying purchases.

Koi’s account says there is no generic grammar or regex-definition layer between the text and the highlighting decisions. Its homepage says the lexers are written in C++ and optimized for speed. These are Koi’s descriptions of its own design; the published account does not provide an independent inspection of the implementation.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Three jobs for a lexer

Koi describes each lexer as handling three related questions:

  • What kind of token is this? A token may be a keyword, string, number, operator, comment or another language element.
  • What state should carry forward? The lexer may need to remember context as it proceeds through the text, so later characters can be interpreted appropriately.
  • Where does a fold begin or end? The lexer also determines fold levels, linking language-aware parsing decisions to code folding.

Koi’s design rationale is that ordinary code can express unusual, language-specific conditions directly rather than forcing them into a generic rule format. That is an argument about flexibility, not evidence that every lexer is simpler to write or maintain.

How Koi contrasts its approach with Sublime Text and Zed

Koi’s article characterizes Sublime Text as using declarative syntax definitions built around regular expressions and contexts. It characterizes Zed as using Tree-sitter grammars and parsers to build a syntax tree, with highlight queries applied to that structure. This is Koi’s comparison, not a complete independent account of either editor’s internals.

Approach as described by Koi How rules are expressed Relationship to highlighting and folding
Koi Purpose-built, language-specific lexer code The lexer directly determines token styling and fold levels.
Sublime Text Declarative syntax definitions using regexes and contexts Koi presents syntax definitions as the rule layer for highlighting; its comparison does not establish a full account of folding internals.
Zed Tree-sitter grammar and parser, plus highlight queries The parser produces a syntax tree and queries identify highlighting; Koi’s comparison does not establish a full account of folding internals.

The practical distinction is where language-specific behavior is encoded: directly in lexer code or in grammar, context and query layers. Code can give an implementer fine-grained control over special cases. The trade-off, as a reasonable inference from Koi’s per-language design, is responsibility for implementing and maintaining those rules for each language.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Folding is a policy choice, not just a brace-counting rule

Koi’s examples show why two editors can fold the same text differently. The results depend on both the language mode and the folding policy the editor applies; a fold level need not simply correspond to visually indented lines.

  • In one set of short examples, Koi does not fold the sample when it is treated as plain text, but folds all three Python examples. Koi’s author says he enabled folding on sets and did not require the surrounding syntax to be valid.
  • In a separate indentation-only example, Sublime Text and Zed fold the sample in C, while Koi does not.

These are results for the specific samples in Koi’s article, not a general ranking of the editors’ folding behavior. They illustrate that folding can be deliberately tied to language constructs and context rather than indentation alone.

Language coverage and what it tells you

Koi’s product page reports support for 30+ languages, including C, C++, Python, Rust, JavaScript, TypeScript and Markdown. Its documentation lists theme categories such as keyword, string, number, operator, comment, type and builtin. It also documents a show_active_lexer status-bar setting, which can help users identify the active lexer.

Koi’s changelog records new lexers for Mojo and MATLAB and says XML and plist use the HTML lexer. It also records the removal of smart line comments for HTML containing nested JavaScript, CSS and PHP because those nested forms were not supported in the lexer at that point. Those are dated build notes; they establish what the changelog says for the relevant builds, not necessarily the product’s present-day limits.

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

What Koi’s latency figures do—and do not—show

Koi defines typing latency as the time from a keypress until the updated frame is painted. On its latency page, Koi reports a 2026 test using a one-million-line Odin source file, default editor settings, a Mac mini M2 with 16 GB of RAM, macOS 15.7.5 and a 60 Hz display. Koi says it ran 100 typing iterations and measured 198 text changes.

Editor in Koi’s 2026 report Reported P95 typing latency Highlighting during test
Koi Editor 17.86 ms Enabled
Sublime Text 58.46 ms Disabled; Koi says repeated crashes with the Odin syntax package led it to test without highlighting.
Zed 89.97 ms Disabled

These are vendor-published results, not independently reproduced measurements. Because syntax highlighting was on for Koi and off for Sublime Text and Zed, the comparison is not equivalent in features. It cannot show that Koi’s lexer architecture alone caused the latency difference, or predict what a particular user will experience with another file, machine or configuration.

Koi’s author summarizes one aim of the design this way: “Sometimes doing less work by default is the performance optimization.” That captures the design rationale; the reported benchmark does not isolate the lexer as the cause of its result.

How to compare lexer designs for your work

Architecture labels are less useful than the behavior and authoring trade-offs that matter in your environment. When evaluating Koi or another editor, consider:

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
  • Rule authoring: Are language rules written as dedicated code, declarative grammar definitions, queries or some combination?
  • Special cases: Can the system express the contextual behavior your languages and files require?
  • Feature integration: Does the same language-aware structure inform folding and other editor features, or are those handled separately?
  • Coverage and upkeep: Which languages are supported, and how do new language features or nested-language constructs fare?
  • Observed responsiveness: Test your own representative files with the same features enabled and comparable settings. A cross-editor latency number is meaningful only with its test conditions and feature differences in view.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Leave a Reply

Your email address will not be published. Required fields are marked *

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
Crashes, No Sound, or Screen Glitches?Free driver scan
PC Slower Than It Used to Be?Free scan - under a minute

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.