There is no format that always uses the fewest tokens. JSON, CSV, YAML, and other representations can tokenize differently depending on the exact serialized text and the target model’s tokenizer. To find the best option for your use case, encode the same data in each format, count it with the same model tokenizer, and include request structure when it will be part of the real input.
Why token counts differ by format
A tokenizer divides text into tokens according to rules associated with a model or encoding. The OpenAI Help Center notes that “The same text can produce different token counts depending on the model, its encoding, and the language” (OpenAI Help Center). Token counts can also be affected by spelling, capitalization, spaces, and the text surrounding a value.
As an Amazon Associate I earn from qualifying purchases.
That means a format label alone cannot predict the count. Compact JSON and pretty-printed JSON are different inputs. So are CSV with a header row and CSV without one. YAML indentation, field names, quoting, delimiters, line breaks, and repeated keys all contribute to the serialized text that the tokenizer sees.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchWindows Errors? Fix Them Before They Spread
Repair common Windows errors and clear accumulated junk for a smoother, more stable PC - no reinstall needed.Free scan · no reinstallComparisons are meaningful only when each candidate represents equivalent information. If one version omits field names or hierarchy that another includes, it may have fewer tokens because it carries less information—not because its format is inherently more efficient.
#1 Best Overall
How to make a fair comparison
- Build a representative test corpus. Use the kinds of data that appear in your real prompts: typical and edge-case values, repeated records, nested objects if relevant, and Unicode or escaping cases where they occur. Keep the actual field names.
- Serialize the same data in every candidate format. Decide whether you are comparing compact, pretty-printed, or production-style output. Include headers, quoting, whitespace, and other conventions you expect to use.
- Choose one target model and tokenizer. Count every candidate with the same model-specific tokenizer or encoding. Do not compare counts produced by different tokenizers.
- Record each count and the test inputs. Keep the serialized samples and the counting code. If you report an aggregate, state how it was calculated—for example, total tokens across a fixed set of examples.
- Count the complete request when that matches your use case. If the data is sent with messages, tools, schemas, files, images, or other request elements, count those along with the isolated payload where useful.
- Translate usage into money separately. Apply the current rates for the exact model and usage categories to measured input, cached input, and output tokens as applicable.
- Report the result with its scope. State the model and tokenizer, corpus, serialization choices, and whether you counted plain text or the complete request. Then consider the engineering tradeoffs as well as the token totals.
Count plain text or the complete request?
A string tokenizer is useful for comparing serialized data, but its count may not include message boundaries, roles, tools, schemas, images, files, or other request-level structure. If your real payload is part of a structured API request, an isolated text count answers only how that text tokenizes—not the full request’s input-token usage.
For OpenAI plain-text counts, the Help Center points to tiktoken and selecting the encoding for the target model. For complete Responses API inputs, OpenAI documents an input-token counting endpoint that accepts messages, images, files, tools, and conversations and includes request-formatting tokens. Use the counting method that matches the quantity you need to compare.
For other model families, use that model’s supported tokenizer. The Hugging Face Transformers tokenizer documentation describes loading model tokenizers and configuring special tokens; the tokenizer configuration and preparation behavior can affect the resulting input IDs and count.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Token count is not the same as API cost
Token counts are one input to a cost estimate, not a bill by themselves. Models can tokenize the same text differently, and input, cached input, and output may have different rates. A format that trims a few input tokens does not automatically make a completed task cheaper if output or reasoning usage changes.
Rank #3
For an economic comparison, use actual measured usage for the relevant categories and the current rates for the specific model. OpenAI’s token-counting guide explains how to count tokens, while current model pricing should be checked separately before estimating a cost (OpenAI API token-counting guide).
What published estimates can—and cannot—tell you
The OpenAI Help Center gives rough English-language estimates such as approximately four characters per token and approximately three-quarters of a word per token, with 100 tokens representing about 75 words. It presents these as estimates, not exact counts or format-comparison results. They cannot tell you whether JSON, CSV, or YAML will be cheapest for your data.
Rank #4
- Used Book in Good Condition
No published statistic comparing equivalent JSON, CSV, and YAML payloads is established here. A NeurIPS Spider2-V paper reports page-token counts for HTML, plain text, simplified HTML, and Markdown using TikToken for GPT-3.5 Turbo, but those measurements concern documentation pages; they do not establish which structured-data format uses fewer tokens (NeurIPS, Spider2-V).
Choose for reliability as well as tokens
After measuring token counts, choose a representation that also works for the people and systems that maintain it. Consider whether the format makes data easy to read and edit, preserves hierarchy and types, handles escaping and record boundaries unambiguously, and can be parsed reliably. A slightly shorter serialization may be a poor trade if it is harder to generate correctly or more likely to break a downstream parser.
Best Value
- Made in USA
- No matter how knowledgeable you are, you will find new and interesting information in this book
- Exclusive pressure and velocity factors enable you to accurately calculate pressure and velocity for reduced loads
- A never before published in depth analyses of current load data
- No matter how knowledgeable you are, you will find new and interesting information in this book
There is no universal answer to “How many tokens does JSON use compared with CSV?” or “Is YAML cheaper than JSON?” The answer for a particular task is the count produced by the target model’s tokenizer for equivalent, realistically serialized data—and, when cost is the concern, the resulting measured usage priced at current model-specific rates.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
