October DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsPC HealthRecommendedCrashes, freezes, slowdowns? Check your PC nowSpot repairable issues before they interrupt work.Check PCOctober DealsAmazon USDeal season is back - check today's better picksAmazon US: current deals, useful picks and tech finds.See Picks×
Skip to content
World desk3 min

The Model Obeys Your Schema, Not Your Description—With Important Limits

A schema can constrain a model’s response format, but a valid response may still misunderstand the task or contain false information.
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

A schema can constrain the shape of a model’s response; it cannot, by itself, ensure the model understood your request or returned correct facts. In supported strict structured-output modes, the schema acts as an output contract, while the task description still supplies the job and context. A response can satisfy every required field and still be wrong.

What structured outputs actually enforce

With ordinary prompting, you ask a model to follow a format. In a supported strict structured-output mode, the API uses a schema to constrain the generated response. OpenAI describes converting JSON Schema into a grammar and allowing only tokens compatible with that grammar as generation proceeds; Anthropic also describes schema-constrained generation for its JSON structured-output feature. See OpenAI’s Structured Outputs announcement and Anthropic’s documentation.

As an Amazon Associate I earn from qualifying purchases.

That guarantee is limited to supported constraints, schema features, and request configurations. Providers do not necessarily support every JSON Schema feature, and API modes can behave differently. Google, OpenAI, and Anthropic each document limitations or supported subsets in their respective guides: Google, OpenAI, and Anthropic.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Why a valid response can still be wrong

A schema can require fields, data types, or enumerated values when the platform supports those constraints. It does not establish that the chosen value is true, relevant, or responsive. A model might return all required fields while selecting the wrong category, extracting an incorrect date, misreading the input, or inventing a value to fill a field.

OpenAI warns that structured outputs can still contain mistakes and that unrelated input can lead to hallucinations as the model attempts to satisfy the schema. Its guide recommends telling the model how to respond when the input cannot produce a valid answer. For extraction tasks, that may mean defining an explicit “not found” or “cannot determine” value rather than leaving the model to guess.

Schema descriptions are not necessarily irrelevant to the model’s interpretation: they can communicate what a field means. A 2026 preprint, “Your Prompt Is Not the Only Prompt,” reports that descriptions did not consistently outperform prompt-based placement on its tested classification task, and that accuracy fell when schema and prompt instructions conflicted. Those results are specific to the tested task and models; they do not establish that schemas always override prompt instructions.

What benchmark figures do—and do not—show

In its August 6, 2024 announcement, OpenAI reported that gpt-4o-2024-08-06 with Structured Outputs scored 100% on the company’s complex JSON-schema-following evaluation, compared with less than 40% for gpt-4-0613 in the same comparison. OpenAI separately said its trained gpt-4o-2024-08-06 model scored 93% on the benchmark before constrained decoding was added. These are vendor-reported results for specific models and an evaluation of schema following—not general rates of factual correctness or independent comparisons across providers. The figures and their context are in OpenAI’s announcement.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

The 2025 JSONSchemaBench paper describes an evaluation comprising 10,000 real-world JSON schemas. It examines constraint-compliant output, coverage of constraint types, and output quality rather than declaring a universal provider winner. Read the JSONSchemaBench paper.

How to use schemas without mistaking format for correctness

  1. Choose the API mechanism for the job. Use function calling when the model needs to connect to tools, functions, or data; use structured response formatting when the answer itself must follow a schema. OpenAI explains this distinction in its structured outputs guide.
  2. Make fields clear. Use intuitive key names and clear titles or descriptions for important fields. A schema is both a structural contract and, through its field meanings, part of the information the model receives.
  3. Check the provider’s supported schema subset. Do not rely on a constraint until you have confirmed that the API mode supports it. The OpenAI, Anthropic, and Google guides document their respective behavior and limitations.
  4. Specify what to do with missing or incompatible input. Include an explicit representation for unknown or unavailable information when appropriate, and tell the model how to handle cases where the input cannot support a valid answer.
  5. Test the content as well as the structure. Validate that outputs fit the supported schema, then use task-specific evaluations or human review to check whether they answer correctly. OpenAI recommends evals to find a structure that works for a use case; JSONSchemaBench also treats output quality as distinct from constraint compliance.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

How to compare structured-output features

A meaningful comparison should separate four questions:

  • Which API mechanism is being used: response formatting, tool calling, or another mode?
  • Which JSON Schema features and constraints does that mode support?
  • How does the platform handle refusals, interrupted responses, and input that cannot support the requested output?
  • How well does the model perform the underlying task, measured separately from schema compliance?

The cited provider documentation describes meaningful differences in features and supported schemas, but it does not establish a controlled, current, like-for-like performance ranking across OpenAI, Anthropic, and Google.

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More from the Wire

  1. World desk4 min
    How to Spot an AI Voice Scam Before Sending MoneyDon’t rely on how a caller sounds. Pause, call back through a known number, and verify the emergency with another trusted person before sending money.
  2. Mountain View desk4 min
    Google’s SynthID Detector: How to Check AI-Generated Images, Video and AudioGoogle’s SynthID Detector looks for an embedded watermark in supported images, video and audio. Here is what its results do—and do not—show.
  3. Redmond desk20 min
    How to create a link to File or Folder in Windows 11Windows 11 gives you several ways to point to a file or folder without moving or duplicating it. You can create a desktop shortcut,…
Recommended PC Tool
Recommended PC Tool
Outdated Drivers Are Slowing You DownFree scan - exact matches
Windows Errors? Fix Them Before They SpreadFree repair scan

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.