Do these 3 things before closing this tab:
1Fix the driver behind crashes, sound loss and screen glitches2Repair Windows errors before they cause bigger problems3Scan for outdated or missing drivers - takes under a minuteA claim that one tool schema was changed four ways across three agent frameworks in 36 runs is not enough, by itself, to establish which framework broke or why. Without the experiment’s source, exact schema variants, framework versions, model, task, and run results, its outcomes cannot be independently checked. What can be said is that schema handling is framework-specific—and a useful comparison must separate invalid arguments from failures to select, execute, or recover from a tool call.
What is known about the 36-run comparison?
The title describes an experiment, but the underlying post, code, run data, and methodology are not available in the sources cited here. The number of runs and the statement that four schema variations were tested should therefore be treated as claims in the title, not as independently verified study results. No framework can responsibly be named as the winner or as having failed in a particular way on that basis.
As an Amazon Associate I earn from qualifying purchases.
To interpret such a comparison, readers would need the exact framework and version names; model and settings; task and prompt; tool definitions before and after each change; allocation of runs across conditions; and a definition of success. They would also need to know whether reported failures meant invalid arguments, an unselected tool, execution errors, failure to finish the task, or inability to recover after a validation error. These are distinct outcomes.
Free tools Windows power users keep installed
One-click scans. No signup required.
Why a tool schema can change agent behavior
A tool schema describes the inputs a model is expected to provide. Changing a field’s name, type, required status, description, or allowed values can change what a valid call looks like. The framework then has to expose that contract to the model and, depending on its design, convert or validate the resulting arguments.
#1 Best Overall
Frameworks expose schemas differently
Microsoft Agent Framework’s Go documentation says a function signature determines a function tool’s input schema, and shows structured fields with descriptions and enum constraints. Microsoft’s function-tools documentation describes that Go-specific surface; it should not be taken as a statement about every SDK or language.
The JavaScript guide for the OpenAI Agents SDK documents conversion of Standard Schema parameters to JSON Schema and local validation. It also documents non-strict raw JSON Schema tools, where the developer is responsible for validating input. Those are different validation paths within that documented SDK surface, not evidence that every framework handles malformed calls the same way. See the OpenAI Agents SDK tools guide.
Not every tool failure is a schema failure
Microsoft Research’s AgentRx taxonomy calls malformed, missing-argument, or schema-invalid calls “Invalid Invocation.” It distinguishes these from planning, intent, tool-output interpretation, guardrail, and system failures. That distinction matters: a model can produce schema-valid arguments for the wrong tool, or invoke the right tool with valid arguments and still fail because the tool errors or its output is misunderstood. AgentRx reports 115 manually annotated failed trajectories and improvements of 23.6% in failure-localization accuracy and 22.9% in root-cause attribution; those figures describe that report, not the 36-run comparison. Microsoft Research’s AgentRx report.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Names can matter independently of arguments
AG2 documents a separate behavior: only one tool per name is exposed in a turn, and a later tool with the same name replaces an earlier one. This is a documented AG2 behavior, not a general rule for agent frameworks. It illustrates why a comparison should record tool registration and naming as well as parameter schemas. See AG2’s tools documentation.
How to evaluate schema changes fairly
A useful test makes the changed contract explicit and records outcomes at more than one layer. Keep the task, prompt, model, tool implementation, and execution environment constant where possible; change one schema property at a time if the goal is to isolate its effect. If multiple properties change together, report them as a combined condition rather than attributing the result to one field.
- Record the baseline contract. Save the exact tool name, schema, required fields, types, constraints, descriptions, and implementation. Include the framework and version.
- Describe each variant precisely. State exactly what changed—for example, which field became required or which enum values were allowed. Do not use broad labels such as “stricter schema” without the actual difference.
- Hold other conditions steady. Report the model, settings, prompt, task, and any retry or validation behavior. Note unavoidable differences between framework APIs.
- Log each stage separately. Track whether the tool was selected, whether arguments passed validation, whether execution succeeded, whether the task completed, and whether the agent recovered after an error.
- Repeat and report the allocation. Show how many trials ran for each framework and schema condition, how outcomes were counted, and any uncertainty. A total run count alone does not reveal how much evidence supports each comparison.
This reporting makes a failure diagnosable. For production systems, preserve schema versions alongside tool code, validate at the execution boundary, and add regression cases for old and new contracts. A test should check not only that arguments match the schema, but also that valid calls still produce the intended effect and that invalid calls fail safely.
What related schema research does—and does not—establish
Furkan Sakizli’s May 4, 2026 arXiv preprint on TSCG reports approximately 19,000 calls across 12 models and five scenarios, and discusses schema-representation results. Those are findings attributed to that preprint, not a replication or verification of the title’s 36-run comparison. The work is a preprint, so its results should be read as such. Read the TSCG preprint on arXiv.
Quick wins for a faster PC:
Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →How to handle tool schema versioning in production
Treat a tool schema as part of the tool’s interface contract. If a change can invalidate previously generated arguments or alter tool behavior, version it deliberately and test the transition rather than assuming the model or framework will adapt automatically.
- Keep the schema, tool implementation, and validation logic under version control.
- Make compatibility changes explicit; test requiredness, types, enums, and field names against representative calls.
- Validate arguments before side effects, and return actionable errors that identify the violated contract without exposing sensitive data.
- Log the schema version and classify failures by stage so an invalid invocation is not confused with tool execution or task-planning failure.
- When changing a live contract, run regression tests for both expected valid inputs and malformed or outdated inputs.
These practices do not predict which framework will perform best in a particular task. They make framework comparisons and production failures easier to interpret.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
