Halloween DealsAmazon USHalloween week deals: useful picks before month-endAmazon US: current deals, useful picks and tech finds.Check DealsSlow PC?RecommendedPC slow today? Run a repair scan before it gets worseResolve common Windows issues and optimize system performance.Scan NowHalloween DealsAmazon USOctober picks are moving - compare before checkoutAmazon US: current deals, useful picks and tech finds.See Picks×
FREEDOM251

Freedom251 › Blog › AI Research

Can AI Solve Research-Level Math Problems Reliably? Evidence Through 2026

By Freedom251 Team · · 8 min read

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Not reliably across new research problems. Current AI systems have produced genuine mathematical advances and can sometimes solve or substantially advance difficult open questions. However, controlled evaluations on curated, research-level tasks still show low, uneven success rates. A successful demonstration proves that a system can contribute; it does not establish that the system will solve the next unfamiliar research problem dependably.

The most accurate answer is therefore two-part: AI has real research-level mathematical capability, but reliable, general-purpose autonomous problem solving has not been demonstrated.

As an Amazon Associate I earn from qualifying purchases.

What “research-level mathematics” means

Research-level mathematics asks for more than applying known methods to a difficult exercise. A problem may require a genuinely new theorem, counterexample, construction, classification, or proof strategy. The necessary background can span several specialized fields, and even an apparently plausible argument may conceal a gap that takes an expert weeks to find.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Why competition performance is not enough

Olympiad and other competition benchmarks usually have a known answer, a short intended solution, and a tightly bounded range of techniques. Research problems can be open-ended, poorly specified until the work begins, and dependent on discovering useful definitions or intermediate conjectures. A model that performs well on competition mathematics has demonstrated valuable reasoning ability, but that score does not establish dependable performance on novel research.

#1 Best Overall
Texas Instruments TI-84 Plus CE Color Graphing Calculator, Black | Computer version included, USB charging, Full color, Faster than TI-84 Plus
  • Makes understanding math and science topics quicker and easier — ideal for middle school through college
  • Built-in MathPrint feature allows you to input and view math symbols, formulas and stacked fractions exactly as they appear in textbooks
  • Graph in vibrant colors to make faster, stronger connections. Powered by a TI Rechargeable Battery that can last up to one month on a single charge.
  • 4-year subscription for the TI-84 Plus CE online calculator included with purchase
  • Lightweight yet durable enough to withstand the demands of the classroom year after year

Riemann-Bench was designed around this distinction. Its authors describe a private research-level set and report that every evaluated frontier model scored below 10%. That result is not a universal measure of mathematical ability, but it is evidence that high performance on easier or more familiar tasks does not transfer automatically.

What the strongest evaluations report

There is no single “AI math accuracy” number. The benchmarks below use different problems, access rules, budgets, scoring procedures, and verification standards. Their results should be read as separate measurements.

Evaluation Task and protocol Reported result What it shows
FrontierMath initial evaluation (Epoch AI, 2024) Hundreds of expert-crafted problems across modern mathematics; intended to require specialists hours or days. Six leading language models were evaluated. No model solved more than 2%. Early frontier systems struggled on a broad, difficult research-oriented set. This is not a current score for every model.
FrontierMath v2 (Epoch AI, released June 12, 2026) Updated benchmark organized into tiers; Tier 4 is identified as research-level. Errors were addressed in 42% of problems. Benchmark quality and verifier maintenance materially affect reported results.
FrontierMath Erdős (Adamczewski and Bloom, preprint submitted September 6, 2026) 68 open Erdős problems; five systems; each problem evaluated with a shared $300 budget; a Lean proof or disproof was required. GPT-6 Astra: 3%. Each of the other four systems: 0%. Even with a substantial fixed budget, autonomous success remained rare on this fixed open-problem collection.
Soohak (version revised May 19, 2026) Mathematician-curated research-level benchmark with separate Challenge and refusal subsets. Challenge subset: Gemini-3-Pro 30.4%, GPT-5 26.4%, Claude-Opus-4.5 10.4%. No model exceeded 50% on the refusal subset. Scores vary sharply with task design; a model’s willingness to decline an unsolved or unsuitable task is itself measured.
Riemann-Bench (version revised June 23, 2026) Private “moonshot mathematics” research-level set evaluated on frontier models. Every evaluated model scored below 10%. Results remain weak on another independent, high-difficulty set.

FrontierMath’s early result

Epoch AI’s original FrontierMath account described expert-created problems spanning modern mathematics, with many expected to take a specialist hours or days. In the initial evaluation, none of six leading models solved more than 2% of the problems. Because this was an early evaluation, the figure should not be presented as a current all-model rate. It is best understood as a baseline showing how far systems had to go on carefully selected difficult tasks.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Why the Erdős result matters

The FrontierMath Erdős study narrows the question to 68 open problems associated with Paul Erdős and applies the same $300-per-problem budget to five systems. GPT-6 Astra solved 3%; the other four systems solved 0%. The authors required a Lean proof or disproof, which makes the success criterion more checkable than an unverified natural-language answer. A fixed budget and fixed collection do not represent every research setting, but they make the comparison meaningful within that study.

How to read Soohak’s higher percentages

Soohak reports substantially higher numbers on its Challenge subset: 30.4% for Gemini-3-Pro, 26.4% for GPT-5, and 10.4% for Claude-Opus-4.5. Those percentages cannot be averaged with FrontierMath or Riemann-Bench. Soohak uses its own mathematician-curated tasks and also reports a separate refusal subset, on which no model exceeded 50%. The difference illustrates how selection, expected answer format, tool access, and scoring can change a headline result.

Evidence that AI can make real mathematical contributions

Low benchmark scores do not mean that every reported advance is illusory. They mean that isolated successes should not be converted into a reliability claim.

Rank #2
Sale
Texas Instruments TI-Nspire CX II CAS Color Graphing Calculator with Student Software (PC/Mac)
  • Color Screen. The screen size is 320 x 240 pixels (3.5 inches diagonal) and the screen resolution is 125 DPI; 16-bit color
  • Rechargeable battery included. Can last up to two weeks on a single charge
  • Handheld-Software Bundle. Includes the TI-Inspire CX Student Software delivering enhanced graphing capabilities and other functionality.
  • Thin Design and lightweight with easy touchpad navigation.Quick alpha keys
  • Six different graph styles and 15 colors to select from for differentiating the look of each graph drawn

OpenAI’s October 2026 disclosure

In a disclosure dated October 6, 2026, OpenAI described a broad set of new mathematical results produced with an internal frontier model. It said that many proofs are being released in Lean formalizations and that details of the process are available. The disclosed average effort used compute equivalent to roughly three hours of ChatGPT Pro thinking. That figure characterizes the effort behind the reported results; it is not an average accuracy rate or a promise that a user will obtain a proof in three hours.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Formalized proofs provide a strong check that the stated derivation compiles in the chosen formal system. They do not, by themselves, establish that a result is important, novel, optimally formulated, or correctly attributed. Those judgments still require mathematical review.

Human-plus-AI work in the Open Problems tracker

Epoch AI’s Open Problems tracker separates autonomous solutions from “human-plus-AI” cases. Its September 2026 changelog says that this category is used when “AI was instrumental in solving a problem, but did not solve it autonomously.” That distinction matters: a model may discover a key lemma, search a large case space, or suggest a construction while a mathematician chooses the direction, repairs gaps, and establishes novelty.

The tracker also records provisional classifications and changes when verification or notability standards are not met. For example, it treats a reported Hadamard matrix solution as provisional and did not classify the M23 inverse Galois result as solved by AI because the core AI contribution was not unambiguous. These cases show why “AI solved it” is often too coarse a description.

Capability and reliability are different claims

Capability asks whether a system can produce a correct, useful contribution under at least some conditions. The OpenAI results and the human-plus-AI entries tracked by Epoch are evidence that the answer is yes.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Reliability asks whether the system succeeds at a predictable rate on a broad stream of new problems, while recognizing when it is wrong and avoiding unsupported conclusions. The reviewed evaluations do not establish that. Success rates range from zero or low single digits to higher percentages on particular subsets, and the task definitions are not interchangeable.

Rank #3
Sale
Casio fx-9750GIII Graphing Calculator, Python Programming, Black
  • USER-FRIENDLY DISPLAY – Natural Textbook Display℠ shows expressions and results exactly as they appear in textbooks, simplifying writing and interpreting complex math.
  • STUDENT FRIENDLY - Combines ease of use with advanced functionality—ideal for courses from Pre-Algebra to AP Statistics. Supports graph plotting, vectors, probability distributions, spreadsheets, eActivities, integrals, and more for a full range of math and science applications.
  • PYTHON INTEGRATION – Program with MicroPython directly on the calculator, or connect to a PC to transfer, store, or share your programs.
  • EXAM-APPROVED – Approved for use in AP, SAT, ACT, IB, and other standardized exams, making it a reliable choice for students.
  • USB CONNECTIVITY: Easily store and transfer files to and from a computer using the included USB cable.

There is also a difference between an answer that sounds convincing and a result that survives verification. A fluent proof can contain a subtle invalid inference; conversely, a system may find a useful idea without presenting a complete proof. Reliability therefore includes problem selection, intermediate checking, error detection, and appropriate refusal—not just the final percentage of accepted answers.

How to compare an AI-math claim responsibly

Before treating a result as evidence of general research ability, record the following details:

  1. Task set and difficulty: Is it a public exercise collection, an unpublished private set, or genuinely open research? Are the problems known to the model or newly held out?
  2. Model and version: Name the exact system, release, context window, and any specialized mathematical mode.
  3. Tools: State whether the model could use a computer algebra system, web search, code execution, theorem prover, retrieval system, or other external aid.
  4. Budget: Report token, time, inference, or dollar limits. The $300-per-problem limit in FrontierMath Erdős is not equivalent to an unrestricted laboratory run.
  5. Autonomy: Say whether a human selected subproblems, supplied hints, corrected errors, or decided when to stop.
  6. Verification: Identify whether answers were checked by Lean, a bespoke programmatic verifier, independent referees, or only a language-model judge.
  7. Benchmark revision and date: Note the release version and when the evaluation occurred. Epoch reported that 42% of FrontierMath problems had errors addressed in its June 12, 2026 v2 release, demonstrating why revisions can change what a score means.

Without these details, two percentages may look comparable while measuring different abilities.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

What AI is useful for in a research workflow

The evidence supports using AI as an assistant whose output is aggressively checked, not as an autonomous replacement for mathematical judgment. A practical workflow keeps the human responsible for correctness and significance.

1. Define the exact question

State the objects, assumptions, field, and desired conclusion precisely. Ask the system to list ambiguities and edge cases before attempting a proof. This reduces the risk that it silently solves a nearby but different problem.

2. Use it for exploration

Request candidate lemmas, examples, counterexamples, equivalent formulations, and small computational experiments. Treat every proposal as a conjecture. An attractive pattern can fail outside the tested range.

Rank #4
Sale
TI-84 Evo Graphing Calculator Texas Instruments, White | Intuitive Icon-based Homescreen, USB-C Charging, Python Programming, Faster Performance, Bigger Graphs than TI-84 Plus CE
  • Newest in the TI-84 series: Built for everyday classroom use
  • Icon-based home screen: Popular math tools are front and center for faster, more intuitive navigation
  • 3x faster performance: A powerful processor delivers quicker calculations and smoother graphing
  • Bigger, clearer graphs: 50% more graphing space makes it easier to see patterns and relationships
  • Simplified keypad design: Larger buttons and reduced clutter help you work faster with fewer steps

3. Separate discovery from proof

Once a promising route appears, ask for a dependency graph of lemmas and for explicit statements of every assumption. Check the simplest cases independently. Do not accept a polished narrative as evidence that all implications hold.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

4. Formalize critical steps

Where feasible, encode the argument in Lean or another suitable proof assistant, or build a programmatic verifier for finite and computational claims. Formalization can expose missing hypotheses and illegal transformations, while still leaving novelty and mathematical importance for human assessment.

5. Reproduce and challenge the result

Run the argument from a clean session, vary examples, and ask another person or system to search specifically for counterexamples. Keep unsuccessful attempts; they reveal where the method is fragile.

6. Make attribution explicit

Record which ideas came from the model, which were supplied by researchers, and who closed proof gaps. A contribution can be substantial without being an autonomous solution.

Why benchmark maintenance and refusal matter

A benchmark is not a permanent measuring stick. Problems can contain errors, have ambiguous verifiers, become widely known, or turn out not to meet the intended standard of difficulty or notability. Epoch’s revisions and removals show that maintaining a trustworthy test can be as important as running the models.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Refusal is another part of dependable behavior. A system that declines a problem it cannot justify may be more useful than one that produces an elegant but false proof. Soohak’s separate refusal subset makes this behavior visible instead of hiding it inside a single success score.

The practical verdict

AI can solve some research-level mathematics problems and can materially accelerate others. The strongest evidence includes independently checkable formalizations and documented human-plus-AI advances. But the controlled results remain inconsistent: below 10% for every model tested on Riemann-Bench, at most 3% for the best system in the fixed-budget FrontierMath Erdős evaluation, and no more than 2% for any of six systems in FrontierMath’s initial evaluation. Higher Soohak Challenge scores demonstrate that task design matters, not that a general reliability rate has been established.

For a new problem, the defensible expectation is assistance with exploration, computation, proof search, and formalization—not dependable autonomous resolution. Until evaluations cover broader and changing problem pools with transparent budgets, robust verification, and clear authorship categories, the answer to “reliably?” remains no.

Quick Recap

Bestseller No. 1
Texas Instruments TI-84 Plus CE Color Graphing Calculator, Black | Computer version included, USB charging, Full color, Faster than TI-84 Plus
Texas Instruments TI-84 Plus CE Color Graphing Calculator, Black | Computer version included, USB charging, Full color, Faster than TI-84 Plus
4-year subscription for the TI-84 Plus CE online calculator included with purchase; Lightweight yet durable enough to withstand the demands of the classroom year after year
$110.44
SaleBestseller No. 2
Texas Instruments TI-Nspire CX II CAS Color Graphing Calculator with Student Software (PC/Mac)
Texas Instruments TI-Nspire CX II CAS Color Graphing Calculator with Student Software (PC/Mac)
Rechargeable battery included. Can last up to two weeks on a single charge; Thin Design and lightweight with easy touchpad navigation.Quick alpha keys
$155.99

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

AI ResearchArtificial Intelligencemath benchmarksmathematicsproof assistants

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Recommended PC Tool
Recommended PC Tool
Outdated Drivers Are Slowing You DownFree scan - exact matches
Windows Errors? Fix Them Before They SpreadFree repair scan

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.