Recommended Free Tools
Yes—some early users reported conspicuous mistakes from OpenAI’s o1-preview, the model reported under the codename “Strawberry.” Examples published in September 2024 included illegal chess moves, a botched river-crossing puzzle and inconsistent answers to a strawberry-themed logic puzzle. Those anecdotes show that strong benchmark results do not guarantee flawless performance on every simple-looking task, but they do not establish how often the model makes such mistakes.
What “Strawberry” refers to
“Strawberry” was the codename used in reporting; OpenAI’s public name for the model was o1-preview. OpenAI introduced it on September 12, 2024, as an early preview for ChatGPT and its API. The examples that prompted the “idiotic mistakes” framing appeared in a Futurism article published the following day.
As an Amazon Associate I earn from qualifying purchases.
That timing matters: the reported errors concern early interactions with o1-preview in 2024. They are not tests of today’s o1 deployment or of later models.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
What mistakes did early users report?
Futurism’s Victor Tangermann collected user-reported examples from several kinds of tasks. The reports were individual demonstrations, not a controlled evaluation using a representative sample of prompts and users.
- Chess: Researcher Mathieu Acher reported that the model made illegal moves.
- A river-crossing puzzle: Meta AI scientist Colin Fraser reported that it abandoned a correct answer.
- Strawberry puzzles: Users showed varying answers to a strawberry-themed logic puzzle and said the model struggled to count the letter “r” in “strawberry.”
- A riddle: One user reported waiting 92 seconds for a response. That is a single anecdotal timing, not a typical latency figure.
These examples help explain why an answer can look impressive in one setting and plainly wrong in another. But they do not tell us the probability that o1-preview would fail at chess, counting letters or puzzle-solving across ordinary use. A user’s quoted “75 percent” result referred to one prompt; it is not an overall accuracy rate.
Why strong benchmark scores do not settle the question
OpenAI’s September 12, 2024 launch announcement reported strong results on selected math and coding evaluations. Its reasoning model scored 83% on a qualifying exam for the International Mathematics Olympiad, while GPT-4o scored 13% on the same evaluation. OpenAI also reported coding performance at the 89th percentile in Codeforces competitions.
| Evidence | What it measured or described | What it does not establish |
|---|---|---|
| OpenAI’s 83% result for the reasoning model | Performance on a qualifying exam for the International Mathematics Olympiad, as reported by OpenAI in September 2024. | An error rate for everyday questions, chess, letter counting or arbitrary puzzles. |
| OpenAI’s 13% result for GPT-4o | GPT-4o’s score on that same IMO qualifying-exam evaluation, reported by OpenAI in September 2024. | That GPT-4o was less capable on every task or in every common use. |
| OpenAI’s 89th-percentile coding result | Reported performance in Codeforces competitions. | Reliability on tasks outside that coding evaluation. |
The benchmark scores and the user anecdotes answer different questions. The scores describe performance on specified evaluations; the anecdotes show that particular prompts produced errors. Neither is a general-purpose measure of correctness across all tasks, and the cited materials do not provide an independent, representative error-rate estimate for the reported basic mistakes.
The Tool Desk
Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Outbyte PC Repair FREEClear out junk files and repair common Windows errorsFree Scan →What OpenAI said about the model at launch
OpenAI said o1-preview was trained to spend more time thinking, refine its process, try strategies and recognize mistakes. It also called the release an early model and acknowledged that it lacked some ChatGPT features, including web browsing and file and image uploads. The announcement cautioned: “For many common cases GPT‑4o will be more capable in the near term.”
That qualification is important to the launch-era comparison: o1-preview’s reasoning focus did not mean it was automatically the best choice for every ChatGPT task. The announcement also said o1-mini was 80% cheaper than o1-preview at launch; that was a comparison stated in 2024, not current pricing.
What the system card can—and cannot—tell us
OpenAI’s o1 System Card, updated December 5, 2024, describes the family as trained with large-scale reinforcement learning to reason using chain-of-thought. It reports evaluations of specified checkpoints and warns that production performance can vary with system updates, final parameters, system prompts and other factors.
The card’s preparedness scorecard includes ratings such as medium for persuasion and CBRN, and low for cybersecurity and model autonomy. Those are safety-category assessments; they are not scores for everyday factual accuracy or puzzle-solving skill. The card therefore offers useful context about the model family and deployment variability, but it does not turn the 2024 user reports into a current reliability test.
So, does o1 still make “idiotic mistakes”?
The supported answer is narrower than the headline’s broad wording: early users reported obvious failures by o1-preview on specific tasks in September 2024, while OpenAI reported high performance on selected math and coding benchmarks. These facts can coexist. The anecdotes do not prove that the model commonly makes such mistakes, and the benchmark results do not prove it is dependable on every basic-looking task.
Because the cited examples concern an early preview and the system card notes that deployment can change, they should not be presented as evidence of how current o1 or successor models perform. For the specific question of how often these basic errors occur, the cited sources do not give a representative rate.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
