Quick wins for a faster PC:
Scan for outdated or missing drivers - takes under a minuteDriver Scan →Repair Windows errors before they cause bigger problemsFix Now →Fix the driver behind crashes, sound loss and screen glitchesFind Drivers →Some links on this page are affiliate links: if you buy through them we may earn a commission, at no extra cost to you.
The visual trap failed: when a sharp but dull image was paired with a blurry, vivid one, GPT-4o, Claude Sonnet 5, and Gemini 3.1 Flash Lite did not choose the colorful blur. The more consequential weaknesses were elsewhere. GPT-4o often selected an image even when both images had the same tilt, while Claude showed a strong tendency to choose the image in the second position on compression questions.
That distinction matters if you use a vision model as an image-quality judge. A model can recognize a named detail and rank obvious edits accurately, yet still be inconsistent about image order or reluctant to admit that two images are equivalent. The experiment below shows how to test those behaviors instead of trusting a single “which is better?” score.
As an Amazon Associate I earn from qualifying purchases.
What exactly was tested?
Abdullahi Dattijo built a controlled benchmark from 12 original AI-generated images. The set included scenes such as a café sign, a chalkboard menu, a harbor and a desert road. Each source image was turned into five paired comparisons, with one controlled change at a time:
- blur;
- grain;
- heavy JPEG compression;
- darkening; and
- a deliberate trap pairing sharp, dull colors against blurry, vivid colors.
Those edits produced 60 image pairs. Every pair received three questions, and every question was asked twice with the image order reversed. GPT-4o, Claude Sonnet 5 and Gemini 3.1 Flash Lite therefore answered 1,080 questions in total. Answers used image A or image B, while the tilt question also allowed SAME.
#1 Best Overall
- Read Before You Buy — No Video Output: These adapters support charging and USB 2.0 data transfer, but cannot transmit video signals. Except for standard USB webcams (which use USB data only), they are not compatible with HDMI/DisplayPort cables, video-capable USB-C hubs, or docking stations with video output.
- Convert USB-A Ports to USB-C: Designed to connect USB-C earphones, cables, flash drives, card readers, and other USB-C accessories to standard USB-A ports. Plug-and-play with no drivers or software required.
- Aluminum Alloy Housing: Built with a sturdy aluminum alloy shell that aids in heat dissipation and protects against daily wear and scratches. Designed to maintain a stable and secure connection.
- Compact & Travel-Friendly: The ultra-compact design allows the adapter to stay plugged into your device without blocking adjacent ports or adding bulk, reducing wear and tear on your original USB ports.
- 12-Month Warranty: Backed by a 12-month manufacturer warranty for peace of mind. Designed to meet strict quality control standards for reliable everyday performance.
The design isolates one visual property at a time. That makes a failure easier to diagnose than a contest between unrelated photographs, but it also means the result does not predict how a model will judge a busy real-world image with several differences at once.
Did the models identify the obvious quality difference?
On ordinary, non-trap pairs, all three systems were generally strong at the broad “which is better?” decision. They were also asked to identify a named detail, which tests whether the model is judging the requested property rather than producing a generic preference.
Recommended Free Tools
| Measure | GPT-4o | Claude Sonnet 5 | Gemini 3.1 Flash Lite |
|---|---|---|---|
| “Which is better?” on plain pairs | 97.9% | 92.7% | 100.0% |
| Named-detail accuracy on plain pairs | 100.0% | 86.5% | 100.0% |
| Named-detail accuracy on trap pairs | 100.0% | 100.0% | 100.0% |
| Chose the vivid, blurry image in trap pairs | 0.0% | 0.0% | 0.0% |
The trap result is useful but narrow. It shows that these tested versions did not simply equate saturation or visual excitement with quality when sharpness and colorfulness were placed in conflict. It does not prove that the models have a general resistance to attractive images, nor that they will behave the same way with photographs, illustrations or newer model releases.
What happened when the image order changed?
Swapping the images is one of the simplest ways to expose a position shortcut. If a model tracks the underlying image, its answer should change from A to B when the images change places. If it keeps choosing the same letter, it is following a screen position instead.
Across the swap checks, the percentage of cases in which the model gave the same letter after the images were exchanged was 0.0% for GPT-4o, 15.0% for Claude Sonnet 5 and 0.0% for Gemini 3.1 Flash Lite. The aggregate result makes GPT-4o and Gemini look position-consistent in this particular measure, while Claude shows a limited same-letter tendency.
Rank #2
- 5-in-1 USB-C Hub: Experience comprehensive connectivity featuring a Power Delivery input, two USB-A 2.0 ports, a USB-A 3.0 port, and an HDMI port. (Note: The USB-C power delivery input port is only for connecting an external wall charger to power your laptop and cannot power peripheral devices.)
- 90W Pass-Through Charging: Achieve optimal charging with 90W pass-through power to your laptop, supported by a total input of 100W, with the hub reserving 10W for operational efficiency. (Note: Wall charger not included.)
- Quick Data Transfers: Accelerate your productivity with rapid data transfers using a high-speed 5Gbps USB 3.0 port and two 480Mbps USB 2.0 ports.
- 4K HDMI Display: Enhance your visual experience with a hub capable of delivering 4K resolution at 30Hz in both mirror and extend modes. Please note that this hub is compatible with MacBook (macOS 12 and newer), Windows 10 and 11, ChromeOS, and laptops equipped with DP Alt Mode and Power Delivery. Note: This device is not compatible with Linux.
- What You Get: Anker USB-C Hub (5-in-1, 4K HDMI), welcome guide, 18-month warranty, and our friendly customer service.
A more striking pattern appeared in the compression subset: Claude selected the second-position image in 23 of 24 questions. That is a different cut of the data from the aggregate same-letter statistic, but it demonstrates why a single headline score can hide a localized failure. A model may look stable over all transformations and still default to the second slot for one question family.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchPC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11Why did a model choose an image when neither image was different?
The tilt question included SAME, allowing the judge to say that the two images did not differ on the requested property. Even with that escape, GPT-4o selected one image in 72.5% of no-difference trials. Claude did so in 17.5% and Gemini 3.1 Flash Lite in 28.3%.
This is not evidence that the models detected a hidden visual difference. In a forced comparison, a language model may treat choosing A or B as the expected completion, infer that the user wants a decisive answer, or rely on a positional pattern learned from the prompt format. The benchmark records the behavior; it cannot identify which internal mechanism caused it.
For practical evaluation, the willingness to answer SAME deserves its own score. A judge that ranks genuine differences well but invents a winner on equivalent inputs can create false confidence in automated curation, dataset cleaning or quality-control pipelines.
Which metrics should an image-quality benchmark report?
A useful report separates at least five dimensions instead of collapsing every answer into one accuracy number.
The Tool Desk
Outbyte PC Repair FREERepair Windows errors before they cause bigger problemsFix Now →Outbyte Driver Updater FREEFix the driver behind crashes, sound loss and screen glitchesFind Drivers →Rank #3
- Sleek 7-in-1 USB-C Hub: Features an HDMI port, two USB-A 3.0 ports, and a USB-C data port, each providing 5Gbps transfer speeds. It also includes a USB-C PD input port for charging up to 100W and dual SD and TF card slots, all in a compact design.
- Flawless 4K@60Hz Video with HDMI: Delivers exceptional clarity and smoothness with its 4K@60Hz HDMI port, making it ideal for high-definition presentations and entertainment. (Note: Only the HDMI port supports video projection; the USB-C port is for data transfer only.)
- Double Up on Efficiency: The two USB-A 3.0 ports and a USB-C port support a fast 5Gbps data rate, significantly boosting your transfer speeds and improving productivity.
- Fast and Reliable 85W Charging: Offers high-capacity, speedy charging for laptops up to 85W, so you spend less time tethered to an outlet and more time being productive.
- What You Get: Anker USB-C Hub (7-in-1), welcome guide, 18-month warranty, and our friendly customer service.
Correctness for a named quality
State the property explicitly—sharpness, compression, brightness or another defined attribute—and score whether the selected image has that property. This prevents a model from receiving credit for an unexplained aesthetic preference.
Resistance to image-order bias
Ask every comparison in both orders. Record both the chosen image identity and its position. Report same-letter-after-swap frequency and the share of answers that favor the first or second slot.
Use of SAME on equivalent inputs
Include deliberately matched pairs and offer SAME. Measure how often the model abstains from inventing a difference, not just how often it chooses the “correct” image.
Robustness across transformations
Keep the source image fixed while changing one property at a time. A judge that handles blur may still fail on compression or darkening, so publish per-transformation results rather than only a grand average.
What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
Reproducibility
Save the exact prompt, image order, model version, temperature, image files and response. Repeat a sample to see whether the result survives a new run. Without those records, a later model update or a minor wording change can look like a mysterious regression.
Rank #4
- Dual Converters, Infinite Potential：Includes 2× USB C male to USB A female adapters and 2× USB A male to USB C female adapters. Perfect for a wide range of uses—tablets with Bluetooth keyboards, expand USB ports on macbook, and more. Two different converters for all your daily needs
- Next-Level 10Gbps & 3A Charging: No more slow 480Mbps, this usb to usb c adapter has a transfer speed of up to 10Gbps, allowing you to do more transferring in less time. This usb adapter fits both USB A and USB C charger, supporting up to 3A fast charging
- Upgraded Exquisite Craftsmanship: With an aluminum alloy housing and metal connector, the usbc to usb adapter is extremely durable and sturdy. Rigorously tested to withstand more than 10,000 times of plugging and unplugging, ensuring long-lasting performance
- Broad Compatible: The usb c to usb adapter widely supports all USB C/ USB A devices like laptops, tablets, cellphones, car chargers, and phone chargers. Such as compatible with MacBook Pro/Air 2023/2022, Thunderbolt 4/3 Devices,Apple MagSafe Watch 9/8/7/SE/Ultra, iPad Pro 2022/2021, Samsung Galaxy S23/S20/S10, and iPhone 17/16/15 Pro. Plug and play
- Please Note: To reach 10Gbps speed, keep the cable under 3.3 ft. For USB A Male to USB C adapters, try flipping the USB C connector. USB C Male to USB A adapters support bidirectional 10Gbps transfer within 3.3 ft
How can you reproduce the test?
The original workflow can be recreated with Python 3.9 or newer, NumPy, Pillow, an OpenAI API key, and a folder of rights-cleared PNG files. The important requirement is not a particular library command; it is that every edit and every prompt be fixed and logged.
- Prepare the source set. Collect a varied group of PNGs for which you have usage rights. Keep the originals untouched and assign stable names such as
scene_01.png. - Create controlled variants. Use Pillow to make blur, grain, heavy JPEG-compression and darkening versions. Choose one parameter value for each transformation, apply it consistently, and store the parameter in your log. Make the trap pair by combining sharpness and colorfulness in opposite directions.
- Write matched questions. Prepare one quality-ranking question, one named-detail question and one tilt question that explicitly permits SAME. Use the same wording for every model in the first pass.
- Run both orders. Submit each pair as A/B and then as B/A. Do not rely on a model to remember an earlier answer; treat every request as an independent trial.
- Fix and record model settings. The original test used temperature 0 for GPT-4o and Gemini 3.1 Flash Lite, while Claude Sonnet 5 used its default. For a new benchmark, either hold settings constant where the APIs permit or clearly label the difference. Record the model name and version because these findings describe a fixed point in time.
- Save structured responses. For every question, record the source image, transformation, prompt wording, order, model, setting, raw response, parsed choice, chosen image identity and whether SAME was available.
- Calculate the metrics. Compare the parsed choice with the known edited image, calculate named-detail accuracy, count SAME responses on matched pairs, and compare answers before and after swapping. Break out each transformation and each model.
- Inspect the failures. Read the raw responses behind surprising scores. A model may answer with an explanation that mentions the correct visual property but still select the wrong letter, or it may select a letter while admitting that the images look equivalent.
An experiment-tracking dashboard can make this easier: use one run per judge, attach the prompt and image metadata, and generate a comparison report from the same recorded fields. The dashboard is useful for auditability, but it cannot correct a flawed pair design or an ambiguous question.
How should the results be interpreted against other benchmarks?
A contextual benchmark reported by VisionQ authors in 2026 cited 63.1% as the best AI-judge accuracy in that benchmark and 32.2% as a chance-level result. Those figures are not measurements from Dattijo’s experiment: he did not run VisionQ’s images, and the datasets, prompts and scoring rules differ. They can provide context for how widely reported judge performance varies, but they should not be placed on the same leaderboard without a matched protocol.
The custom test is best read as a diagnostic stress test. Its value is showing which failure modes appear when the visual conflict, order and no-difference option are deliberately controlled. It is not a certification that any model is a reliable universal image critic.
What are the experiment’s main limitations?
- Small, artificial sample: only 60 pairs came from 12 generated images, and each pair changed one controlled property.
- One wording per question: alternate phrasing was not tested, so the scores may reflect prompt wording as well as visual judgment.
- Unequal temperature settings: GPT-4o and Gemini used temperature 0, while Claude used its default setting.
- Model-size difference: Gemini 3.1 Flash Lite is the smaller Flash Lite model, so its behavior should not be generalized to every model in that family.
- Limited SAME coverage: SAME was available for the tilt question, not every comparison.
- Version sensitivity: the findings describe the versions available for the test; model updates can change results.
- Generalization risk: generated scenes and isolated edits do not represent real photographs containing exposure, focus, composition, subject and color changes at the same time.
What should a team do before trusting an AI image judge?
Start with a small paired pilot, but treat it as a bias audit rather than a proof of quality. Include clean pairs, no-difference pairs, reversed orders and transformations that matter to your application. Require the model to name the property it used and preserve the raw answer so a human can review disagreements.
Best Value
- 5-in-1 Connectivity: Equipped with a 4K HDMI port, a 5 Gbps USB-C data port, two 5 Gbps USB-A ports, and a USB C 100W PD-IN port. Note: The USB C 100W PD-IN port supports only charging and does not support data transfer devices such as headphones or speakers.
- Powerful Pass-Through Charging: Supports up to 85W pass-through charging so you can power up your laptop while you use the hub. Note: Pass-through charging requires a charger (not included). Note: To achieve full power for iPad, we recommend using a 45W wall charger.
- Transfer Files in Seconds: Move files to and from your laptop at speeds of up to 5 Gbps via the USB-C and USB-A data ports. Note: The USB C 5Gbps Data port does not support video output.
- HD Display: Connect to the HDMI port to stream or mirror content to an external monitor in resolutions of up to 4K@30Hz. Note: The USB-C ports do not support video output.
- What You Get: Anker 332 USB-C Hub (5-in-1), welcome guide, our worry-free 18-month warranty, and friendly customer service.
Set acceptance thresholds separately for ranking accuracy, SAME usage and order consistency. If a model fails the no-difference check, route uncertain cases to a human instead of forcing a winner. If performance changes when you swap A and B, redesign the prompt or parser and retest; do not repair the metric by ignoring the offending trials.
Finally, rerun the suite after a model version, temperature, image-processing library or prompt change. The central lesson from this test is not that vision judges are useless. It is that “which image is better?” is too broad a question to expose their specific weaknesses. A benchmark that tests order, abstention and controlled transformations can show you where a judge is dependable—and where it is merely completing the format.
Frequently Asked Questions
Can I use the same benchmark prompts for every vision model?
Use matched prompts for a fair first comparison, but add a second pass with equivalent wording. Prompt phrasing can influence whether a model treats the task as a forced choice, so a single template should not be treated as a universal measurement.
What does a good SAME option look like?
Define SAME in the prompt as “neither image is better on the named property” and include truly matched pairs. Keep SAME available whenever the task can produce an equivalence; offering it only in one question type limits what the score means.
How many images are enough for a trustworthy benchmark?
There is no universal cutoff established by this experiment. Twelve source images are useful for finding obvious failure modes, but a production decision should expand the set across subjects, styles and capture conditions and report uncertainty around each metric.
Should I test generated images or photographs?
Generated images are convenient for isolating one edit because the source can be held constant. Add rights-cleared photographs before deployment, since real scenes combine focus, lighting, noise, composition and subject differences that a one-edit pair does not capture.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
