What’s actually slowing this PC down?
Pick the symptom - the matching free tool is one click away.
In a 2024 study, ChatGPT 3.5 supplied the correct final diagnosis in 74 of 150 medical case challenges—49% of that test set. That result is a warning against treating the chatbot as a doctor, but it is not a measure of every ChatGPT model, every medical question, or real-world patient outcomes.
What did the study actually test?
Ali Hadi, Edward Tran, Branavan Nagarajan, and Amrit Kirpalani evaluated ChatGPT 3.5 using 150 Medscape Clinical Challenges published between September 2021 and January 2023. The cases were written clinical scenarios, not live consultations with patients. The study appeared in PLOS ONE on July 31, 2024. Read the study.
As an Amazon Associate I earn from qualifying purchases.
The headline “ChatGPT Is Absolutely Atrocious At Being a Doctor” came from a Futurism article published August 18, 2024. The study behind it assessed a specific model on a defined set of case challenges; it did not test whether ChatGPT can practice medicine or improve patient outcomes. Futurism’s coverage.
Outdated Drivers Are Slowing You Down
One free scan finds every outdated or missing driver and matches the right update for your exact hardware.Free scan · exact hardware matchPC Slower Than It Used to Be?
A free scan shows the junk files, broken settings and background clutter dragging Windows down - then fixes them in one click.Free scan · Windows 10 & 11How often did it identify the final diagnosis?
ChatGPT 3.5 gave the correct final diagnosis in 74 of 150 cases, or 49%, according to the authors. That is the clearest answer to how often it got the final diagnosis right in this test.
#1 Best Overall
- SMART STETHOSCOPE — The CORE 500 is the modern stethoscope replacement, blending 3-lead ECG with AI insights, unparalleled audio clarity, waveform visualizations, and exam recording and sharing capabilities.
- AI DETECTION WITH EKO+ — Your purchase includes a free 14-day Eko+ trial to unlock murmur and AFib detection, plus unlimited recording. Membership is $119.99/year afterwards. You can downgrade anytime. Even without Eko+, you can enjoy basic features of the app.
- SEE MORE INSIGHTS — Visualize what you’re hearing during your exam. Connect to the Eko App for waveform visualization and single sound recording with real-time playback during exams.
- NEXT-GEN AUDIO — Advanced audio technology minimizes artifact and delivers the most precise sound with background noise reduction and up to 40x amplification. Pick up heart, lung, and body sounds with precision using Cardio, Pulmonary, and Wide audio filters.
- FULL-COLOR DISPLAY — Heart rate and ECG data, exam insights, and device settings are visible directly on the stethoscope’s screen for a comprehensive view of your patient’s heart.
The paper also reports 74% overall accuracy. That is a separate metric and should not be substituted for the final-diagnosis result. It additionally reports precision and sensitivity of 48.67%, specificity of 82.89%, and an area under the curve (AUC) of 0.66. These figures describe different aspects of the study’s evaluation; they do not mean the chatbot named the correct final diagnosis in 74% of cases.
Why could the answers go wrong?
The authors report that ChatGPT 3.5 struggled to interpret laboratory values and imaging results and could overlook details relevant to a diagnosis. In a clinical case, a small finding may change which explanation is most plausible, so a fluent response is not evidence that the model has noticed or correctly weighed every important clue.
Rank #2
- The 3M Littmann CORE Stethoscope connects with Eko software on a smart device to visualize, record and share data. (Smart device not included. Some features require a subscription)
- Connects to Eko software to visualize and share heart sound waveforms
- Up to 40x amplification (at peak frequency, vs. analog mode)
- Active noise cancellation reduces unwanted background sounds
- Toggle between analog and amplified listening modes; Designed for use with adult and pediatric patients
The authors concluded that “ChatGPT in its current form is not accurate as a diagnostic tool.” That conclusion applies to the model and test they studied; it is not a blanket measurement of newer models or a clinical trial finding about patient care.
Free tools Windows power users keep installed
One-click scans. No signup required.
Does this result apply to newer ChatGPT models?
No. The study tested ChatGPT 3.5 on cases from a particular source and time period. It does not establish how a newer model, a different configuration, or a different way of presenting a case would perform. Nor does a case-challenge score establish safety or effectiveness in a clinic.
Rank #3
- SMART STETHOSCOPE — The CORE 500 is the modern stethoscope replacement, blending 3-lead ECG with AI insights, unparalleled audio clarity, waveform visualizations, and exam recording and sharing capabilities.
- AI DETECTION WITH EKO+ — Your purchase includes a free 14-day Eko+ trial to unlock murmur and AFib detection, plus unlimited recording. Membership is $119.99/year afterwards. You can downgrade anytime. Even without Eko+, you can enjoy basic features of the app.
- SEE MORE INSIGHTS — Visualize what you’re hearing during your exam. Connect to the Eko App for waveform visualization and single sound recording with real-time playback during exams.
- NEXT-GEN AUDIO — Advanced audio technology minimizes artifact and delivers the most precise sound with background noise reduction and up to 40x amplification. Pick up heart, lung, and body sounds with precision using Cardio, Pulmonary, and Wide audio filters.
- FULL-COLOR DISPLAY — Heart rate and ECG data, exam insights, and device settings are visible directly on the stethoscope’s screen for a comprehensive view of your patient’s heart.
Other studies can report different results when they use different specialties, cases, model versions, inputs, or scoring methods. For example, a separate 2024 study of neurology cases reported higher diagnostic performance in its own setup. It is context, not a head-to-head comparison with the 150 Medscape challenges. Read the neurology study.
Can ChatGPT still be useful in medical learning?
The PLOS ONE authors discuss possible educational uses, including simplifying complex concepts and suggesting differential diagnoses or next steps. Those are potential supports for learning, not evidence that the model can safely make an autonomous diagnosis. A learner would still need to verify its claims against reliable clinical sources and qualified supervision.
Rank #4
- SMART STETHOSCOPE — The CORE 500 is the modern stethoscope replacement, blending 3-lead ECG with AI insights, unparalleled audio clarity, waveform visualizations, and exam recording and sharing capabilities.
- AI DETECTION WITH EKO+ — Your purchase includes a free 14-day Eko+ trial to unlock murmur and AFib detection, plus unlimited recording. Membership is $119.99/year afterwards. You can downgrade anytime. Even without Eko+, you can enjoy basic features of the app.
- SEE MORE INSIGHTS — Visualize what you’re hearing during your exam. Connect to the Eko App for waveform visualization and single sound recording with real-time playback during exams.
- NEXT-GEN AUDIO — Advanced audio technology minimizes artifact and delivers the most precise sound with background noise reduction and up to 40x amplification. Pick up heart, lung, and body sounds with precision using Cardio, Pulmonary, and Wide audio filters.
- FULL-COLOR DISPLAY — Heart rate and ECG data, exam insights, and device settings are visible directly on the stethoscope’s screen for a comprehensive view of your patient’s heart.
Kirpalani, one of the study authors, said that guidance around prompt engineering may be needed as generative AI is used in medical education. Western University’s coverage reports his comments and identifies him as an assistant professor in its Department of Paediatrics. Western University’s coverage.
What should patients take away?
The 49% figure means ChatGPT 3.5 supplied the correct final diagnosis in 74 of the 150 study cases. It does not tell you the odds that a chatbot will correctly diagnose your own symptoms. A case challenge lacks the full context of an individual’s history, examination, testing, and follow-up, and this study did not measure patient outcomes.
Best Value
- The 3M Littmann CORE Stethoscope connects with Eko software on a smart device to visualize, record and share data. (Smart device not included. Some features require a subscription)
- Connects to Eko software to visualize and share heart sound waveforms
- Up to 40x amplification (at peak frequency, vs. analog mode)
- Active noise cancellation reduces unwanted background sounds
- Toggle between analog and amplified listening modes
Futurism’s article relayed Kirpalani’s caution, “They should not replace your doctor yet,” via Live Science. This is a media-reported quotation, not a formal clinical guideline. Live Science’s report.
Quick Recap
Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.
