Harvard Just Published a Study Showing o1 Out-Diagnoses ER Doctors. The Scariest Part Isn't the AI — It's the Humans.

A Harvard study just dropped in Science. The headline number is spicy: OpenAI's o1 reasoning model correctly diagnosed 67% of emergency room patients versus 50-55% for the human doctors given the exact same patient records. In a separate task — writing long-term treatment plans — o1 scored 89% while 46 human doctors mustered 34%.

If those numbers don't make you pause, you haven't been paying attention to what "clinical reasoning benchmark" progress actually means.

But here's the thing: the headline that's going viral is missing the genuinely unsettling part. It's not that an LLM beat doctors on a narrow diagnostic task. That was already plausible — these models have been passing the USMLE for a while, and diagnostic pattern-matching from text is exactly the kind of thing transformers are good at. The unsettling part is buried deeper in the study, and in how the doctors reacted.

The Study, Actually

The trial was run at Boston's Beth Israel Deaconess Medical Center, led by Arjun Manrai (Harvard Medical School) and Dr. Adam Rodman. Published in Science on April 30. They gave 76 real ER patient records — vital signs, demographics, a nurse's brief intake note — to both o1 and pairs of human doctors. No visual examination. No smelling the patient. No noticing they're shivering or avoiding eye contact. Just text.

AI: 67% correct. Humans: 50-55%.

When more clinical detail was available later in the patient's stay, the gap narrowed (AI 82% vs humans 70-79%), and that difference wasn't statistically significant. But here's where it gets interesting: the AI's treatment plans were where it really pulled away. The AI and 46 doctors each designed care plans for five clinical cases. AI scored 89%. Doctors using conventional resources (search engines, references) got 34%.

Thirty-four percent. That's not a typo.

The study had a particularly compelling anecdote: a patient presented with a lung blood clot that was getting worse. The human doctors assumed the anti-coagulants were failing. The AI noticed something in the patient history they didn't — a lupus diagnosis — and correctly identified that as the root cause of the lung inflammation. The AI was right.

The Methodology Asterisks (Because There Are Always Methodology Asterisks)

Let's be fair. The study didn't test the full sensory experience of an ER doctor. No visual cues, no physical exam, no reading the patient's body language, no noticing they smell like a diabetic crisis. The AI was doing a purely text-based task — more like a radiologist reading a chart than an ER attending managing a chaotic bay.

HN commenter gpm pointed out a related risk: a recent paper showed AI beating radiologists on X-ray interpretation... when the AI didn't actually have access to the X-rays. The benchmark was accidentally leaking the answer through side channels in the text. These things happen more often than anyone wants to admit.

Dr. Wei Xing from the University of Sheffield also flagged that the study didn't break down which patients the AI was worse at diagnosing. Was it struggling more with elderly patients? Non-native English speakers? We don't know. That matters a lot before anyone deploys this in a real triage setting.

The Actually Scary Part: Human Deference

There's a behavioral finding buried in the paper that should keep you up at night. When doctors were shown the AI's answer, they tended to unconsciously defer to it — even when they had been right and the AI was wrong.

This is the automation bias problem that aviation has been dealing with for decades. Pilots trust the autopilot even when their own instruments say otherwise. Now it's showing up in medicine, and the stakes are considerably higher than a go-around at LaGuardia.

Rodman himself said: "There is not a formal framework right now for accountability." If an AI says "this is lupus" and the doctor goes with it and the patient dies — who's liable? The hospital? OpenAI? The doctor who deferred? Right now the answer is "nobody knows, and that's the problem."

Meanwhile, nearly one in five US physicians are already using AI to assist with diagnosis. In the UK it's 16% daily and 15% weekly. These doctors aren't waiting for liability frameworks. They're just... using it. Anecdotally, multiple HN commenters noted they've seen their own doctors consulting ChatGPT during visits. Not as a reference tool — as a colleague.

The Triadic Care Model

Manrai and Rodman aren't arguing for AI replacement. They're proposing something they call the "triadic care model" — doctor, patient, and AI system working together. The AI as a second-opinion engine that catches things humans miss (like the lupus case), while the human handles everything the AI can't: physical exams, emotional support, shared decision-making, reading the room.

Prof. Ewen Harrison from the University of Edinburgh said the study showed these systems "are no longer just passing medical exams or solving artificial test cases. They are starting to look like useful second-opinion tools for clinicians, particularly when it is important to consider a wider range of possible diagnoses and avoid missing something important."

That framing — "useful second-opinion tools" — is probably the right one, and it's a lot less exciting than "AI REPLACES DOCTORS" clickbait. But it's also a lot more realistic.

What This Actually Means

The honest read is: AI diagnostic reasoning has crossed a threshold that matters. Not because it's flawless (it isn't) or because it should replace doctors (it shouldn't). But because the gap between what a good LLM can do and what an average doctor does with the same text-based information is now measurable and significant.

The question isn't whether AI will be part of clinical medicine. It already is — 20% of US doctors are on board without anyone's permission. The question is whether we build the frameworks — liability, training, workflow integration, bias monitoring — before the technology outpaces our ability to control it.

Knowing how tech adoption usually goes, I'm not optimistic about that timeline.

Sources: The Guardian coverage of the Science paper (link); Hacker News discussion (thread); the referenced X-ray benchmark shenanigans paper (arXiv:2603.21687); Royal College of Physicians survey on UK physician AI usage.