Six Reasons We Are Not Ready for Autonomous AI Doctors

I have been carrying two papers around for a few days now. Both came out in the same journal, on the same day, in the same week. They are pulling in opposite directions.

The first paper said AI's job is to dig the doctor out from under the pile of paperwork, so the doctor can be at the bedside again (Martinelli et al., 2026). That paper has been on my mind. The argument is one I find convincing — that AI is a tool that gives clinicians their work back.

The second paper, by Bergman, Wachter, and Emanuel, takes the opposite line. Same journal. Same week. The authors propose that AI should be licensed to see patients on its own (Bergman et al., 2026). To diagnose. To recommend treatment. To triage — that is, to decide who is sick enough to see a real doctor and who can be sent home.

All of that without a doctor reviewing each case.

What the proposal actually says

The authors are not careless. They are serious people making a serious case. Their words for what they are proposing are exact:

Autonomous clinical AI refers to systems that make care determinations, including diagnoses, treatment recommendations, and triage decisions, without clinician review of each case.

That is from the JAMA paper itself (Bergman et al., 2026). The word autonomous here means on its own. No human looking over the shoulder of every visit. The AI sees the patient. The AI decides.

Why this is different from what already exists

AI in medicine is not new. A lot of hospitals already use it. But almost everywhere, the AI is a helper — it suggests, and a doctor decides. A radiologist looks at the AI's read of a scan and signs off. A primary care doctor looks at the AI's draft of a visit note and edits it.

That is not what this paper is proposing. The paper is proposing that AI should be allowed to do the doctor's job. Not just help with it.

The case for this is not silly. There is a real shortage of doctors. There are entire counties in the United States with no primary care physician at all. Some rural areas have lost more than a thousand family doctors in just six years (Fogarty et al., 2025). For someone living there, no doctor is the current option. A licensed AI is, at minimum, more than that.

So the question is not silly either. It is just the question I think we are answering too fast.

Where I land, and where I want to go next

The space between those two papers is where I want to live for a minute. Same journal, two weeks apart. One paper saying AI is here to give the doctor back. The other saying we should let AI be the doctor.

My honest read is that the world is not ready for the second one yet. But I do not want to wave that away. The next chapter is about taking the strongest version of the case for before I lay out why I do not think we are there.

References

Bergman, A., Wachter, R. M., Emanuel, E. J. (2026). A Licensure Framework for Autonomous Clinical AI. JAMA. doi:10.1001/jama.2026.5483

Martinelli, C., Carnevale, V., Ercoli, A., et al. (2026). Artificial Intelligence Is Not the End of the Physician. JAMA. doi:10.1001/jama.2026.4356

Fogarty, C. T., Byun, H., Huffstetler, A. N. (2025). Family physician workforce trends: the toll on rural communities. Annals of Family Medicine, 23(6), 535–538. doi:10.1370/afm.240549

Before disagreeing with someone smart, you have to be able to make their argument better than they made it. That is the rule. Otherwise you are just yelling at a version of the argument you made up yourself.

So let me try.

The shortage is real, and it is getting worse

The United States is short of doctors. Projections in the JAMA paper point to tens of thousands of physician shortfall in the coming decade (Bergman et al., 2026). The reasons are simple. People are getting older. Older people need more care. Doctors are leaving the job faster than new ones are arriving. Rural areas are hit the hardest.

The authors put the problem plainly. Building more doctors takes time. Medical school is four years. Residency is three to seven more. By the time we train our way out of this shortage, a generation of patients will already have gone without.

People in those rural counties are not waiting for a policy paper. They are waiting for someone — anyone — to look at the rash, listen to the cough, refill the prescription.

The AI has actually gotten good

This is the part that is easy to miss. AI in medicine has changed in the last two or three years. It is not what it was.

The paper cites a 2025 study from Kenya that ran AI-supported decision tools across nearly 40,000 primary care visits (Korom et al., 2025). The visits with AI support had fewer wrong diagnoses and fewer wrong treatments than the visits without.

It cites another study from Pakistan in early 2026 — a true randomized trial, the kind we trust most. Doctors trained to use large language models (the kind of AI behind tools like ChatGPT) had much better diagnostic reasoning than doctors who were not trained to use them (Qazi et al., 2026).

And it cites the NOHARM study, posted in late 2025, which compared the strongest current language model to human physicians on safety and on completeness of clinical responses. The result, in the authors' own words:

The strongest large language model outperformed physicians on safety… clinicians not outperforming large language models on any metric.

That is a real result (Wu et al., 2025). The AI was safer than the doctors on the test. Read it again.

The licensing idea is more careful than it sounds

If you only read the headline, license AI to practice medicine sounds reckless. Reading the actual proposal, it is more careful than that.

The authors say the AI would have to:

Pass standardized exams. Score at or above the median of recent human test takers on all three parts of the US Medical Licensing Examination.

Do a residency. Enter a supervised deployment period before being trusted to work alone — same idea as a new doctor.

Stay in its lane. Be allowed to do certain things (gather histories, recommend next steps in primary care) and not others (prescribe certain medicines without a clinician approving).

Renew its license. Every two years or so, prove again that it is still safe.

Be subject to discipline. If outcomes go bad, the license can be restricted, suspended, or taken away.

That is not reckless. That is more like how we license a doctor. Or a teacher. Or a driver. Pass a test. Do supervised practice. Stay within a scope. Get checked again later.

Where this argument actually stands

So we have a real shortage, real new AI ability, and a real proposal for how to put guardrails on autonomous use. None of those three are nothing. Anyone who waves the whole thing away has not read the paper.

And yet.

The next chapter is the six things I keep coming back to. The reasons that, even after granting all of this, I do not think the world is ready for an AI doctor without a human doctor in the room.

References

Bergman, A., Wachter, R. M., Emanuel, E. J. (2026). A Licensure Framework for Autonomous Clinical AI. JAMA. doi:10.1001/jama.2026.5483

Korom, R., Kiptinness, S., Adan, N., et al. (2025). AI-based clinical decision support for primary care: a real-world study. arXiv. doi:10.48550/arXiv.2507.16947

Qazi, I. A., Ali, A., Khawaja, A. U., et al. (2026). Large language model diagnostic assistance for physicians in a lower-middle-income country: a randomized controlled trial. Nature Health, 1(2), 198–205. doi:10.1038/s44360-025-00007-8

Wu, D., Haredasht, F. N., Maharaj, S. K., et al. (2025). First, do NOHARM: towards clinically safe large language models. arXiv. doi:10.48550/arXiv.2512.01241

I want to be careful here. Saying wait has a real cost. People in counties without a doctor are not a hypothetical. But going now has a cost too. And the costs land on the same patients.

These are the six things I keep coming back to.

1. Tests are not patients

Even the JAMA authors say this in passing. Their own sentence:

Performance on written examinations provides important but incomplete evidence of clinical competence.

That is from the paper itself (Bergman et al., 2026). Passing the doctor's exam is not the same thing as taking care of a patient. The exam asks for the textbook answer. The patient asks for the answer for them — sitting in this room, with this history, with the thing they are too scared to say out loud.

Think of a kid who can score perfectly on a math test about swimming. That does not mean you let them lifeguard the pool.

2. Most of the evidence is for AI-helped doctors, not AI alone

Read the studies the paper cites and a pattern shows up. The Kenya study was about doctors using AI tools (Korom et al., 2025). The Pakistan trial was about doctors trained to use language models well (Qazi et al., 2026). Even the NOHARM study compared the AI to physicians on benchmarks, not in the actual job of seeing real patients on its own (Wu et al., 2025).

All of that evidence supports the first paper's idea — that AI helps doctors do their job better. None of it tells us much about AI on its own, with no doctor in the loop, taking care of real people.

That is a different question. We barely have data for it.

3. AI fails confidently

Current AI systems have a particular kind of mistake that is dangerous in medicine. They get things wrong while sounding right. The technical word is hallucination — when the AI invents a fact and presents it as true.

Imagine a friend who never says I do not know. Every question gets an answer. The answers sound good. About 9 out of 10 times the answer is right. The other time, it is wrong, and the friend is just as sure as before.

With a doctor in the loop, this is recoverable. The doctor catches the wrong answer. Without a doctor in the loop, the wrong answer is the diagnosis. There is no second pair of eyes.

4. People are not ready

This is the reason most of the policy papers leave out, and the one I keep returning to.

Even if AI passed every exam, even if it had perfect data, even if it never hallucinated — people are not ready to be cared for by an AI alone when they are sick or scared.

A 2026 study in JAMA Network Open surveyed 3,000 US adults on what they actually want from medical AI (Bracic et al., 2026). The pattern was clear. People were much more comfortable with AI when a clinician was present. When the AI was approved by a trusted body. When the data behind it looked like them. The thing patients wanted most was a real human in the room.

A separate 2025 randomized survey of 1,762 US adults found that just mentioning a doctor used AI made some patients trust the doctor less (Chen and Cui, 2025). That is not a tech problem. That is a relationship problem.

Care is not only about the right answer. It is about being heard. About someone making eye contact when the news is hard. About the small human things that happen between two people in a room. We do not yet know how to give that without a person there.

5. The first deployments will land on people who already get less

Read the paper carefully and you will find the example the authors give. Autonomous AI is most needed, they say, in rural federally qualified health centers — clinics that serve communities without much else (Bergman et al., 2026). That is the example.

That is also the worry.

People in well-resourced places will keep their human doctors. People in poor and rural places will be the first to be cared for by an AI on its own. The two-tier risk is not theoretical. It is in the proposal, named clearly.

If autonomous AI works, this is a real expansion of access. If it does not work, the harms land on the people who already have the fewest options.

6. The rules to hold anyone responsible do not exist yet

Imagine an autonomous AI gives a patient the wrong diagnosis tomorrow and the patient is hurt. Who is responsible?

The paper says the developer is primarily responsible. The hospital that deployed it is secondarily responsible. That sounds reasonable. But the legal system to actually hold those parties to account does not exist yet. There is no AI medical malpractice law in any state. There is no precedent. There is no clear way for a patient to sue.

When a self-driving car hurts someone today, the lawsuit takes years and goes through layers of liability that nobody planned for. We are about to do the same thing in medicine, only with patients who cannot wait years.

The licensing idea is good. The order is wrong. We should build the rules first — then issue the licenses.

Where this leaves us

Six reasons. Some technical, some human, some legal. Together they are why I think the world is not ready for autonomous AI doctors today, even though the case for them is real.

The next chapter is about what we should be doing instead.

References

Bergman, A., Wachter, R. M., Emanuel, E. J. (2026). A Licensure Framework for Autonomous Clinical AI. JAMA. doi:10.1001/jama.2026.5483

Bracic, A., Spector-Bagdady, K., Towle, S., Zhang, R., James, C. A., Price, W. N. II. (2026). Factors for Patient Trust and Acceptance of Medical Artificial Intelligence. JAMA Network Open, 9(3), e260815. doi:10.1001/jamanetworkopen.2026.0815

Chen, C., Cui, Z. (2025). Impact of AI-Assisted Diagnosis on American Patients' Trust in and Intention to Seek Help From Health Care Professionals. Journal of Medical Internet Research, 27, e66083. doi:10.2196/66083

Korom, R., Kiptinness, S., Adan, N., et al. (2025). AI-based clinical decision support for primary care: a real-world study. arXiv. doi:10.48550/arXiv.2507.16947

Qazi, I. A., Ali, A., Khawaja, A. U., et al. (2026). Large language model diagnostic assistance for physicians in a lower-middle-income country: a randomized controlled trial. Nature Health, 1(2), 198–205. doi:10.1038/s44360-025-00007-8

Wu, D., Haredasht, F. N., Maharaj, S. K., et al. (2025). First, do NOHARM: towards clinically safe large language models. arXiv. doi:10.48550/arXiv.2512.01241

Not yet is not the same as never. And it is definitely not the same as do nothing.

If autonomous AI is not the answer yet, but the doctor shortage is real, then the question becomes harder: what should we be doing right now? Here is where I land.

1. Use AI to free up the doctors we have

This is the idea from the first JAMA paper from earlier in the week (Martinelli et al., 2026). AI is at its strongest when it absorbs the work that pulled the doctor away from the patient in the first place — the paperwork, the notes, the inbox, the insurance forms.

Free up the doctors. Do not replace them. Same goal — more access to good care — different path. And the path with much better evidence behind it (Korom et al., 2025; Qazi et al., 2026).

2. Train every clinician to use AI well

Read the studies the JAMA authors cite carefully and one fact jumps out. The Pakistan trial showed huge gains for doctors trained to use AI. A US trial showed no gain when AI was dropped into clinics without training and workflow integration (Goh et al., 2024). Same tools. Different results.

Training is the leverage point. Right now, most doctors using AI in their practice are figuring it out alone. That is wasted gain. A national investment in teaching every clinician to use AI well would do more, faster, with less risk than any autonomous deployment.

3. Fix the workforce pipeline

Some of this is dull policy. More residency slots. Better paths for doctors trained in other countries. Less crushing administrative load on the ones we already have.

None of that is glamorous. None of it is AI-shaped. But the reason there are so few rural primary care doctors is not a lack of willing people. It is a system that grinds them down. AI can help with a piece of that. It cannot replace the work of fixing it.

4. Build the rules first, then issue the licenses

The licensing idea in the JAMA paper is genuinely good. I think it will be necessary one day. But the order they propose has the rules and the licenses arriving at the same time.

That is the wrong order.

First, build the rules. The malpractice law for AI. The clear path for a patient to be heard when something goes wrong. The disclosure standards so a person knows whether the entity in front of them is a human, an AI, or some mix. The independent monitoring of patient outcomes — not self-reported by the developer, but actually checked by an outside body.

Then, once those exist, issue the first licenses. Carefully. In small numbers. With patient outcomes tracked from day one.

5. Listen to what people actually want

This is the one most policy proposals miss. We have good evidence, in real surveys with thousands of people, about what patients want from medical AI. They want a clinician in the room. They want oversight from trusted bodies. They want to know when AI is being used (Bracic et al., 2026; Chen and Cui, 2025).

That is not a panel of experts speculating. That is patients answering directly. If the entire goal of expanding access is to take better care of people, the people themselves get a vote on how. So far, autonomous AI in primary care is not what they are voting for.

Where the line is

There is a sentence in the JAMA paper that I agree with completely:

As clinical AI increasingly resembles clinicians in its capabilities, our regulatory frameworks must evolve accordingly.

That is correct (Bergman et al., 2026). The frameworks have to evolve. I just do not think evolve yet means license to practice alone.

The careful version of progress here is something like this. Rush to make AI useful. Do not rush to make it autonomous. Use it to give the existing workforce more time and more reach. Build the rules and the trust before the licenses. And keep the human doctor in the room while we figure out — with evidence, not assumption — what AI alone can and cannot safely do.

If you are reading this and thinking about your own care, this article is policy commentary, not health advice. The right person to talk to about your medical decisions is your own doctor.

References

Bergman, A., Wachter, R. M., Emanuel, E. J. (2026). A Licensure Framework for Autonomous Clinical AI. JAMA. doi:10.1001/jama.2026.5483

Bracic, A., Spector-Bagdady, K., Towle, S., Zhang, R., James, C. A., Price, W. N. II. (2026). Factors for Patient Trust and Acceptance of Medical Artificial Intelligence. JAMA Network Open, 9(3), e260815. doi:10.1001/jamanetworkopen.2026.0815

Chen, C., Cui, Z. (2025). Impact of AI-Assisted Diagnosis on American Patients' Trust in and Intention to Seek Help From Health Care Professionals. Journal of Medical Internet Research, 27, e66083. doi:10.2196/66083

Goh, E., Gallo, R., Hom, J., et al. (2024). Large language model influence on diagnostic reasoning: a randomized clinical trial. JAMA Network Open, 7(10), e2440969. doi:10.1001/jamanetworkopen.2024.40969

Korom, R., Kiptinness, S., Adan, N., et al. (2025). AI-based clinical decision support for primary care: a real-world study. arXiv. doi:10.48550/arXiv.2507.16947

Martinelli, C., Carnevale, V., Ercoli, A., et al. (2026). Artificial Intelligence Is Not the End of the Physician. JAMA. doi:10.1001/jama.2026.4356

Qazi, I. A., Ali, A., Khawaja, A. U., et al. (2026). Large language model diagnostic assistance for physicians in a lower-middle-income country: a randomized controlled trial. Nature Health, 1(2), 198–205. doi:10.1038/s44360-025-00007-8