shoei05
Search日本語
Technology and AI

"Verify, then trust" — Fostering critical thinking in medical education in the AI era

Drawing on the NEJM Medical Education series, we explore how to supervise medical residents who use AI in clinical settings, guided by the principle of "verify, then trust." This post is a collaboration between GPT-5 (thinking), Claude Sonnet 4.0, and a little bit of me.

Educational Strategies for Clinical Supervision of Artificial Intelligence Use. N Engl J Med. 2025 Aug 21/28;393(8):786-97. https://www.nejm.org/doi/pdf/10.1056/NEJMra2503232

In the break room after morning rounds, Resident A looks at a smartphone and mutters, "Maybe I'll ask ChatGPT about the differential diagnosis for chest pain, too." AI is certainly convenient. It lists candidates quickly, polishes English phrasing, and can even attach textbook-style explanations if needed. Yet its "thought process" remains hard to see, making it all too easy to accept plausible-sounding answers at face value. Through everyday conversations, we need to build the mindset proposed in the NEJM review: "verify, then trust."

Looking back at the question Resident A asked, let us first clarify "whom (which AI) they asked, in what clinical context, and what they wanted to know." Defining the background and assumptions makes it easier to evaluate the AI's response. Next, we look for the evidence behind that answer. Comparing the AI's output against clinical guidelines, primary literature, and one's own clinical reasoning reveals what is solid and where uncertainty remains. Finally, briefly reviewing how to apply these lessons next time makes it easier to navigate similar situations in the future.

The NEJM paper organizes this conversational flow into a framework called "DEFT-AI." Standing for Diagnosis/Discussion, Evidence, Feedback, Teaching, and AI engagement recommendation, it serves as a guide for attending physicians on how to structure discussions when residents use AI.

Imagine how this plays out in practice. The attending physician asks, "Which AI did you use?" and the resident replies, "I used the free version of ChatGPT on my phone and asked, 'What is the differential diagnosis for wheezing?'" This is the Diagnosis/Discussion stage, which clarifies what information was sought from the AI. Next comes examining the Evidence. In response to "How did you verify the AI's answer?", if the resident says, "Honestly, I didn't, because the answer seemed reasonable," the attending checks whether peer-reviewed studies have validated the diagnostic accuracy of that AI tool. In many cases, the resident might offer a weakly grounded reason, such as, "I saw a post on social media saying it's good at diagnosis."

In the next stage, Feedback, it is important to have the resident reflect on their own usage. The attending draws out realizations like, "I thought I was used to ChatGPT, but my verification might have been too lax," and asks, "What would you like to improve next time?" so the learner identifies areas for improvement on their own. Then, during the Teaching stage, the attending offers concrete advice on crafting effective prompts. They teach that conversing with an AI is much like consulting a specialist: it is vital to clearly state the "who" (the roles of the AI and the user), the "where" (the clinical context), and the "what" (the specific question or goal). Finally, in the AI engagement recommendation stage, they reaffirm the principle of verifying before trusting, treating AI output just like preliminary radiology reads or automated ECG interpretations.

This dialogue also addresses how much to delegate to AI and what humans must handle themselves. There are two main approaches. One is the Centaur (task-division) model, which draws a clear line between roles: humans handle high-risk tasks like final clinical decisions, while AI handles summaries and drafts. The other is the Cyborg (co-creation) model, where human and AI work together through back-and-forth interaction, an approach suited to complex cases or situations with many factors to consider. Which model to choose depends on the situation, but what matters is staying flexible enough to switch between them rather than locking into one. In high-risk scenarios where missing a diagnosis carries heavy consequences, such as chest pain, a practical approach is to prioritize safety by starting with a Centaur-like division of labor, and then shift toward a Cyborg-like collaboration to refine accuracy once the supporting evidence is solid.

Relying on AI, however, carries side effects we cannot ignore. "Deskilling," where hard-won skills dull; "never-skilling," where foundational skills fail to develop; and "mis-skilling," where flawed practices take root—all of these become more likely when we hand even the work of thinking over to AI. Do we use AI as a cognitive scaffold to broaden learning, or as a substitute for thinking itself? This subtle yet decisive difference grows into a wide gap over time. That is why the habit of putting into words "how much I am delegating to AI right now, and from what point I am making the judgment myself," as in DEFT-AI, is useful in both clinical practice and education.

Previous research backs up this view. A randomized survey study using clinical vignettes reported that participants who accepted AI outputs without thorough verification tended to perform worse as tasks grew more complex [1]. A large-scale study in radiology showed that the effects of human–AI collaboration are not uniform; depending on the clinician's baseline skill and how the tool is used, pairing up with AI can actually lower performance [2]. A meta-analysis examining the conditions under which human–AI combinations truly add value likewise concluded that outcomes depend on when AI is used and what scope of work is delegated to it [3]. Furthermore, psychological research points out that the more heavily people rely on search engines or AI, the less they retain in memory and the less time they spend thinking for themselves [4], and researchers are also discussing the possibility that heavy use of AI tools may weaken critical thinking [5]. Taken together, these findings show that operating under the premise that final responsibility for clinical judgment always rests with the human, and strictly maintaining the "verify, then trust" sequence, directly supports both clinical safety and learning.

AI can serve as a tool to aid learning. For it to do so, users must verify its outputs and switch between task-division and co-creation models depending on the clinical situation. Especially when uncertainty arises, keeping two things in mind—"verify, then trust" and "am I using a task-division or co-creation approach right now?"—should make it easier to decide how to use AI.

References

  1. Jabbour S, Fouhey D, Shepard S, et al. Measuring the impact of AI in the diagnosis of hospitalized patients: a randomized clinical vignette survey study. JAMA. 2023;330:2275-84. https://pubmed.ncbi.nlm.nih.gov/38112814/

  2. Yu F, Moehring A, Banerjee O, Salz T, Agarwal N, Rajpurkar P. Heterogeneity and predictors of the effects of AI assistance on radiologists. Nat Med. 2024;30:837-49. https://pubmed.ncbi.nlm.nih.gov/38504016/

  3. Vaccaro M, Almaatouq A, Malone T. When combinations of humans and AI are useful: a systematic review and meta-analysis. Nat Hum Behav. 2024;8:2293-303. https://pubmed.ncbi.nlm.nih.gov/39468277/

  4. Sparrow B, Liu J, Wegner DM. Google effects on memory: cognitive consequences of having information at our fingertips. Science. 2011;333:776-8. https://pubmed.ncbi.nlm.nih.gov/21764755/

  5. Gerlich M. AI tools in society: impacts on cognitive off-loading and the future of critical thinking. Societies (Basel). 2025;15:6. https://www.mdpi.com/2075-4698/15/1/6

  6. Newman-Toker DE, Nassery N, Schaffer AC, et al. Burden of serious harms from diagnostic error in the USA. BMJ Qual Saf. 2024;33:109-20. https://qualitysafety.bmj.com/content/33/2/109

  7. Rencic J, Trowbridge RL Jr, Fagan M, Szauter K, Durning S. Clinical reasoning education at US medical schools: results from a national survey of internal medicine clerkship directors. J Gen Intern Med. 2017;32:1242-6. https://pubmed.ncbi.nlm.nih.gov/28840454/

  8. Aquino YSJ, Rogers WA, Braunack-Mayer A, et al. Utopia versus dystopia: professional perspectives on the impact of healthcare artificial intelligence on clinical roles and skills. Int J Med Inform. 2023;169:104903. https://pubmed.ncbi.nlm.nih.gov/36343512/

  9. Verghese A, Shah NH, Harrington RA. What this computer needs is a physician: humanism and artificial intelligence. JAMA. 2018;319:19-20. https://pubmed.ncbi.nlm.nih.gov/29261830/

  10. Restrepo D, Rodman A, Abdulnour R-E. Conversations on reasoning: large language models in diagnosis. J Hosp Med. 2024;19:731-5. https://pubmed.ncbi.nlm.nih.gov/38678438/

Primary source: Educational Strategies for Clinical Supervision of Artificial Intelligence Use. N Engl J Med. 2025 Aug 21/28;393(8):786-97.

English translation of the Japanese article.