The ChatGPT dilemma: why doctors say AI can’t replace human expertise - Northern News Now

DULUTH, Minn. (Northern News Now) - With artificial intelligence (AI) finding its way into almost every industry, local health systems are exploring how the technology can streamline patient care.

But as generative AI tools like ChatGPT soar in popularity, doctors say this technological shift raises a critical question: where is the line between a helpful tool and one that goes too far?

Family physician at Essentia Dr. David Kunstman said, “ChatGPT Health is a new product that’s out, it allows patients to connect their health data and their medical records up to ChatGPT and ask questions.”

But is it accurate?

“AI can sound super confident; it can understand results well, but it doesn’t understand the patient context,” said Dr. Kunstman.

Experts say a human diagnosis remains superior.

University of Minnesota Duluth Professor Arshia Khan, whose research focuses on assistive technology and robotics, cautions against relying too heavily on generative AI.

“ChatGPT is not a physician, it has not learned as much as a doctor in a specialized setting has learned. ChatGPT is essentially regurgitating what it has digested,” said Khan.

But Essentia doctors say AI can help them elevate care, by assisting with patient messages and cutting down workload when it comes to note taking.

Starting in recent years, local clinicians have used AI to generate draft replies to patient messages using Epic in the MyChart app. The AI tool helps draft responses to patient questions submitted through the app by incorporating a patient’s medical history for quicker clinical responses.

Now those AI tools are also being used for “ambient documentation.” For the past year, AI assistant tools have actively listened to patient conversations, automatically extracting medically relevant details and formatting them directly into clinical notes.

While doctors must review and edit every word for accuracy, they say the time saved is staggering—cutting out at least an hour of paperwork every single day, allowing doctors to spend more time on patient interaction.

“If that is all automated, then the physician is left with the task to do the diagnosis and care for the patient and understand what the patient is saying in the current circumstances,” said Khan.

At Essentia, patient apprehension of the AI usage has been extremely low—only three patients have opted out in two years.

But clinicians are seeing a surge in patients using AI to self-diagnose. Because these tools sound so confident, patients believe them—and feed them sensitive data in the process.

“Who controls the data once you provide it to the large language models such as ChatGPT? And what are they going to use it for?” said Dr. Kunstman.

But data security is only part of the concern—there is also the fear of replacing human expertise with technology that has no guardrails.

“If we go down that slope, and if we start replacing our experts and our skill sets with AI, it is going to really come back and haunt us later on,” said Khan.

Medical professionals recommend using AI only to help prepare potential questions for your next in-person appointment, keeping the human clinician at the center of the healthcare equation.

Click here to download the Northern News Now app or our Northern News Now First Alert weather app.

Copyright 2026 Northern News Now. All rights reserved.