American Medical Association Shares Framework to Address the Escalating Risk of Physician Deepfakes

Shalini Kathuria Narang, JMIR Correspondent

As AI becomes more sophisticated, new risks are emerging. In this News and Perspectives article, JMIR Correspondent Shalini Kathuria Narang reports on the American Medical Association’s response to the rising issue of physician deepfakes.

Key Takeaways:

Deepfakes created to impersonate doctors, manipulate the public, and endorse unproven treatments pose threats to patients and the broader health care system.
The American Medical Association has released a new policy framework to establish clear, enforceable protections for physicians against unauthorized AI-generated deepfakes.
Some states’ legislatures in the United States have passed laws to discipline vendors where an AI chatbot is engaged in the practice of medicine or is impersonating a human engaged in the practice of medicine.

A deepfake is a video, photo, or audio recording that seems real but has been manipulated with AI. The underlying technology can replace faces, manipulate facial expressions, and synthesize speech, depicting humans appearing to say or do something that they have never said or done.

AI-generated deepfake “doctors” impersonating physicians garner millions of views on social media, misleading the public and endorsing unproven treatments. The content typically promotes products for their creators’ financial gain while exposing the viewers to scams that erode the patient-physician relationship, undermine confidence in evidence-based care, and put the public at risk of harm.

According to the factchecking organization Full Fact, hundreds of AI-generated deepfakes of real doctors and academics to promote health products with bogus endorsements are damaging the reputations of those they impersonate and could lead to people making decisions about their health based on fake claims.

“[The] mental health space is an area of big concern, including significant privacy concerns, more with some of the general purpose chatbots claiming to help support mental health. We know many, many, many individuals and patients are using these chatbots for support that can cross the line and be dangerous, as we’ve seen suicides and bad outcomes related to the use of these chatbots in certain ways,” says Shannon Curtis, JD, Senior Director of Policy Development for the American Medical Association (AMA) Center for Digital Health and AI.

She adds that the AMA is looking for solutions to begin addressing the issue, including transparency to help users understand that they’re not actually engaging with a licensed clinician.

“We want to see a prohibition on chatbots from claiming they’re a licensed professional, or that they’re providing equivalent services as a licensed professional,” says Curtis. “We have called for more action to update our regulatory structure... A call for new, updated, appropriate regulatory oversight structures that can help provide that clarity and consistency for consumers, for physicians, for the industry, about what needs to be regulated, and what does not. And hopefully, turning that into a more appropriate, risk based regulatory system that fits where AI is, and hopefully fits where it is going.”

As AI chatbots become more sophisticated, and more harms are realized, it is likely more states will create regulations and exercise enforcements.

Responding to the escalating risk of AI deepfakes, and recognizing that existing privacy, employment, and intellectual property laws do not adequately address AI deepfake risks in clinical contexts, the AMA Center for Digital Health and AI has created a policy framework to modernize physician identity protections and close legal gaps to uphold patient safety, professional integrity, and public trust.

The framework is built on 7 policy principles:

Physician identity as a protected right: A physician’s name, image, likeness, voice, and digital replicas are protected interests and should not be used beyond the scope of the clinician’s consent.
Prohibition on deceptive medical impersonation: AI-generated or altered content falsely conveying a physician’s endorsement, authorship, or medical judgment likely to mislead in a health-related decision must be prohibited.
Informed, opt-in, and revocable consent: Use of a physician’s identity in AI-generated or manipulated content requires affirmative, informed, opt-in consent.
Mandatory transparency and labeling: AI-generated or altered content depicting a physician must be clearly labeled. Patients interacting with an AI-generated health professional must be alerted before the interaction.
Shared responsibility to prevent impersonation: All participants—platforms, hospitals, health systems, and AI vendors—share responsibility for preventing identity misuse.
Enforcement and practical remedies: Processes must be in place to allow physicians to document, escalate, and address identity misuse. Institutions and platforms must facilitate investigation and escalation.
Minimizing administrative burden: Protecting physician identity should be a default that doesn’t place any undue burden on the physician.

“AI deepfakes that impersonate physicians are not just scams—they are a public health and safety crisis,” said AMA CEO John Whyte via a statement. “When bad actors exploit a doctor’s identity, they undermine patient trust and can steer people toward harmful, unproven care. We need strong action by federal and state lawmakers to protect physicians’ identities, ensure transparency, and stop this fraud. Safeguarding professional integrity is essential to preserving trust and delivering high-quality care in a rapidly evolving digital landscape.”

The new AMA framework aims to guide how the organization works with government officials and industry partners to stop AI-generated deepfakes of physicians. The AMA is eager to collaborate with lawmakers, regulators, and industry to protect patients and doctors from these risks.

“We were seeing a lot of physicians having their names stolen. It’s turning up on things like academic papers, that they had no part of. Our framework seeks to address those issues by creating a prohibition and requiring affirmative, opt-in consent from the physician in question if their name, image likeness, voice, etc, is going to be used going forward,” says Curtis.

While the framework currently functions as a set of guidelines, Curtis notes that the AMA intends for it to lead to enforceable standards.

“Via the policy framework, we are seeking to create statutory or legal obligations that would prohibit a third party from creating a digital replica or a deepfake that utilizes a person’s name, image, likeness, or voice in a nonconsensual way,” she says.

She emphasizes the importance of enforcement mechanisms like statutory or regulatory requirements on the platforms that end up hosting this content and enforcement by a government agency like the Federal Trade Commission for those bad actors that are creating the deepfakes and disseminating them.

“Data privacy is a huge issue that is sometimes not well understood by individuals. We’d like to see some more action to help protect patients’ personal health information,” says Curtis. The AMA is collaborating with other groups to support the notice-and-removal obligations in the Take It Down Act and a growing coalition in support of the enactment of the NO FAKES Act.

Regulating medical practice typically falls under the jurisdiction of state medical boards in the United States. In addition to a recent landmark lawsuit filed by the Governor of Pennsylvania against Character.AI, some states’ legislatures have passed laws—for example, AB 489 in California, HB 2748 in Oregon, and the Conversational AI Safety Act in Nebraska—to discipline vendors where an AI chatbot is engaged in the practice of medicine or is impersonating a human medical professional.

‎

Journal of Medical Internet Research - American Medical Association Shares Framework to Address the Escalating Risk of Physician Deepfakes