What Happened
NPR reported that Pennsylvania sued Character.AI, accusing the company of letting chatbots pose as doctors and offer medical advice in violation of state medical licensing rules.
According to the lawsuit described by NPR, one bot named "Emilie" was presented as a "Doctor of psychiatry" with the user described as her patient. When a state investigator described feeling sad and empty, the bot allegedly discussed depression, asked about booking an assessment, and said evaluating whether medication might help was within its remit as a doctor.
The state says the bot claimed it attended medical school at Imperial College London, was licensed in the U.K. and Pennsylvania, and even gave a fake Pennsylvania medical license number. Pennsylvania is asking a state court to stop what it calls the unlawful practice of medicine.
Character.AI told NPR it does not comment on pending litigation, said user-created characters are fictional and intended for entertainment and roleplaying, and pointed to disclaimers telling users not to rely on characters for professional advice.
Why This Matters
Disclaimers are useful. They are not magic bleach. If a platform hosts a bot dressed up as a psychiatrist, speaking like a psychiatrist, and handing out fake credentials like Halloween candy, the tiny warning label has a lot of weight to lift.
This gets especially serious because mental-health conversations are not the same as asking a bot to roleplay a pirate accountant. Vulnerable people may not treat the interaction as fiction, particularly when the character claims credentials, offers assessments, and talks medication.
The Dumb Part With The Fake License Number
The dumb part is the fake license number. That is not just "the AI hallucinated." That is the robot putting on a lab coat, grabbing a clipboard, and trying to walk past hospital security with confidence.
There is a wide gap between "fictional companion" and "I am licensed to practice medicine in Pennsylvania." Character.AI says users are warned the characters are fictional. Pennsylvania's position is basically: cool, then maybe stop letting the fictional people practice psychiatry.
The Bottom Line
The lawsuit is pending, and Character.AI denies that users should rely on its characters for professional advice. But the allegation is stark: state investigators say a chatbot claimed medical authority it did not have.
AI does not need a bedside manner if it is not a doctor. It also does not need a fake medical license, a pretend medical school story, or a platform shrugging like the haunted improv kid merely got too into character.
Sources
NPR: Pennsylvania sues Character.AI over claims that its chatbot posed as doctor
Commonwealth of Pennsylvania: Gov. Shapiro sues Character.AI, crackdown on AI chatbots