But these big, new chatbots can, as we know, also generate human-sounding responses to prompts and questions. And in a recent head-to-head test, that capability gave the bots a surprising superiority in what ought to be one of the most intrinsically human of all activities: serving as a doctor.
To conduct the test, a team of researchers from the University of California in San Diego lurked on r/AskDocs, a Reddit forum where registered, verified healthcare professionals answer people's medical questions. The researchers selected nearly 200 representative questions on the forum, from the silly-sounding ("Swallowed a toothpick, friend said I'm going to die") to the terrifying ("miscarriage one day after normal ultrasound?"). They then fed the questions into the virtual maw of the bot ChatGPT, and had a separate group of healthcare experts conduct a blind evaluation of answers from both AI and MDs.