A recent study published in Nature Medicine reveals that AI tools designed to provide medical symptom advice do not outperform conventional internet search engines in helping patients make healthcare decisions. The research tested AI chatbots against standard online searches and found AI’s accuracy in identifying medical conditions was roughly comparable to non-AI methods. Users relying on AI reached the correct diagnosis about one-third of the time, with only 45% choosing the appropriate medical response.
Despite AI systems scoring highly in medical licensing exams, this study highlights significant gaps in their reliability when applied to real-world medical self-diagnosis. Furthermore, both experts and users exhibit reduced trust in medical advice when it is known to be AI-generated, associating it with lower reliability and empathy compared to human-provided guidance. The study raises important concerns about the risks of depending on AI chatbots for healthcare without professional oversight, as inaccuracies and inconsistencies in AI-delivered advice could pose public health hazards.
This outcome underscores the complexities involved in developing AI technologies that can responsibly and effectively support patients in sensitive and high-stakes areas such as health. While AI continues to advance, its current iteration should not replace traditional medical consultation or authoritative resources. Instead, AI might best serve as a complementary tool, integrated cautiously within broader healthcare delivery systems.
