Our new publication asking whether ChatGPT chatbots can reliably give advice to help people quit smoking? We examined chatbots including WHO's Sarah in providing assistance on quitting smoking. World Health Organization David Broniatowski Christina Wysota
Key points:
- WHO’s chatbot Sarah outperformed others, following guidelines 72% of the time. -Some responses contained misinformation (22%), including bizarre suggestions like endorsing using a necklace to quit smoking.
- Queries about evidence-based quitting methods (e.g., medications) scored higher than less conventional ones (e.g., hypnosis, vapes).
- Despite these gaps, chatbots resisted adversarial attacks and consistently recommended professional counseling. Why does this matter?
As AI plays a growing role in healthcare, ensuring that chatbots provide accurate, evidence-based guidance is crucial. Our study highlights both the potential and the risks of AI-driven health interventions.