While ChatGPT 3.5 and 4.0 can serve as supplementary sources for health information, a recent study underscores the need for validation and scrutiny of AI-generated content. 

By First Check Team

As health misinformation continues to spread across various online platforms, the reliability of sources becomes paramount. An interesting study sought to evaluate the potential of artificial intelligence (AI) language models, specifically ChatGPT 3.5 and 4.0, in addressing this need and compared their performance against authoritative sources like the World Health Organization (WHO).

The study, which utilised 487 COVID-19 questions from the WHO website and involved assessment by medical experts, found that both ChatGPT 3.5 and 4.0 demonstrated moderate to good performance in generating responses to COVID-19 questions. However, ChatGPT 4.0 emerged slightly ahead, excelling in providing accurate, comprehensive, relevant, and clear answers compared to its predecessor. 

Despite the promising performance, the study also identified several limitations inherent in AI language models. While generally accurate, they cannot fully replace authoritative sources like the WHO due to the potential for gaps and misinformation.

The study underscores the need for continued validation and scrutiny of AI-generated content. These models lack the capacity to provide personalised medical advice or treatment recommendations, a task best left to healthcare professionals.

One of the key takeaways from the study is the importance of cross-referencing AI-generated information with authoritative sources. While ChatGPT 3.5 and 4.0 can serve as valuable supplementary sources for health information, human-authored content remains indispensable for its depth and nuance. Leveraging technology, alongside human expertise, will be crucial in combating health misinformation, and ensuring that public health communication is accurate and reliable.

Read More : ChatGPT gives incorrect diagnoses, but holds promise