In a groundbreaking study conducted by the Italian Society of Neurology, an AI chatbot named Chat GPT has outperformed human neurologists in providing responses to multiple sclerosis patients. The study involved blind testing, where patients were not aware that they were interacting with both human doctors and an AI bot. Patients were asked to express their preference between two alternative answers—one from a human neurologist and the other from Chat GPT—regarding various aspects of their disease, including technical, pharmacological, and scientific questions. Interestingly, the preference for Chat GPT varied depending on the educational background of the patient; those with higher education levels favored the more detailed answers provided by the neurologists. However, the AI bot seemed to establish a sufficiently “empathetic” rapport with its users, reassuring them on a conversational level.
The study, published in the Journal of Neurology and conducted in collaboration with 20 different hospitals and universities in Italy, involved renowned clinical centers such as the Vanvitelli University, Federico II University, and the Department of Hygiene and Public Health. Dr. Luigi Lavorgna, a neurologist at Vanvitelli University, coordinated the study, highlighting the significance of having millennial researchers, such as Elisabetta Maida and Marcello Moccia, who are digital natives, specializing in Neurology and Neurological Research, respectively. Dr. Lavorgna emphasized the researchers' main objective: “We wanted to evaluate how much and how effectively Chat GPT could assist patients in addressing doubts, fears, and uncertainties related to their health in a chronic disease like multiple sclerosis, which, by definition, does not have a cure but rather experiences relapses, remissions, and requires constant medical presence for therapy adherence.”
The choice to study multiple sclerosis patients was strategic, as they are inherently inclined to use digital tools to manage their condition due to their young age of onset and the chronic nature of the disease. The researchers presented patients with various frequently asked questions directed to doctors and provided two responses—one from Chat GPT and the other from expert neurologists. The patients were unaware of which response was generated by the neurologists and which one was from the AI bot, ensuring a blind study. Dr. Lavorgna clarified that “it should be noted that Chat GPT formulates scientifically correct responses.”
The results of the study revealed that in 51% of cases, patients preferred the responses from Chat GPT, perceiving them as “more empathetic” compared to those from human neurologists. Additionally, patients found the language used by Chat GPT to be “simpler and more understandable” than that used by the neurologists, highlighting a communication weakness between doctors and patients. The simple, immediate, and informal language of Chat GPT appealed to the majority of non-educated individuals.
Dr. Lavorgna concluded by emphasizing the need for neurologists to be more attentive to the language they use, tailoring its complexity based on the cultural level of the patient. Simultaneously, he underscored the importance of Chat GPT increasing its level of complexity to cater to users with higher cultural backgrounds, adapting both language and semantics accordingly.
This study marks a significant milestone in the field of AI-enabled healthcare. As AI technologies continue to advance and adapt to human needs, the potential for sophisticated AI chatbots like Chat GPT to support and enhance patient-doctor interactions is promising. This empathetic AI could be an invaluable aid for patients seeking immediate information, support, and reassurance regarding their conditions, ultimately improving the overall healthcare experience.
Use the share button below if you liked it.