Logo

American Heart Association

  2
  0


Final ID: Su3122

Simplifying Cardiology Research Abstracts: Assessing ChatGPT's Readability and Comprehensibility for Non-Medical Audiences

Abstract Body (Do not enter title and authors here): Background:
Artificial Intelligence (AI)-powered chatbots like ChatGPT are increasingly used in academic medical settings to help with tasks such as evidence synthesis and manuscript drafting. They have shown potential in simplifying complex medical texts for non-medical audiences like patients and journalists. However, less is known about whether simplified texts may exclude important information or be of interest to patients or other non-medically-trained people such as journalists.

Objective:
This study aims to assess ChatGPT's capacity to simplify cardiology research abstracts by eliminating jargon and enhancing universal comprehension.

Methods:
We analyzed all abstracts and scientific statements published from July to November 2023 in Circulation (n=113). These abstracts were processed through ChatGPT with the prompt: "Please rewrite the following text to be comprehensible at a 5th-grade reading level. Retain all original information and exclude nothing". We assessed the readability of both original and simplified texts using Flesch-Kincaid Grade Level (FKGL) and Reading Ease (FKRE) scores. Additionally, a panel of five physicians and five laypeople evaluated these texts for completeness, accuracy, and understandability.

Results:
ChatGPT transformation or abstracts reduced the required reading level from a college graduate to 8-9th grade by both FKGL (18.3 to 8.6; p<.001) and FKRE (14.6 to 68.5; p<.001). The physician panel generally found that the simplified abstracts retained all essential information without introducing errors. Both physicians and laypeople found the simplified texts more comprehensible. Laypeople appreciated the ease of understanding but did not strongly express a desire for more such simplified texts, and some still worried about the simplified abstracts missing important details.

Conclusion:
ChatGPT simplifies medical research abstracts to a level understandable by those with less than a high school education without excluding important information. This approach may be useful to communicate new research findings to patients and other non-medically-trained people.
  • Malkani, Kabir  ( NewYork-Presbyterian Weill Cornell , New York , New York , United States )
  • Kini, Vinay  ( NewYork-Presbyterian Weill Cornell , New York , New York , United States )
  • Zhang, Ruina  ( NewYork-Presbyterian Weill Cornell , New York , New York , United States )
  • Falk, Zachary  ( NewYork-Presbyterian Weill Cornell , New York , New York , United States )
  • Tawde, Prianca  ( NewYork-Presbyterian Weill Cornell , New York , New York , United States )
  • Hughes, Ryan  ( NewYork-Presbyterian Weill Cornell , New York , New York , United States )
  • Parker, Melissa  ( NewYork-Presbyterian Weill Cornell , New York , New York , United States )
  • Collins, Griffin  ( NewYork-Presbyterian Weill Cornell , New York , New York , United States )
  • Maizes, Danielle  ( NewYork-Presbyterian Weill Cornell , New York , New York , United States )
  • Zhao, Alexander  ( NewYork-Presbyterian Weill Cornell , New York , New York , United States )
  • Author Disclosures:
    Kabir Malkani: DO NOT have relevant financial relationships | Vinay Kini: DO NOT have relevant financial relationships | Ruina Zhang: DO NOT have relevant financial relationships | Zachary Falk: DO NOT have relevant financial relationships | Prianca Tawde: DO NOT have relevant financial relationships | Ryan Hughes: DO NOT have relevant financial relationships | Melissa Parker: DO NOT have relevant financial relationships | Griffin Collins: No Answer | Danielle Maizes: DO NOT have relevant financial relationships | Alexander Zhao: DO NOT have relevant financial relationships
Meeting Info:

Scientific Sessions 2024

2024

Chicago, Illinois

Session Info:

LLMs Friend or Foe?

Sunday, 11/17/2024 , 03:15PM - 04:15PM

Abstract Poster Session

More abstracts on this topic:
Digital Health Intervention to Enhance Optimization of Heart Failure Care: From Reciprocal Innovation using Human-Centered Design to Pilot Study

Polachini Beatriz, Beaton Andrea, Longenecker Chris, Ribeiro Antonio Luiz, Brant Luisa, Ciminelli Ana Luiza, Lino Rafael, Bramucci Victoria, Ribeiro Edmar, Rezende Lilian, Spaulding Erin, Sandhu Alexander, Azizi Zahra

A Chemical Language Model for the Design of De Novo Molecules Targeting the Inhibition of TLR3

Velasco Juan

More abstracts from these authors:
You have to be authorized to contact abstract author. Please, Login
Not Available