Logo

American Heart Association

  113
  0


Final ID: Su3122

Simplifying Cardiology Research Abstracts: Assessing ChatGPT's Readability and Comprehensibility for Non-Medical Audiences

Abstract Body (Do not enter title and authors here): Background:
Artificial Intelligence (AI)-powered chatbots like ChatGPT are increasingly used in academic medical settings to help with tasks such as evidence synthesis and manuscript drafting. They have shown potential in simplifying complex medical texts for non-medical audiences like patients and journalists. However, less is known about whether simplified texts may exclude important information or be of interest to patients or other non-medically-trained people such as journalists.

Objective:
This study aims to assess ChatGPT's capacity to simplify cardiology research abstracts by eliminating jargon and enhancing universal comprehension.

Methods:
We analyzed all abstracts and scientific statements published from July to November 2023 in Circulation (n=113). These abstracts were processed through ChatGPT with the prompt: "Please rewrite the following text to be comprehensible at a 5th-grade reading level. Retain all original information and exclude nothing". We assessed the readability of both original and simplified texts using Flesch-Kincaid Grade Level (FKGL) and Reading Ease (FKRE) scores. Additionally, a panel of five physicians and five laypeople evaluated these texts for completeness, accuracy, and understandability.

Results:
ChatGPT transformation or abstracts reduced the required reading level from a college graduate to 8-9th grade by both FKGL (18.3 to 8.6; p<.001) and FKRE (14.6 to 68.5; p<.001). The physician panel generally found that the simplified abstracts retained all essential information without introducing errors. Both physicians and laypeople found the simplified texts more comprehensible. Laypeople appreciated the ease of understanding but did not strongly express a desire for more such simplified texts, and some still worried about the simplified abstracts missing important details.

Conclusion:
ChatGPT simplifies medical research abstracts to a level understandable by those with less than a high school education without excluding important information. This approach may be useful to communicate new research findings to patients and other non-medically-trained people.
  • Malkani, Kabir  ( NewYork-Presbyterian Weill Cornell , New York , New York , United States )
  • Kini, Vinay  ( NewYork-Presbyterian Weill Cornell , New York , New York , United States )
  • Zhang, Ruina  ( NewYork-Presbyterian Weill Cornell , New York , New York , United States )
  • Falk, Zachary  ( NewYork-Presbyterian Weill Cornell , New York , New York , United States )
  • Tawde, Prianca  ( NewYork-Presbyterian Weill Cornell , New York , New York , United States )
  • Hughes, Ryan  ( NewYork-Presbyterian Weill Cornell , New York , New York , United States )
  • Parker, Melissa  ( NewYork-Presbyterian Weill Cornell , New York , New York , United States )
  • Collins, Griffin  ( NewYork-Presbyterian Weill Cornell , New York , New York , United States )
  • Maizes, Danielle  ( NewYork-Presbyterian Weill Cornell , New York , New York , United States )
  • Zhao, Alexander  ( NewYork-Presbyterian Weill Cornell , New York , New York , United States )
  • Author Disclosures:
    Kabir Malkani: DO NOT have relevant financial relationships | Vinay Kini: DO NOT have relevant financial relationships | Ruina Zhang: DO NOT have relevant financial relationships | Zachary Falk: DO NOT have relevant financial relationships | Prianca Tawde: DO NOT have relevant financial relationships | Ryan Hughes: DO NOT have relevant financial relationships | Melissa Parker: DO NOT have relevant financial relationships | Griffin Collins: No Answer | Danielle Maizes: DO NOT have relevant financial relationships | Alexander Zhao: DO NOT have relevant financial relationships
Meeting Info:

Scientific Sessions 2024

2024

Chicago, Illinois

Session Info:

LLMs Friend or Foe?

Sunday, 11/17/2024 , 03:15PM - 04:15PM

Abstract Poster Session

More abstracts on this topic:
More abstracts from these authors:
Pulsed Field Ablation for the Treatment of Octogenarians with Atrial Fibrillation

Miyama Hiroshi, Liu Christopher, Ip James, Goljo Elden, Lerman Bruce, Cheung Jim, Lam Matthew, Mccoy Sean, Parker Melissa, Maizes Danielle, Collins Griffin, Perera Ruwanthi, Markowitz Steven, Thomas George

Assessing Racial Disparities in Heart Transplant Allocations Post-2018 Policy Change

Malkani Kabir, Zhang Ruina, Li Han, Ezema Ashley, Steitieh Diala, Purkayastha Subhanik, Kini Vinay

You have to be authorized to contact abstract author. Please, Login
Not Available