Logo

American Heart Association

  2
  0


Final ID: Sa4113

Large Language Models Improve Readability in Primary Responses to Coronary Artery Bypass Graft Questions for Patient Education

Abstract Body (Do not enter title and authors here): Introduction:
The advent of Large Language Models (LLMs) such as ChatGPT and Google Gemini yields a new paradigm for patient education. Coronary artery bypass grafts (CABGs) are open-heart surgeries that naturally gives to many questions in potential patients. Prior research suggested that patient education material should be written at a 6th grade reading level maximally to be comprehended by the majority of the US adult population. However, most education material is written at a much higher literacy level. Now with the increasing use of LLMs, the potential for informing patients at an appropriate readability level exists.

Research Question:
Can ChatGPT and Google Gemini improve the readability of their primary responses to patient education questions about Coronary Artery Bypass Grafts to a 6th grade reading level?

Methods:
We obtained 80 questions from 4 independent participants regarding information they would want to know prior to undergoing a CABG procedure. Questions that could not be answered using public information, required clinician input, and repeats were filtered out leaving 20 questions. These questions were asked to ChatGPT 3.5 and Google Gemini. Then both LLMs were asked to answer the question at a 6th grade level within the same conversation. A new conversation was created each of the 20 questions. All responses were then evaluated using a Flesch-Kincaid calculator where the Flesch-Kincaid reading ease score (FKRE) and the Flesch-Kincaid grade level (FKGL) were obtained. Data analysis was performed using Microsoft Excel and Graphpad Prism.

Results:
For the ChatGPT responses, the average FKRE was 31.865 and FKGL was 12.545. Following the 6th grade level request for ChatGPT, the average FKRE was 74.57 (p < 0.0001) and FKGL was 5.93 (p < 0.0001). For the Google Gemini responses, the average FKRE was 46.395 and FKGL was 10.545. Following the 6th grade level request for Google Gemini, the average FKRE was 72.945 (p < 0.0001) and FKGL was 6.76 (p < 0.0001).

Conclusion:
Both LLMs significantly increased the readability of their responses and did so to a 6th grade level. However, ChatGPT was closer than Google Gemini in reaching the desired FKGL. Nonetheless, our work showed that both LLMs could successfully work with CABG information and yield responses at a readability level befitting the US population. Future studies need to assess the accuracy of this information for LLMs to have practical use in cardiac surgery patient education.
  • Paruchuri, Venkata  ( UQ-Ochsner Clinical School , New Orleans , Louisiana , United States )
  • Patel, Dhaval  ( UQ-Ochsner Clinical School , New Orleans , Louisiana , United States )
  • Fang, Alexander  ( UQ-Ochsner Clinical School , New Orleans , Louisiana , United States )
  • Levitch, Ethan  ( UQ-Ochsner Clinical School , New Orleans , Louisiana , United States )
  • Kiely, Conor  ( UQ-Ochsner Clinical School , New Orleans , Louisiana , United States )
  • Kaur, Karmveer  ( UQ-Ochsner Clinical School , New Orleans , Louisiana , United States )
  • Sikder, Amit  ( University of Queensland , Brisbane , Queensland , Australia )
  • Author Disclosures:
    Venkata Paruchuri: DO NOT have relevant financial relationships | Dhaval Patel: DO NOT have relevant financial relationships | Alexander Fang: No Answer | Ethan Levitch: DO NOT have relevant financial relationships | Conor Kiely: DO NOT have relevant financial relationships | Karmveer Kaur: DO NOT have relevant financial relationships | Amit Sikder: No Answer
Meeting Info:

Scientific Sessions 2024

2024

Chicago, Illinois

Session Info:

Professional Development

Saturday, 11/16/2024 , 02:00PM - 03:00PM

Abstract Poster Session

More abstracts on this topic:
Hypertensive Disorders During Pregnancy: Patient Perspectives on Care, Barriers, and a Path Forward

Rasmusson Kismet, Dizon-townson Donna, Poll Justin, May Heidi, Anderson Angela, Hesterberg Kirstin, Graham Kaley, Petersen Sheralee, Jiricko Audrey, Campbell Amy

Health Literacy in Men and in Women With Type 2 Diabetes Undergoing Coronary Angiography

Neyer Magdalena, Frick Matthias, Drexel Heinz, Saely Christoph H, Vogel Johannes, Elsner Pascal, Plattner Thomas, Vonbank Alexander, Mader Arthur, Larcher Barbara, Leiherer Andreas, Muendlein Axel

You have to be authorized to contact abstract author. Please, Login
Not Available