Abstract Body (Do not enter title and authors here): Background
Cardiovascular magnetic resonance (CMR) is the gold standard non-invasive assessment of cardiomyopathy (CM); however, interpretation and CM classification from reports by clinicians and researchers at scale can be unclear and time-consuming.
Research Question
Can a large language model accurately classify CM from clinical CMR reports, to improve diagnostic accuracy?
Aims
To establish a novel natural language processing (NLP) model employing expression matching for CM detection and differentiation, and to compare its effectiveness with established methods.
Methods
In our study, an unsupervised rule-based NLP model was developed on 3561 consecutive text impressions from clinical CMR reports at Beth Israel Deaconess Medical Center and compared against physician adjudicated diagnosis (gold-standard labels) of 6 broad groups of CM (Figure 1). A semi-supervised biomedical bidirectional encoder representation from transformers (BioBERT) model was fine-tuned (80% training/20% validation) using NLP labels to improve generalization. The models’ performance was compared with comprehensive keyword search and ChatGPT4. We performed external validation by testing 1202 clinical CMR text impressions from Brigham and Women’s Hospital.
Results
The stepwise classification of our NLP model is shown in Figure 2. NLP and NLP-based BioBERT outperformed other methods (Table 1). In external validation, the mean AUC of NLP (0.92) and NLP-based BioBERT (0.93) had better performance compared to keyword search (0.71) and ChatGPT4 (0.73).
Conclusion
Our rule-based NLP and NLP-based BioBERT models accurately identify CM from CMR reports, enabling applications in widespread CMR report comprehension and large dataset research of cardiomyopathies.