Abstract Body (Do not enter title and authors here): Background: Sudden cardiac death (SCD) is a leading cause of mortality in patients with hypertrophic cardiomyopathy (HCM). Currently, calculating the 5-year SCD risk score is manual and time-consuming, limiting dynamic risk reassessment and timely monitoring. Recent advances in chain-of-thought large language models (LLMs) have the potential to provide accurate, automated risk stratification.
Hypothesis: Local, open-source LLMs can efficiently calculate SCD risk from routine clinical documentation and echo reports, enabling accurate risk stratification of patients with HCM within the hospital environment.
Methods: Clinical documentation and echo reports for 200 adult patients with HCM, managed between 2011 and 2023, were systematically extracted (Figure 1). Two medical professionals independently annotated each document (mean Cohen’s κ = 0.80; mean intraclass correlation coefficient = 0.96). Five DeepSeek-R1 models (1.5B–70B; 4-bit quantized) were deployed locally on GPUs and prompted in a zero-shot manner to extract all variables required for SCD risk stratification. Numeric annotations were considered correct if within ±2 mm or ±10 mmHg. Categorical annotations were scored as ‘present’, ‘absent’, or ‘not documented’. The primary endpoint was concordance between model predictions and the adjudicated SCD risk classification.
Results: Model metrics for each variable assessed are summarized in Table 1. Of the patients studied, 86.5% were classified as low-risk, 8% as intermediate-risk, and 5.5% as high-risk.
Successful extraction of all variables required for SCD risk calculation improved markedly with increasing model size, rising sharply from 41% at 1.5B parameters to 94% at 14B (Figure 1). Sensitivity showed a similar dependence on model size, especially for intermediate and high-risk categories. Models with 14B parameters or larger correctly identified about 80% of high-risk patients, whereas sensitivity for intermediate-risk patients plateaued around 70%. Notably, the 14B model had only one critical error (high- to low-risk misclassification), while the 32B and 70B models had none. Processing time per document increased with model size.
Conclusion: The 14B DeepSeek-R1 model hits the optimal trade-off between accuracy and latency, making it well-suited for real-time clinical workflows. It provides an excellent foundation for further fine-tuning and the development of a fully automated, agentic workflow, enabling dynamic longitudinal risk stratification.