Abstract Body (Do not enter title and authors here): Background: Assessing clinical complexity in heart failure (HF) patients is essential for effective resource allocation. Despite notable limitations, CMS currently uses administrative claims to estimate complexity and determine provider payments. Unstructured clinical notes contain valuable patient information that can enhance risk assessment. We introduce a novel deep-learning approach that identifies clinically significant features from notes to improve risk assessment in HF.

Methods: We identified all individuals with hospitalizations for HF in a large, diverse health system and defined CMS HF comorbidities using ICD codes. In parallel, we extracted clinical notes from these encounters to determine information relevant to the same conditions using a novel natural language processing-based topic modeling approach. We digitized all notes into a vector space using a previously trained embedding model (NV-embed) with learned contextual word associations. We projected the descriptions of CMS ICD-code-based risk factor categories into the embedding and used cosine similarity to identify whether notes provided additional information. We used logistic regression models for 30-day readmission and 30-day and 1-year mortality to evaluate the predictive performance of the approaches.

Results: We identified 40,604 HF hospitalizations, with 30-day and 1-year mortality rates of 6.4% and 18.2%, and 30-day readmission rate of 23.3%. Clinical notes of those with vs without a code-based comorbidity had the information identifiable on topics (cosine similarity, 0.73 vs 0.57). Among those without coded conditions, notes varied substantially in information corresponding to the topic (range of coefficient of variation, 0.21-0.50). Models using note-defined topics as predictors consistently outperformed those using CMS risk scores, with higher AUROCs for 30-day mortality (0.81 vs 0.63), 1-year mortality (0.85 vs 0.64), and 30-day readmissions (0.78 vs 0.59). The average net reclassification index was 0.37, indicating improved risk stratification using information from notes vs structured CMS risk scores.

Conclusion: A novel deep-learning approach applied to unstructured clinical notes more accurately identifies conditions of interest, with the models based on these features significantly outperforming the current CMS risk-adjustment models for post-hospitalization events. This has important implications for defining the next generation of tools for hospital profiling for HF care.