Abstract Body (Do not enter title and authors here): Adjudication of clinical events by a central event committee (CEC) is the gold standard to assess outcomes in randomized cardiovascular outcome trials (CVOT). The standard adjudication process is very complex, expensive and labor-intensive but is considered essential to ensure sensitivity, specificity and consistency in the determination of clinical outcome. The emerging Artificial Intelligence (AI) technologies could automate, streamline the process and improve timeliness, reproducibility and costs.
In this analysis we sought to determine whether an AI based model could reproduce the results of a traditional CEC adjudication in a standard multi-national double blinded CVOT. We proposed an automated LLM-based solution which includes: 1) A pre-processing pipeline to clean and remove irrelevant information, addressing the challenges of multi-modal and highly variable input sizes. 2) A predictive model based on LLAMA 3.1 is built by fine-tuning the refined annotation data in 2881 patients from the Dulaglutide and cardiovascular outcomes in type 2 diabetes (REWIND) and determining whether an event occurred or not. We then compared the results of AI-adjudication vs the original CEC-adjudication. There were 9,901 people in the study. Out of these, 594 people in the Dulaglutide group and 663 people in the Placebo group had one of these events, as confirmed by the CEC. The AI model was able to identify with accuracy of 84% (k = 0.69) of CEC confirmed CV deaths, 83% (k = 0.64) of myocardial infarctions (MI) and 84% (k = 0.68) of strokes. The time to event hazard ratio (HR = 0.88, 95% CI (0.79- 0.99)) for the MACE-3 event by AI model generated MACE-3 (HR = 0.87, 95% CI (0.78 - 0.98)) is very comparable with CEC adjudicated MACE -3.
An AI model based on text review of clinical documents provided a substantial agreement with the traditional CEC showing the potential for AI to be routinely integrated in the adjudication process. Improvement in the models, including capability to interpret additional data could further enhance the accuracy and represent the future of clinical event adjudication.