Abstract Body (Do not enter title and authors here): Hypothesis and Purpose:
We previously developed RECTIFIER, a Retrieval-Augmented Generation system for clinical trial patient screening that surpassed manual methods in accuracy and efficiency, closely matching expert clinician assessments at substantially lower cost. We hypothesize that AI-assisted patient screening using RECTIFIER will significantly increase the likelihood of eligibility determination and enrollments for an active heart failure (HF) clinical trial.
Study Design and Methods:
A prospective randomized controlled trial was conducted with 4,476 patients identified through a structured query within Mass General Brigham. Patients were randomized in a 1:1 ratio to either manual screening or AI-assisted screening (Figure 1). Each study staff member was allocated an equal number of hours each week for AI-assisted and manual screening.
Sample Size:
4,476 patients were randomized.
Population Studied:
Patients identified as potentially eligible for an active HF clinical trial.
Intervention:
Two different screening methods: manual screening by study staff and AI-assisted screening using RECTIFIER.
Power Calculations:
We estimated the study's power using bootstrapping and simulations. Survival data were analyzed with Fine-Gray for the primary endpoint and win-ratio for the secondary endpoint. At an alpha level of 0.05, a total of 4,476 patients provided over 99% power for both endpoints.
Primary End Points:
Eligibility determination
Secondary End Point:
Hierarchical win ratio prioritizing enrollments over eligibility determinations.
Outcomes:
Patients in the AI-assisted screening group were significantly more likely to have the primary endpoint of eligibility determination than those in the manual group (458 vs 284 individuals, 20.4% vs 12.7%, p <0.001) with a subdistribution hazard ratio of 1.78 (95% CI: 1.54–2.06, p < 0.001) (Figure 2). This indicates that those randomized to AI-assisted screening had a 78% higher rate of eligibility determination with AI-assisted screening compared to manual screening, in patients who are not ineligible. The secondary outcome, assessed using the hierarchical win ratio, was 1.90 (95% CI: 1.87 - 2.61, p < 0.0001). The AI-assisted screening method had more wins over the manual screening method for both enrollment and eligibility determination components of the hierarchical outcome. The AI-assisted method had 35 (1.56%) enrollments compared to 19 (0.85%) enrollments in the manual method at the end of the trial (p = 0.041).