Abstract Body (Do not enter title and authors here): Advances in modeling methodology and increased democratization of software led to rapid increase in novel prediction models. In clinical research, these models proclaim novel predictors, promise superior accuracy and meaningful improvements in patient outcomes. Developing an accurate prediction model requires rigorous methodology with great attention to study design and conduct. Failure to adhere leads to prediction bias and may result in harmful decisions.
We conducted a systematic review of research articles in heart failure listed in PubMed 2018 – 2023 and presented by their authors as developing prediction models, either diagnostic or prognostic. The study variables were selected based on PROBAST and TRIPOD (Table 1). These were summarized and the study conducts were evaluated against the gold standard of prediction modeling per PROBAST and TRIPOD. For each study, we calculated the optimal sample size, i.e. the minimal size required for at least 5% prediction accuracy and 90% shrinkage in parameters. Sentiment analysis was performed to estimate the prevalence of promotional language in the abstract corpus.

From 6,429 studies, 212 studies were presented as developing prediction models. Of those, 83 (39%) were truly about developing prediction models (TPM) and the rest were impostors, i.e. inferential models misrepresented as predictive.

Mostly, TPMs were prognostic, modeled binary outcomes, and employed machine-learning methods (Table 1). Commonly, essential data volume summaries were lacking and handling of missing data was inadequate. The deficit in the number of samples in model training was 1499 (-316, 3866) samples with 52(71%) studies below the optimal size.

Model validation or testing was not performed in 30% of the studies. Discrimination statistics were reported often in testing but not in validation. The calibration was rarely assessed at either phase. Thirty (36%) studies had enough information to implement risk score calculations. The sentiment analysis showed increased use of hype words in both TPMs and impostors relative to 2020-rates in Millar et al (2022).

Most TPM studies exhibit poor design, insufficient sample size, mishandling of missing data, and inadequate model evaluation. These deficiencies result in biased risk estimates, degrade the performance in external data limiting generalizability and clinical utility of the developed models. Excessive use of hype can be misleading and may impede objective evaluation by the readers.