RCT Risk Assessment

How can we estimate the risk of a clinical trial failing to be informative?

A challenge in clinical research is the large number of clinical trials which end without delivering results that can be used to inform policy or which otherwise increase the scientific body of knowledge. We can term these trials “uninformative” clinical trials.

Here, we use the definition of “informativeness” proposed by Zarin et al in Harms From Uninformative Clinical Trials, JAMA 2019.[1]

This can be due to several factors, including:

Sample size: If not enough participants are enrolled, the trial may lack the statistical power to detect a real effect.
Duration of trial: Trials need to be long enough to capture the full impact of the intervention being studied.
Incomplete plans: A missing Statistical Analysis Plan (SAP) raises a red flag, as it outlines how data will be analyzed, reducing the risk of bias later.

Studies have shown that a surprisingly low percentage of trials meet the criteria for being informative. The risk of a clinical study being uninformative is just as serious as regulatory, ethical, health, and other risks.

This highlights the need for better tools to assess the risk of a trial failing to deliver valuable insights.

Fast Data Science’s Clinical Trial Risk Tool

Fast Data Science has developed a tool called the Clinical Trial Risk Tool that uses Natural Language Processing (NLP) to analyze clinical trial protocols. You can read about how the Clinical Trial Risk Tool quantifies the risk of a trial ending without delivering informative results in our article in Clinical Leader.

This tool can identify factors in the protocol text that could increase the risk of the trial ending uninformatively. We initially focused on two pathologies: HIV and TB. We have now expanded the Clinical Trial Risk Tool to cover other disease areas such as Enteric and diarrheal diseases, Influenza, Motor neurone disease, Multiple sclerosis, Neglected tropical diseases, Oncology, COVID, Cystic fibrosis, Malaria, and Polio.

How does it work?

Protocol Analysis: The tool reads lengthy PDF documents containing details about the trial design, objectives, and methods.
Key Factor Extraction: Using NLP, the tool extracts crucial features such as planned sample size, effect estimate, presence/absence of a statistical analysis plan (SAP).
Risk Assessment: Based on these extracted factors, the tool sums a score to calculate a traffic-light risk level.

Benefits of the Tool

Early Identification of Design Flaws: By analyzing protocols before the trial begins, the tool can identify potential weaknesses in the trial design and allow researchers to make adjustments to improve the trial’s informativeness.
Improved Resource Allocation: Organizations can prioritize trials with a lower risk of failure, making the most of their resources.
Data-Driven Decisions: The tool provides valuable insights to researchers and decision-makers, enabling them to design and implement more informative trials.

Try the tool now by loading a trial protocol into it!

Fast Data Science’s Clinical Trial Risk Tool is a powerful example of how machine learning can be used to improve the efficiency and effectiveness of clinical research, as well as reduce financial risks.

How accurate is the Clinical Trial Risk Tool?

We have a blog post about the accuracy of the Clinical Trial Risk Tool and we have published details of its development in Gates Open Research under the title Clinical Trial Risk Tool: software application using natural language processing to identify the risk of trial uninformativeness.

References