Guest post by Safeer Khan, Lecturer at Department of Pharmaceutical Sciences, Government College University, Lahore, Pakistan*
Pilot studies are a cornerstone of modern clinical research. These preliminary trials allow researchers to assess the feasibility of their hypotheses, refine methodologies, and identify potential obstacles before embarking on larger, more expensive trials with significant ethical implications. A well-structured pilot study not only saves time and resources but also reduces the chances of costly failures during later phases of the clinical trial process. In contrast, poorly planned pilot studies often lead to what is known as “research waste.” This issue was highlighted in a review of 640 Phase 3 trials involving novel therapeutics, which found that 54% of them failed during clinical development [1].
This blog will delve into the four key elements that contribute to the success of a pilot study: study design, cost modelling, risk management, and the value of informativeness.
A poorly designed pilot study can lead to misleading conclusions. Underpowered pilots may falsely suggest that an intervention is ineffective, resulting in its abandonment. Conversely, overly optimistic results can advance weak interventions into larger trials, where they may ultimately fail. Similarly, a good pilot study design reassures regulators, funders, and ethics boards that further investment is justified.
While large trials aim for statistically significant results, pilot studies focus on fundamental questions such as whether recruitment targets can be met, if interventions are deliverable, and whether participants adhere to the study protocols. Similarly, in terms of sample size, pilot studies are not intended to detect treatment effects. Instead, they are selected to test feasibility endpoints. Julious (2005) suggested a “rule of 12,” meaning at least 12 participants per group to estimate parameters such as variance and recruitment rates [2]. While seemingly modest, such numbers can provide enough signal to forecast challenges in scaling up to hundreds or thousands of participants.
A well-designed pilot study design plays a crucial role in feasibility testing. Recruitment is often one of the first obstacles faced in trials. A survey by the Institute of Cancer Research, which involved 500 cancer patients, revealed that while 95% of patients were interested in participating in a clinical trial, only 11% actually enrolled. By simulating recruitment pathways, pilot studies help identify potential issues and reduce the risk of stalled enrollment, which is a major cause of premature trial termination. Beyond recruitment, pilots fine-tune protocols: randomization procedures, electronic data capture systems, and follow-up schedules. Such refinements protect full trials from costly operational missteps.
Check your trial design
Clinical trials are known for their high costs. National Institutes of Health (NIH) spends an average of $33.8 million on developing each drug, with $13.9 million allocated to Phase 1, $22.2 million to Phase 2, and $12.9 million to Phase 3 trials [3]. Read more: Clinical Study Cost Breakdown. Moreover, poor financial management is a major contributor to trial failure. A study found that 22% of Phase 3 trials were discontinued due to budgetary issues [1].
Pilot study cost modelling helps prevent such failures by acting as a “financial stress test.” For example, if a pilot involving 50 participants costs $250,000, investigators can estimate whether enrolling 5,000 participants in Phase III would be financially feasible. Additionally, procedures that are resource-intensive, such as MRI imaging, may seem manageable in a 20-patient pilot but could become unfeasible at a larger scale. Identifying such issues early on can help avoid significant financial misallocations.
Pilot studies also provide valuable financial insights by estimating recruitment costs, intervention delivery expenses, and monitoring needs. As funding bodies increasingly require clear evidence of feasibility and cost awareness before approving large grants, a pilot study that includes data-driven cost models strengthens the credibility of funding proposals. Due to this reason, the NIHR Guidance on Pilot Studies strongly recommends incorporating cost analysis in feasibility studies to ensure better budgeting accuracy.
When transparently reported, pilot study risks serve as “early warning systems” that strengthen the resilience of full trials. By converting risk into foresight, pilots transform potential liabilities into protective mechanisms. In particular, a pilot study can be employed to proactively address the following risks:
Every clinical trial carries inherent risks, but pilot study risk highlights them in a condensed form. The most serious ethical concern arises when pilots expose participants to interventions without generating valuable knowledge. Similarly, research lacking social or scientific value is inherently unethical, regardless of sample size. Therefore, ensuring a well-designed and purposeful pilot study is not only critical for the trial’s success but also for maintaining the ethical integrity of the research process.
Pilot studies are crucial in uncovering scientific risks, particularly the risk of misinterpretation. Since pilot studies are typically underpowered, they may produce false signals, either suggesting that an intervention is effective when it is not or discouraging further exploration of potentially promising treatments. Identifying these issues early helps prevent misdirection in research, ensuring that valuable interventions are not prematurely abandoned.
Pilot studies provide a valuable opportunity to uncover operational risks early on. These failures often surface first in pilot trials, such as low recruitment rates, poor adherence, or logistical bottlenecks. Identifying these issues in a pilot study allows researchers to address them before they jeopardize larger trials.
Pilot studies reveal financial risks by identifying cost-intensive elements that cannot scale. Without this early insight, full-scale trials are at risk of budget overruns, necessary amendments, or even collapse. Pilots help pinpoint inefficiencies in resource allocation, staffing, and infrastructure, allowing for adjustments before committing to large-scale expenditures.
A common criticism of pilot studies is their limited pilot study informativeness. Uninformative pilots waste resources and expose participants without contributing meaningful insights. Chalmers & Glasziou (2009) estimated that up to 85% of research investment is wasted [4], partly due to poorly designed and poorly reported pilots.
Pilot Study Informativeness describes how well the study contributes to understanding critical aspects of the research, such as feasibility, potential risks, and the effectiveness of interventions. For example, a surgical pilot study design that enrolls only eight patients without clear feasibility endpoints might add little value and could be criticized as unethical. On the other hand, an informative pilot in maternal health might demonstrate that sending text message reminders increases prenatal visit adherence by 35%, leading to a scalable intervention in Phase III.
Pilot studies play a pivotal role in the clinical trial process, laying the groundwork for successful large-scale research. By addressing key aspects such as design, cost efficiency, risk management, and the ability to generate useful data, pilot studies reduce uncertainties and ensure the ethical integrity of clinical trials. As the demand for evidence-based treatments continues to grow, the value of well-conducted pilot studies cannot be overstated. They provide the critical insights needed to refine methodologies, minimize risks, and maximize the potential for successful outcomes, ultimately advancing the field of medical research.
Hwang, T.J., et al., Failure of investigational drugs in late-stage clinical development and publication of trial results. JAMA internal medicine, 2016. 176(12): p. 1826-1833.
Julious, S.A., Sample size of 12 per group rule of thumb for a pilot study. Pharmaceutical Statistics: The Journal of Applied Statistics in the Pharmaceutical Industry, 2005. 4(4): p. 287-291.
Zhou, E.W., M.J. Jackson, and F.D. Ledley. Spending on phased clinical development of approved drugs by the US National Institutes of Health compared with industry. in JAMA Health Forum. 2023. American Medical Association.
Chalmers, I. and P. Glasziou, Avoidable waste in the production and reporting of research evidence. The Lancet, 2009. 374(9683): p. 86-89.
Guest post by Safeer Khan, Lecturer at Department of Pharmaceutical Sciences, Government College University, Lahore, Pakistan Multi-Arm & Multi-Stage (MAMS) Clinical Trials Design Tips The design of clinical trials is increasingly challenged by the Rising Costs, limited availability of eligible patient populations, and the growing demand for timely therapeutic evaluation. Traditional parallel-group designs, which typically compare a single intervention to a control, are often insufficient to meet these pressures in terms of speed, efficiency, and resource utilization.
You can use the t-test when you want to compare the means (averages) of continuous data between two groups, such as blood pressure or maximum concentration of a drug in urine (Cmax). If you have data with a dichotomous outcome, you can use the Chi-Squared test instead - please try our Chi-Squared sample size calculator. The calculator below will calculate the minimum sample size for you. Your expected effect size d is the standardised effect size according to Cohen’s definition.
You can use the Chi-Squared test to analyse your trial data or A/B test data if you have two groups with a dichotomous outcome. For example, you have two arms in your trial: the placebo and the intervention arm, and your endpoint is either yes or no, such as “did the subject experience an adverse event during the trial”. The calculator below will calculate the minimum sample size for you. Your expected effect size w is the standardised effect size according to Cohen’s definition.
