Open access

Research Article

9 December 2024

Characterization of medical device randomized controlled trials with adaptive designs

Authors: Gui Su [email protected], Dan Shen, Dongyuan Deng, Qianqian Bai, and Hui XieAuthor Info & Affiliations

Publication: Journal of Comparative Effectiveness Research

Volume 14, Number 1

https://doi.org/10.57264/cer-2024-0011

PDF

Abstract

Aim: Adaptive designs are frequently used in drug randomized controlled trials (RCTs). However, their use in medical device RCTs remains unclear. We aimed to characterize medical device RCTs with adaptive designs. Materials & methods: We searched for adaptive RCTs in the following databases: ClinicalTrials.gov, International Clinical Trials Registry Platform and the International Standard Randomised Controlled Trial Number registry. Adaptive design keywords and medical device corporation names were used as terms to search the trial records registered between 1 January 2000 and 18 October 2024 in the databases. The annual number and proportions of adaptive trials were analyzed, and characteristics such as design type, sponsor, therapeutic area, trial stage and regulatory status were summarized. Results: Overall, 105 adaptive RCTs were identified from ClinicalTrials.gov, accounting for 2.112 per 1000 trials in 49,721 medical device clinical trials registered in ClinicalTrials.gov during the period. The average annual number of adaptive RCTs per 1000 clinical trials was the highest (8.55 ± 11.65) during 2005–2010, reduced to 3.33 ± 2.35 during 2011–2016, and significantly decreased to 1.29 ± 0.85 during 2017–2024 (p = 0.011). The most common adaptive designs were group sequential design (GSD, 50.5%), sample size reassessment (SSR, 17.1%) and investigating both superiority and non-inferiority (10.5%). Most RCTs were sponsored by the private sector (62.9%), conducted in Europe/North America (95.2%), in the field of heart disease (46.7%) and post-market trials (76.2%). Compared with pre-market RCTs, post-market RCTs showed more diverse adaptive designs such as response-adaptive randomization and adaptive enrichment. Conclusion: The average annual proportions of adaptive medical device RCTs in ClinicalTrials.gov has reduced in the last 10 years. The most-used adaptive designs in medical device RCTs are GSD, SSR and investigating both superiority and non-inferiority.

Shareable abstract

Adaptive medical device RCTs allow preplanned trial modifications and may be more efficient and better protect patients than conventional ones. Of the 105 medical device adaptive RCTs identified from ClinicalTrials.gov, GSD, SSR and investigating both superiority and non-inferiority are the most-used adaptive designs.

Plain language summary

What is this article about?

An adaptive design allows changing the study design of an ongoing clinical trial based on accumulating study data. Using adaptive designs can potentially increase trial efficiency, better protect patients and improve stakeholder acceptability of the trial. To better understand whether adaptive designs are used in medical device randomized controlled trials (RCTs), we identified and characterized adaptive medical device RCTs from public clinical trial databases.

What were the results?

We identified 105 adaptive RCTs from 49,721 medical device clinical trials registered in ClinicalTrials.gov (2000–2024). The average annual number of adaptive RCTs per 1000 clinical trials decreased significantly from the highest (8.55 ± 11.65) during 2005–2010 to the lowest (1.29 ± 0.85) during 2017–2024 (p = 0.011). The most common adaptive designs were group sequential design (GSD, 50.5%), sample size reassessment (SSR, 17.1%) and investigating both superiority and non-inferiority (10.5%). A GSD allows possible early stopping the trial for success or futility based on the results from analyzing accumulated data in an ongoing trial (interim analysis results). A SSR design allows adjusting the sample size based on interim analysis results. Most of the 105 adaptive RCTs were sponsored by the private sector (62.9%), in the field of heart disease (46.7%) and post-market trials (76.2%).

What do the results mean?

Adaptive designs appear not to be used commonly in medical device RCTs. GSD, SSR and investigating both superiority and non-inferiority are the most-used adaptive designs. Approaches to enhance the understanding of adaptive designs and encourage the adoption thereof in medical device clinical trials should be further developed.

Randomized controlled trials (RCTs) are considered the gold standard to determine the safety and effectiveness of medical devices. However, conducting conventional RCTs for medical devices is more challenging than for drugs because a surgical procedure is often involved in medical device clinical trials [1–4]. One of these challenges is to use a sham surgery as a control in a medical device RCT, which not only lacks benefit but also could impose risks to patients, raising ethical issues [1–3,5,6]. Patients and surgeons might be unwilling to accept the randomized assignment when a sham surgery is required or when surgeons have a strong preference for either the tested or the control procedure [1,3,7]. Another challenge is that medical device operator skills and experience in using the device could influence patient outcomes. Complications or adverse effects could be the learning effects and their incidence rates could reduce as the operator's skill matures [1–4]. Therefore, to evaluate the safety and effectiveness of a new device accurately, the operator's learning curve should be considered when planning an RCT [8–11]. To overcome these challenges, medical device RCTs with adaptive designs, which allow some flexibility during trial conduction without compromising the scientific soundness and integrity of the trial, appear to be a promising solution.

The 2016 US FDA Center for Devices and Radiological Health (CDRH) guidance ‘Adaptive Designs for Medical Device Clinical Studies’ defines an adaptive design study as “a clinical study design that allows for prospectively planned modifications based on accumulating study data without undermining the study's integrity and validity” [12]. In addition to the 2016 FDA/USA guidance, four regulatory guidance documents on adaptive design of drug clinical trials have been published by the FDA [13], European Medicines Agency [14] and China National Medical Products Administration [15,16], respectively (Table 1). These five regulatory guidance documents consistently require that the modifications or adaptations should be based on accumulating trial data (Table 1). The 2016 FDA guidance particularly emphasizes that the modification or adaptation should be prospectively planned.

Table 1. Worldwide guidance on adaptive design.

Organization	Year	Name of guidance	Definition of adaptive design	Ref.
FDA CDRH	2016	1. Adaptive Designs for Medical Device Clinical Studies, July 2016.	An adaptive design for a medical device clinical study is defined as a clinical study design that allows for prospectively planned modifications based on accumulating study data without undermining the study's integrity and validity.	[12]
FDA CDRH	2019	2. Adaptive Designs for Clinical Trials of Drugs and Biologics, November 2019.	An adaptive design is defined as a clinical trial design that allows for prospectively planned modifications to one or more aspects of the design based on accumulating data from subjects in the trial.	[13]
EMA	2007	3. Reflection Paper on Methodological Issues in Confirmatory Clinical Trials Planned with an Adaptive Design, October 2007.	A study design is called ‘adaptive’ if statistical methodology allows the modification of a design element (e.g., sample-size, randomization ratio, number of treatment arms) at an interim analysis with full control of the type I error.	[14]
NMPA	2021	4. Guiding Principles for Adaptive Design in Clinical Trials of Drugs (Trial Implementation) (Notice No. 6 of 2021).	Adaptive design is a clinical trial design that involves making appropriate modifications to the trial based on accumulated data during interim analyses, following a pre-established plan.	[15]
NMPA	2016	5. Biostatistical Principles for Clinical Trials of Drugs (Notice No. 93 of 2016).	Adaptive design refers to a type of design in which modifications are made to one or more aspects of a clinical trial during the process, utilizing accumulated data that are preplanned in the protocol. These modifications are implemented without compromising the integrity and validity of the trial.	[16]

CDRH: Center for Devices and Radiological Health; EMA: European Medicines Agency; FDA: The United States Food and Drug Administration; NMPA: China National Medical Products Administration.

An adaptation can occur in the trial procedure, such as for treatment duration, end point assessment, and/or statistical analysis, including sample size determination and randomization schemes. The 2016 FDA guidance describes several types of adaptive design for medical device clinical studies, including group sequential design (GSD), sample size reassessment (SSR), Bayesian sample size adaptation (BSSA), GSD with SSR, dropping a treatment arm, changing the randomization ratio/response-adaptive randomization, investigating both superiority and non-inferiority, adaptive enrichment, planning to adapt based on the total information, adaptation of the device or end point, seamless studies and adaptation with blinded data (Table 2) [12]. Some adaptive designs, for example GSD, SSR and BSSA, may reduce the number of study participants exposed to sham surgery, risky procedures, or underperforming procedures. Other adaptive designs, for instance seamless studies, may allow integration of the learning effect into the overall assessment of the device safety and effectiveness. Therefore, adaptive methods have the advantages of potentially increasing clinical trial efficiency, saving time and resources, enhancing patient protection and improving stakeholder acceptability of the clinical trial [12].

Table 2. Adaptive design description from the 2016 FDA guidance.

Adaptive design type		Description
A	Group Sequential Design (GSD)	Allow for interim analysis of the outcomes by treatment group (unblinded data) and possible early stopping for success or futility.
B	Sample Size Reassessment (SSR)	A study design with a preplanned sample-size reassessment; one or more preplanned interim observations are conducted to potentially adjust the sample size according to the comparison of the unblinded treatment group results.
C	Bayesian Sample Size Adaptation (BSSA)	Bayesian hierarchical models are used to adjust sample size as information accumulates.
D	GSD with SSR	Combine a GSD with interim observations, not only to stop early for success, but also to re-assess the sample size and to increase it according to a pre-specified plan.
E	Dropping a Treatment Arm (Drop the loser)	Drop one of these experimental arms during the study based on poor effectiveness performance.
F	Changing the Randomization Ratio (CRR)/Response Adaptive Randomization	Allows for a change in the randomization ratio between the control and treatment arms based on treatment outcomes (Response Adaptive Randomization).
G	Investigating both superiority and non-inferiority	To plan the study as a superiority trial and have a fallback hypothesis of non-inferiority, or to plan (and size) the study originally as non-inferiority but allow for an investigation of superiority.
H	Adaptive Enrichment	Plans to investigate, using unblinded data, at one or more interim observations, pre-specified patient subgroups that might have differing responses to the experimental device. Such analyses could be used in a preplanned way to modify the inclusion/exclusion criteria after an interim analysis.
I	Planning to Adapt Based on the Total Information	The stopping rule is based on the amount of information in the unblinded data, and this information is usually measured in terms of the variance of the primary end point.
J	Adaption of the Device or End point	Preplanned device or end point adaptations.
K	Seamless Studies	A feasibility investigation that smoothly transitions to a pivotal study in a preplanned manner, if no significant changes to the device or study are made.
L	Covariate adaptive randomization or covariate adjusted randomization (Adaptation with blinded data)	Adaptive design without breaking the blind. For example, using accumulative baseline data to change the allocation rule on an individual basis to obtain better balance between the control and treatment groups or using aggregate outcome data to make an adaptive decision.

Previous studies on adaptive designs focus on drug RCTs. However, the use of adaptive designs in medical device RCTs remains unclear. Moreover, which adaptive designs of medical device RCTs have been accepted and approved by regulatory authorities has not been explored. We, therefore, aimed to identify adaptive medical device RCTs from public clinical trial databases and characterize the features thereof. Our results may improve understanding of the current practice of adaptive design in medical device RCTs and encourage the adoption of adaptive designs for medical device clinical studies.

Methods

Search strategies for medical device RCTs with adaptive designs

We searched trial records registered between 1 January 2000 and 18 October 2024 in the following clinical trial databases: ClinicalTrials.gov (https://www.clinicaltrials.gov/) [17]; the WHO International Clinical Trials Registry Platform (ICTRP) (http://apps.who.int/trialsearch/) [18]; and the International Standard Randomised Controlled Trial Number (ISRCTN) registry (https://www.isrctn.com/) [19]. Adaptive design keywords and corporation name keywords were used to search the three clinical trial databases [12,20]. Further details of the search strategies are presented in the Supplemental Materials.

Screening method for medical device clinical trial with adaptive designs

Inclusion criteria

Clinical trials fulfilling the following criteria were included in the analysis: the trial is a randomized comparative study of the safety and effectiveness of a medical device or a surgical procedure; the trial has more than one arm/group/cohort; and the trial design belongs to any one of the adaptive designs that are described in the 2016 FDA guidance [12], as shown in Table 2.

Exclusion criteria

We excluded pharmacological trials; trials investigating behavioral interventions, psychotherapy, or rehabilitation procedure not related to medical devices; non-randomized trials; and medical device trials which did not have adaptive designs.

Screening quality control

Four reviewers screened the search results separately and identified medical device RCTs with adaptive designs. These results were compared, and any discrepancies were discussed with all authors to reach an agreement on whether to include the candidate trials as adaptive trials. To ensure that sufficient information on study design was reviewed, the full-text publication, study protocol, and/or statistical analysis plan (if available) of each candidate adaptive trial were retrieved and reviewed. The principal investigators of the candidate adaptive trials, for which the information from clinical trial databases was insufficient to support a decision on adaptive design type and there was no additional information of the trials, were contacted to determine the adaptive design type.

Extraction of clinical trial features

The following features of the identified adaptive RCTs were extracted from the trial records or publications: the year posted in the registry, country in which the trial was conducted, status of the trial (active not yet recruiting, recruiting, active but not recruiting, terminated, or completed), sponsor (private or public sector), therapeutic area, trial stage (pre- or post-market study), regulatory approval (yes or no) for pre-market trials and adaptive design type (GSD, SSR, BSSA, GSD + SSR, dropping a treatment arm, changing the randomization ratio/response-adaptive randomization, investigating both superiority and non-inferiority, adaptive enrichment, planning to adapt based on the total information, adaptation of the device or end point, seamless studies, adaptation with blinded data, others, or unknown type).

Statistical analysis

The number of identified adaptive RCTs in each year from 2000 to 2024 was recorded, and the proportion of adaptive RCTs over all the medical device trials registered in the clinical trial database in each year from 2000 to 2024 was calculated. The average annual numbers of adaptive RCTs per 1000 trials in the following three time periods: 2005–2010, 2011–2016 and 2017–2024 were calculated and compared using the Kruskal–Wallis test. The statistical significance level was set at p < 0.05. SPSS version 24.0 (IBM Corp., NY, USA) was used for the analysis. Descriptive statistics were performed on the extracted features of the identified adaptive RCTs. The number and proportion of each type of adaptive design were calculated and sorted. Other trial features, including country of trial, sponsor (private or public sector) and therapeutic areas were summarized. The adaptive design features were compared between pre- and post-market adaptive RCTs.

Results

A total of 105 adaptive RCTs were identified

In total, 2691 clinical trials were identified from the three clinical trial databases by using the adaptive design keyword search. Of the 2691 clinical trials, 44 adaptive RCTs were confirmed by manual screening and 2647 medical device trials were excluded for not having adaptive designs or for not being randomized device-related trials (Figure 1). The corporate name keyword search found 1954 medical device trials sponsored by or in collaboration with Medtronic, Johnson & Johnson, or Boston Scientific from the ClinicalTrials.gov database. Of these, 998 non-RCTs were excluded, 888 were excluded for not having adaptive designs and seven were duplicate trials of the 44 adaptive RCTs found from the adaptive design keyword search. Thus, 61 RCTs were confirmed to have adaptive designs. Overall, 105 (44 + 61) medical device RCTs with adaptive designs were included for the analysis, all from the ClinicalTrials.gov database (Figure 1). The full list of the 105 trials is provided as Supplemental Table 1.

Flow diagram showing the process of selecting adaptive clinical trials related to medical devices. — Figure 1. Adaptive trial selection and screening.
This displays the selection and screening process for medical device RCTs with adaptive designs from the following three databases: ClinicalTrials.gov (https://www.clinicaltrials.gov/), WHO ICTRRP (http://apps.who.int/trialsearch/) and ISRCTN (https://www.isrctn.com/).
RCT: Randomized controlled trial.

As of 18 October 2024, 49,721 records of medical device clinical trials were posted in the ClinicalTrials.gov database, and the proportion of adaptive RCTs in these total records was 2.112 per 1000 medical device clinical trials (105/49,721). The annual number of adaptive RCTs from 2005 to 2024 ranged from only one trial in 2007 to 12 trials in 2005 (Figure 2A). The second highest number of adaptive RCTs (10) was recorded in 2017. The FDA released the ‘Adaptive Designs for Medical Device Clinical Studies’ guidance in 2016. The annual number of adaptive RCTs per 1000 medical device trials posted in the database ranged from 0.0 in 2024 to 32 in 2005 (Figure 2A) and decreased continuously from 2005 to 2024. From 2005 to 2010, the average annual number of adaptive RCTs per 1000 trials was 8.55 ± 11.65, reduced to 3.33 ± 2.35 during 2011–2016 and further decreased to 1.29 ± 0.85 during 2017–2024. The differences among the three periods were statistically significant (p = 0.006, Figure 2B). A further comparative analysis revealed that the average annual number of adaptive RCTs per 1000 trials during 2017 to 2024 was significantly smaller than that during 2005–2010 (1.29 ± 0.85 vs 8.55 ± 11.65, p = 0.011, Figure 2B).

Graph showing the annual number and proportion of medical device randomized controlled trials with adaptive designs over time. — Figure 2. Annual number and proportion of medical device randomized controlled trials with adaptive designs.
**(A)** Annual number and proportion of adaptive randomized controlled trials (RCTs) from 2005 to 2024. **(B)** Comparison of the average annual number of adaptive RCTs per 1000 trials in three time periods: 2005–2010, 2011–2016 and 2017–2024.
*p = 0.011, 2017–2024 versus 2005–2010.
^#p = 0.044, 2017–2024 versus 2011–2016.

Characteristics of the 105 adaptive RCTs

The general characteristics of the 105 adaptive RCTs are presented in Figure 3. GSD (n = 53, 50.5%) was the most common type of adaptive design, followed by SSR (n = 18, 17.1%), investigating superiority and non-inferiority (n = 11, 10.5%) and GSD + SSR (n = 10, 9.5%) (Figure 3A). Bayesian SSA and dropping a treatment arm were used in two trials, respectively. The DEFUSE 3 trial (NCT02586415) features both adaptive enrichment and GSD, and the ICECAP trial (NCT04217551) shows both response-adaptive randomization and GSD. For our statistical analysis, these two trials were categorized as adaptive enrichment and response-adaptive randomization, respectively. Seven trials (6.7%, NCT02857309, NCT03490396, NCT04363541, NCT05084183, NCT05913622, NCT05685303, NCT05812911) claimed to be adaptive design RCTs, either in the trial title or in the description of study design. However, the information posted in the ClinicalTrials.gov database was insufficient to determine the specific type of adaptive design. Moreover, the principal investigators of these trials did not respond to our request for additional information regarding the design.

Graphs showing various types and characteristics of adaptive designs for medical device randomized controlled trials. — Figure 3. General characteristics of medical device randomized controlled trials with adaptive designs.
**(A)** Adaptive designs of the 105 randomized controlled trials. **(B)** The proportion of private sector sponsored trials was substantially higher. **(C)** Most of the trials were conducted in Europe or North America. **(D)** The therapeutic areas of the studies were diverse.
CRR: Change the randomization ratio; GSD: Group sequential design; SSA: Sample size adaptation; SSR: Sample size reassessment.

The proportion of adaptive RCTs sponsored by the private sector (n = 66, 62.9%) was substantially higher than that by the public sector (n = 39, 37.1%) (Figure 3B). Most of the adaptive RCTs (n = 100, 95.2%) were conducted in Europe or North America, with only four and one trials conducted in Brazil and China, respectively (Figure 3C). The therapeutic areas of the 105 adaptive RCTs are diverse, including heart diseases, neurovascular diseases, peripheral vascular diseases, cancer, respiratory diseases, diabetes, neuroscience, gastrointestinal diseases, urology, orthopedics, obesity and other areas (Figure 3D). Heart disease was the predominant therapeutic area, accounting for 46.7% (n = 49) of the 105 adaptive RCTs.

Adaptive features of pre- versus post-market trials

We further compared the adaptive design features of pre- versus post-market trials. A total of 80 (76.2%) of the 105 adaptive RCTs are post-market clinical trials and 25 (23.8%) are pre-market trials (Figure 4A & Table 3). Of the 25 pre-market adaptive RCTs, 72% (18/25) used GSD, SSR, or GSD + SSR (Table 3). Fourteen devices from the 25 pre-market trials have already been cleared by the FDA. Nine of the 14 approved devices were tested in pre-market trials using GSD (n = 5), SSR (n = 3), or GSD + SSR (n = 1) and the remaining five used the adaptive design of investigating superiority and non-interiority (Table 3). One common adaptive design feature of the pre- and post-market clinical trials was that both groups had the same top four common adaptive designs: GSD, SSR, GSD + SSR and investigating both superiority and non-inferiority (Figure 4B).

Graphs showcasing adaptive design elements in medical device trials at pre-market and post-market phases. — Figure 4. Adaptive design features of pre-market and post-market medical device clinical trials.
**(A)** The proportion of post- and pre-market trials is 76.2% and 23.8%, respectively. **(B)** Adaptive design types of post- and pre-market trials.
CRR: Change the randomization ratio; GSD: Group sequential design; SSA: Sample size adaptation; SSR: Sample size reassessment.

Table 3. Pre-market medical device RCTs with adaptive designs.

	ID/Name	Year	Title	Adaptive design	Trial status	Regulatory approval^†
1	NCT00129545	2005	WATCHMAN Left Atrial Appendage System for Embolic PROTECTion in Patients With Atrial Fibrillation (PROTECT AF)	GSD	Completed	Yes
2	NCT00231114	2005	Safety and Effectiveness of the Alair^® System for the Treatment of Asthma: A Multicenter Randomized Clinical Trial (Asthma Intervention Research (AIR2) Trial	GSD	Completed	Yes
3	NCT00980057	2009	Adaptive Cardiac Resynchronization Therapy Study (aCRT)	SUPR and non-inferior	Completed	Yes
4	NCT01182441	2010	Prospective Randomized Evaluation of the WATCHMAN LAA Closure Device in Patients With Atrial Fibrillation (AF) Versus Long Term Warfarin Therapy (PREVAIL)	GSD + SSR	Completed	Yes
5	NCT01054560	2010	SOLITAIRE™ FR With the Intention For Thrombectomy (SWIFT) Study	SUPR and non-inferior	Completed	Yes
6	NCT01135225	2010	EVOLVE: A Prospective Randomized Multicenter Single-blind Non-inferiority Trial to Assess the Safety and Performance of the Evolution Everolimus-Eluting Monorail Coronary Stent System (Evolution Stent System) for the Treatment of a De Novo Atherosclerotic Lesion (Evolve)	GSD	Completed	Yes
7	NCT01807585	2013	VenaSeal Sapheon Closure System vs Radiofrequency Ablation for Incompetent Greater Saphenous Veins (VeClose)	SUPR and non-inferior	Completed	Yes
8	NCT01966458	2013	A Prospective, Randomized, Controlled, Unblinded, Multi-Center Clinical Trial to Evaluate the HeartWare™ Ventricular Assist Device System for Destination Therapy of Advanced Heart Failure (DT2)	SUPR and non-inferior	Completed	Yes
9	NCT02064764	2014	Renal Nerve Denervation in Patients With Hypertension and Paroxysmal and Persistent Atrial Fibrillation CONDITIONS (Symplicity AF)	GSD	Completed	Not yet
10	NCT02202434	2014	REPRISE III: Repositionable Percutaneous Replacement of Stenotic Aortic Valve Through Implantation of Lotus™ Valve System - Randomized Clinical Evaluation (REPEISE III)	SUPR and non-inferior	Terminated. CAS/21 mm/nested registry terminated	Yes
11	NCT02439749	2015	Global Clinical Study of Renal Denervation With the Symplicity Spyral™ Multi-electrode Renal Denervation System in Patients With Uncontrolled Hypertension in the Absence of Antihypertensive Medications (SPYRAL HTN-OFF MED)	GSD	Active not recruiting	Not yet
12	NCT02439775	2015	Global Clinical Study of Renal Denervation With the Symplicity Spyral™ Multi-electrode Renal Denervation System in Patients With Uncontrolled Hypertension on Standard Medical Therapy (SPYRAL HTN-ON MED)	GSD	Active not recruiting	Not yet
13	NCT02392351	2015	A Boston Scientific Clinical Trial of Renal Denervation Using the Vessix Reduce™ Catheter and Vessix™ Generator for the Treatment of HyperTensioN: REINFORCE (REDUCE HTN:REINFORCE)	GSD	Completed	Not yet
14	NCT02924857	2016	A Randomized Trial to Confirm the Safety and Effectiveness of Chocolate Touch™ Paclitaxel Coated Balloon Catheter in Above the Knee Lesions (Chocolate Touch Study)	GSD + SUPR and non-inferior	Active not recruiting	Yes
15	NCT02733991	2016	Study of MiniMed™ 640G Insulin Pump With SmartGuard™ in Prevention of Low Glucose Events in Adults With Type 1 Diabetes (SMILE)	SSR	Completed	Yes
16	NCT02701283	2016	Transcatheter Aortic Valve Replacement With the Medtronic Transcatheter Aortic Valve Replacement System In Patients at Low Risk for Surgical Aortic Valve Replacement (LOW RISK)	GSD	Active not recruiting	Yes
17	NCT03288714	2017	A Prospective, Multicenter, Double-Blind, Sham-Controlled Adaptive Design Study to Confirm the Safety and Efficacy of NEST sTMS in Subjects With Major Depressive Disorder Who Have Not Responded to at Least One Antidepressant Medication in the Current Episode	SSR	Completed	Yes
18	NCT03299881	2017	Safety and Effectiveness of Transcutaneous Electrical Nerve Stimulation (TENS)-Assisted Weight Loss (ELIRA-1)	SSR	Terminated due to technical problems with the device	No
19	NCT03499964	2018	ROBUST III - Re-Establishing Flow Via Drug Coated Balloon For The Treatment Of Urethral Stricture Disease - A Randomized Control Trial (ROBUST III)	SSR	Active not recruiting	Yes
20	NCT03936699	2019	Safety and Effectiveness of Transcutaneous Electrical Nerve Stimulation (TENS)-Assisted Weight Loss and/or Appetite Suppression-ELIRA 2 study (ELIRA-2)	SSR	Completed	Not yet
21	NCT04612244	2020	A Prospective Randomized Pivotal Trial of the FARAPULSE Pulsed Field Ablation System Compared With Standard of Care Ablation in Patients With Paroxysmal Atrial Fibrillation (ADVENT)	GSD	Active not recruiting	Not yet
22	NCT04555135	2020	A Prospective, Multi-Center, Randomized Controlled Clinical Study To Measure The Effect Of Use Of Artificial Intelligence (AI) Enabled Computer Aided Detection 13 (CADe) Assistance Software In Detecting Colon Polyps During Standard Colonoscopy Procedures	GSD + SSR	Terminated based on interim analysis results	Not yet
23	NCT04235504	2020	ADvanced Hybrid Closed Loop Study in Adult Population With Type 1 Diabetes (ADAPT)	GSD +SSR	Completed	Not yet
24	NCT05883163	2023	STIMIT Activator 1 Pivotal Study	SSR	Planning not yet recruiting	Not yet
25	NCT05685303	2023	Safety and Efficacy of the Alleviant System for No-Implant Interatrial Shunt Creation in Patients With Chronic Heart Failure	Unknown	Recruiting	Not yet

†

The last search for regulatory status was conducted on 23 August 2023.

GSD: Group sequential design; SSR: Sample size reassessment; SUPR and non-inferior: Investigating both superiority and non-inferiority.

The post-market trials show some unique adaptive design features. GSD was predominant in the post-market trials, accounting for 56.3% (n = 44) of the 80 trials, whereas the proportion of GSD in pre-market trials was 36.0% (n = 9). The types of adaptive designs were more diverse in the post-market trials. In addition to the four most common adaptive designs, Bayesian SSA, dropping a treatment arm, adaptive enrichment and response-adaptive randomization were also used in the post-market trials, whereas none of these additional adaptive designs were found in the pre-market trials (Figure 4B). Furthermore, GSD combined with less commonly used adaptive designs (adaptive enrichment and response-adaptive randomization) were observed in two post-market trials: the DEFUSE 3 trial (NCT02586415, GSD + adaptive enrichment) and the ICECAP trial (NCT04217551, GSD + response-adaptive randomization). The DEFUSE 3 and ICECAP trials further highlight the diverse adaptive designs in post-market trials. These two trials are also typical examples of creating novel adaptive design by combining a common adaptive design, such as GSD, and a rarely used adaptive design, for example adaptive enrichment and response-adaptive randomization, to improve the trial efficiency and enhance patient protection.

The DEFUSE 3 trial aimed to investigate whether patients, who were in 6–16 h-window of acute ischemic stroke symptom onset and had a favorable imaging profile on computed tomography perfusion or magnetic resonance imaging, could respond better to mechanical thrombectomy plus medical management than to medical management alone. The adaptive design of the DEFUSE 3 trial, which was based on the closed testing theory and group sequential method, allowed enrichment of the subgroup patients that benefited the most from the experimental treatment. The trial included two pre-planned interim analyses (n = 200 and n = 340, respectively). At each interim analysis, accumulating outcome data was compared with a predefined futility bound to determine whether to continue recruitment in the overall group, shift accrual and testing to a subgroup, or stop in its entirety. If an interim analysis concluded futility and a subgroup with the best chance of showing efficacy was selected, then the adaptive design allowed the subsequent enrollment to be limited to the patients in the subgroup. Thus, the number of participants in this subgroup could expand more than it may have without the enrichment adaptive design [21]. During the trial implementation, an early interim analysis was performed when the number of enrollments was 182. Because the results of the interim analysis exceeded the prespecified efficacy boundary (p < 0.0025), the trial was terminated [22].

The ICECAP trial, a randomized, response-adaptive and duration (dose) finding clinical trial, aims to investigate the association between increasing durations of induced hypothermia and good neurological outcomes and to determine the optimal duration of induced hypothermia for neuroprotection in comatose survivors of cardiac arrest. Response-adaptive randomization and a group sequential method are used in this trial. The response-adaptive randomization is based on a statistical model of the weighted 90-day mRS (the primary end point) of different hypothermia duration arms. The adaptation of patient randomization to different duration arms is based on a statistical model of duration-response curve that is updated frequently with the accumulating primary end point data. In addition to the response adaptative randomization, multiple interim analyses are implemented to stop the trial early for futility if no hypothermia duration arm is highly likely to show a greater benefit than the 6-hour duration arm. The trial is currently recruiting patients [23].

Discussion

A decreasing trend of using adaptive design in medical device RCTs

We identified 105 medical devices RCTs with adaptive designs, accounting for 2.112 adaptive RCTs in every 1000 (105/49,721) medical device clinical trials registered in the ClinicalTrials.gov database for the past two decades (from 2000 to 2024). The average annual number of adaptive RCTs per 1000 trials (1.29 ± 0.85) during the most recent 8 years (2017–2024) was significantly lower than during the earlier period of 2005–2010 (8.55 ± 11.65) and numerically lower than during the earlier period of 2011–2016 (3.33 ± 2.35). This result opposed our expectation that adaptive designs could be used more frequently after the release of the 2016 FDA guidance of ‘Adaptive Designs for Medical Device Clinical Studies’. The number of adaptive RCTs in 2017 (n = 10) was the second highest during 2005–2024; however, the proportion of adaptive RCTs in 2017 was only 3.23 per 1000 medical device clinical trials. Our search methodology might partially contribute to the small number of adaptive medical device RCTs in the recent 8 years (2017–2024). Our search strategies could miss some adaptive RCTs, especially those that are more recent, because of limited clinical trial information posted in the databases and lacking publications for ongoing trials. Providing detailed adaptive design information when registering the trials in the three clinical trial databases is currently not mandatory. There are recommendations to include all possible adaptations in the prospective design, even if the adaptations did not occur during the actual study, in the Consolidated Standards of Reporting Trials (CONSORT) reporting structure [24]. Increasing concerns on data integrity and ethics and the challenges in the implementation of adaptive RCTs are critical barriers for adopting adaptive designs in medical device RCT, which might also explain the decreasing trend of using adaptive designs in the recent 8 years.

Barriers for adoption of adaptive design in medical device RCTs

Firstly, adaptive designs might not have been widely adopted because of researchers' concern on the complexity of implementing adaptive trials and trial ethics. Chongwe et al. collected the opinions on the methodology advantages and ethical aspects of adaptive trials compared with conventional RCTs from 17 researchers with experience in adaptive trials and found a general agreement on implementing adaptive clinical trials being more challenging than conventional trials in terms of obtaining informed consent [25].

Secondly, for the specific adaptive design of changing the randomization ratio/response-adaptive randomization, participants in the study of Chongwe et al. raised the ethical concern on disruption of clinical equipoise after changing randomization allocation [25]. In a sophisticated review on the topic of clinical equipoise concern in adaptive trials, Bothwell and Kesselheim suggested distinguishing between theoretical and clinical equipoise and clarified that theoretical equipoise requires that random allocation of study participants is ethical only if the benefit of a tested treatment is completely uncertain, whereas clinical equipoise allows random allocation until there is sufficient clinical evidence supporting the benefit of the tested treatment [26]. Thus, they believe that the general ethical concern on equipoise of adaptive randomization is based on the theoretical equipoise, while clinical equipoise is optimal in adaptive randomization trials because the adaptation allows assigning more participants to the treatment deemed likely to be beneficial to generate sufficient supporting clinical evidence [26].

Thirdly, consistent with a previous study [27], the participants in the study of Chongwe et al. also expressed concerns on the scientific validity and integrity of adaptive trials [25]. For example, the safety dataset might become insufficient because of early trial termination; operational bias could be introduced when the trial procedure and/or statistical analysis plan are modified [25,27]. Nevertheless, the 17 participants generally agreed on the efficiency advantage of adaptive trials over conventional RCTs, because adaptive trials could potentially be completed more quickly and with fewer participants [25].

GSD & SSR are the most commonly used adaptive design in medical device RCTs

Our results show that the most predominant type of adaptive design used in medical device RCTs is GSD (53/105, 50.5%), followed by SSR (18/105, 17.1%). Similarly, in a survey of adaptive design applications submitted to the Center for Devices and Radiological Health of the FDA between 2007 and 2013, Yang et al. found that the top three most-used adaptive design types were GSD (26.3%), SSR (16.7%), and GSD + SSR (19.1%) [28]. Previous studies on adaptive designs of drug and device trials also showed that GSD and SSR are the most common adaptive designs. An early study by Morgen et al. found that 48% of the identified 23 adaptive trials from ClinicalTrials.gov between 2008 and 2011 are GSD [29]. Furthermore, Lin et al. showed that 47.1% of 87 adaptive trials identified from the FDA CBER database (2008–2013) used GSD and 23.0% used GSD + SSR [30]. In a study focusing on adaptive design features of drug trials, Bothwell et al. found that the most frequently used adaptive design type was seamless phase II/III (57%) followed by GSD (21%) [31]. However, Cerqueira et al. found that only 9% (n = 7) of 78 adaptive trials identified by screening publications between 2006 and 2017 were GSD [32]. Publication bias could contribute to the low proportion of GSD in that study because investigators are often unlikely to publish their trial results when the clinical trials have negative results or were terminated due to futility from an interim analysis.

The reason for GSD being the most widely used adaptive design type is associated with its potential benefits. GSD allows early trial termination or completion depending upon interim analysis results, thus possibly saving time and resources, and minimizing the number of study participants who must be exposed to the clinical risks in clinical trials. These obvious efficiency and ethical advantages of GSD are particularly important for medical device clinical trials using a sham surgery as a control because GSD could potentially reduce the number of participants exposed to sham surgery, risky procedures, or underperforming procedures. Despite the advantages of GSD, statistical considerations, such as appropriately adjusting the significant level and ensuring the sufficiency of safety data, must be implemented to support a scientifically valid early termination [12,27].

We found that adaptive design types used in post-market trials were more diverse than those in pre-market trials. Less common adaptive methods, such as dropping a treatment arm, adaptive enrichment and response adaptive randomization were only found in post-market studies. Time, funding and other resource are the obvious factors to consider when deciding whether to use and what type of adaptive design to use. More importantly, the trial purpose (whether for regulatory approval), study aim, the potential risks to study participants, the study end points and logistics complexity of running adaptive trials should also be taken into account [12]. For example, the enrollment speed and the time to measure the primary end point, which is often used for interim analysis and trial adaption, should be suitable for an adaptive design [12]. If the time to end point measurement is longer than the time required for the total enrollment, then a pre-planned interim analysis seems impossible unless slowing down the subject enrollment, which would most likely increase the cost and reduce trial efficiency. In addition, the trial aim of a pre-market trial usually strictly focuses on safety and effectiveness assessment to meet the regulatory requirements, whereas the study aim or hypothesis of a post-market trial has a much broader scope in addition to safety and effectiveness assessment, such as identifying the patient subgroup that can benefit the most from the investigated device or the therapy (the DEFUSE 3 trial) and discovering the optimal treatment regimen (the ICECAP trial). Furthermore, time and funding pressure on post-market trials might be less than those on pre-market trials. Thus, more complex adaptive designs could be considered for post-market trials.

Regulatory perspective on adaptive designs of medical device clinical trials

We found that 72% (18/25) of the 25 pre-market trials used GSD, SSR, or GSD + SSR and nine of the 14 FDA approved medical devices were tested in pre-market clinical trials using GSD, SSR, or GSD +SSR adaptative designs. These results suggest that the FDA accepts GSD and SSR adaptive designs. The 2016 FDA CDRH guidance ‘Adaptive Designs for Medical Device Clinical Studies’ lists more than 10 types of adaptive designs in addition to the commonly used GSD and SSR [12]. The FDA CDRH does not appear to have any preference to certain adaptive design types. The predominant GSD and SSR in pre-market trial could be attributed to the fact that these two adaptive designs were the most frequently used ones in the adaptive design submissions to FDA CDRH [28,33]. Notably, the 2016 FDA CDRH guidance clarifies the critical aspects of adaptive designs, including the definition, principles, when and whether to use them, different types and how to implement them, and the purpose of the guidance is to encourage investigators or sponsors to consider adaptive designs in medical device trials [12,28,33]. Balser et al. summarized the regulatory perspective on adaptive designs and emphasized that the FDA has become increasingly positive toward adaptive designs [33]. Therefore, consistent with the opinions of the previous study by Balser et al. we also think that FDA CDRH allows any type of adaptive design providing that study participant protection, data validity and integrity and scientific soundness of the trials are not compromised by the adaptive design [33]. Comprehensive advance planning, prespecified type I error control, data monitoring committee engagement, meaningful clinical significance, fully leveraging clinical trial simulation and early and close interaction with regulators are critical for planning an adaptive pre-market clinical trial [12,28,33].

Unique adaptive design consideration for medical device clinical trials

The 105 adaptive RCTs identified in this study include diverse adaptive designs, such as GSD, SSR, Bayesian SSA, GSD + SSR, dropping a treatment arm, investigating both inferiority and superiority, adaptive enrichment and response-adaptive randomization. However, none of the 105 adaptive RCTs considered learning curve effects in the adaptive designs, although the learning process of a surgical procedure can critically impact the overall safety and effectiveness outcome assessment [8–11]. McCulloch et al. recommend a five-stage process to develop and evaluate a new surgical procedure, including idea, development, exploration, assessment and long-term study stages [34]. The learning curve of a new medical device and its associated procedure should be evaluated in the development and exploration stages before implementing an RCT to assess the effectiveness. A Bayesian hierarchical model has been proposed by Cook et al. to evaluate learning curve effects on outcome results in RCTs and adjust the outcome results for the learning curve effects [8]. Tekkis et al. used a risk-adjusted cumulative sum model to assess the learning curve effects on surgeon- and patient-specific outcomes of laparoscopic colorectal resections [35]. Conducting a feasibility/exploratory study to understand the learning curve effects and a pivotal study to assess the effectiveness and safety separately as two clinical trials is not cost-effective and may also delay regulatory approval of novel medical devices. To speed up public access to innovative medical devices, a seamless adaptive design may allow the combination of feasibility or exploration assessment together with a pivotal study evaluating device effectiveness. However, the type one error (alpha) and the corresponding power to detect the predefined effect size must be controlled to assure the validity and integrity of a medical device clinical trial with seamless adaptive design.

Study limitations

There are some limitations in this study. The number of adaptive RCTs might be underestimated, because some medical device clinical trials might not be registered in the three databases and some trials with an adaptive design might not been identified by the search terms because the information posted in the clinical trial databases might not contain these terms. Moreover, only RCTs were considered for adaptive design features in this study. Adaptive designs can also be used in single arm clinical trials. Nevertheless, we exclusively focused on medical device RCTs and provided a relatively thorough overview on adaptive design features of medical device RCTs.

Conclusion

The annual proportion of medical device RCTs with adaptive designs in all medical device clinical trials registered in ClinicalTrials.gov has reduced in the last 10 years, suggesting that adaptive designs might not be commonly used in these trials. GSD, SSR, GDS + SSR and investigating both superiority and non-inferiority are the most-used adaptive designs in medical device RCTs. Approaches to enhance the understanding of adaptive designs and to encourage the adoption thereof in medical device clinical trials should be further developed in the future.

Summary points

•

Conducting conventional randomized controlled trials (RCTs) for medical devices is challenging due to ethical issues with sham surgeries and the influence of operator skills on outcomes.

•

Adaptive designs in clinical trials, which allow for modifications based on accumulating data, may increase trial efficiency, save resources, enhance patient protection and improve stakeholder acceptability.

•

A total of 105 medical device RCTs with adaptive designs were identified from ClinicalTrials.gov database.

•

GSD, SSR, GDS + SSR and investigating both superiority and non-inferiority are the most-used adaptive designs in medical device RCTs.

•

The types of adaptive designs were more diverse in post- than in pre-market trials.

•

GSD combined with less commonly used adaptive designs (adaptive enrichment and response-adaptive randomization) were only observed in post-market trials.

•

The study showed a decrease in the annual proportion of adaptive trials in medical device RCTs over the past decade, suggesting more efforts are needed to enhance the understanding of adaptive designs and to encourage the adoption of adaptive designs in medical device clinical trials.

•

The reason for adaptive designs not being commonly used in medical device RCTs may be associated with complexity in trial execution, trial ethics and concerns about scientific validity and integrity.

•

Learning curve effects may be considered when designing a medical device RCT with an adaptive design.

Author contributions

G Su, D Shen, D Deng and H Xie designed the study. G Su, D Shen, D Deng and Q Bai screened the search results separately and identified medical device RCT with adaptive designs. The screening results of G Su, D Shen, D Deng and Q Bai were compared, and then the discrepancy was discussed with all the authors to reach an alignment on whether to include the trials as adaptive trials. G Su drafted the manuscript. All authors reviewed and provided substantial and comprehensive feedback on each draft of the manuscript. All authors read and approved the final manuscript for submission.

Financial disclosure

All authors are employees of Medtronic. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

Competing interests

The authors have no competing interests or relevant affiliations with any organization or entity with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.

Writing disclosure

No writing assistance was utilized in the production of this manuscript.

Ethical conduct of research

All data included in the analyses were aggregate in nature, derived from trial records and therefore ethical approval was not required.

Data sharing statement

No additional unpublished data are available.

Open access

This work is licensed under the Attribution-NonCommercial-NoDerivatives 4.0 Unported License. To view a copy of this license, visit https://creativecommons.org/licenses/by-nc-nd/4.0/

Supplementary Material

File (supplementary data.docx)

Download
70.57 KB

File (supplementary materials.docx)

Download
26.08 KB

References

Papers of special note have been highlighted as: •• of considerable interest

Bernard A, Vaneau M, Fournel I, Galmiche H, Nony P, Dubernard JM. Methodological choices for the clinical development of medical devices. Med. Devices 23(7), 325–334 (2014).