Optimizing clinical scientific research: the cohort intervention random sampling study with historical controls

In explanatory and pragmatic RCTs, potential participants are informed about both the experimental and control group before randomization (pre-consent conversation). The pre-consent conversation in an RCT could potentially be an unnecessary barrier to recruitment in RCTs for both clinician and potential participant [25,26]. Clinicians may be reluctant to adequately inform the potential participant because they are hesitant to acknowledge uncertainties about alternative treatments [27] or do not (completely) inform them that the treatment they will receive is randomly chosen [10]. For participants, the decision not to participate in an RCT may stem from both a reluctance to engage in scientific research and resistance to the experimental treatment [6]. Therefore, different consent models have been suggested for pragmatic trials [26].

Furthermore, the decision to participate in the trial (apart from altruism) is to receive the experimental treatment, with a 50% certainty, because standard care is always available outside the trial [10,28]. If then placed in the control arm (the nonpreferred arm), this can result in emotional distress which can damage the physician–patient relationship [8,29], disappointment bias and noncompliance in the study arm [28,30,31].

The pre-consent conversation with abovementioned negative consequences was for the first time eliminated in Zelen’s single consent design, introduced in 1979. It is characterized by randomization without prior IC with the goal of improving clinicians’ and patients’ trial participation and to minimize bias [8]. IC for trial participation is only obtained from participants who are allocated to the experimental group. Potential participants who are allocated to the experimental treatment have 100% certainty, instead of 50% in an RCT, to receive the experimental treatment if they agree, without the uncertainty of randomization. However, the fact that control group patients are unaware of study participation is considered a major ethical concern [32,33]. Therefore, the Zelen double consent design was introduced [34]. In this ameliorated design, IC is obtained for both the experimental and control group after randomization. If a control group patient declines to receive standard care, the patient can cross over to alternative therapies, including the experimental treatment [34]. Nevertheless, Zelen’s double consent design fully eliminates the whole purpose of randomization, since each participant gets the opportunity to change to their favorite treatment.

To further ethically optimize the IC procedure, the cmRCT design was introduced [10]. This design exists of an ongoing cohort with the possibility for multiple RCTs over time. Each participant in this study design provides IC for the ongoing cohort and for potential future randomization, knowing that they might end up in an RCT. In each RCT though, only the patients randomly sampled into the intervention group and who accept this intervention, are asked to sign an additional IC.

When an RCT might be unethical or impossible, an observational cohort study is frequently performed. In a cohort study, participants, with varying exposures, are followed over a period of time and the outcome of interest is identified and treatment is allocated on the basis of what a patient needs. This follow-up can be retrospective or prospective and is sometimes combined [35]. No randomization or random sampling is applied in observational cohort studies. As a result, recruitment for observational cohort studies is often less selective and easier than for RCTs, for instance. An overview of the key differences and similarities of the study designs is given in Table 1.

Although RCTs maximize their internal validity [36], many RCTs have problems with generalizability (external validity) [37]. The study population is a poor representation of the real-world clinical population since many RCTs exclude clinically important subgroups who are hard to reach by clinicians or because of narrow entry criteria [38,39]. The goal is to determine evidence that the intervention may have benefits. With the introduction of pragmatic RCTs this limitation is often eliminated, but randomization in pragmatic trials are most often executed on groups of patients, introducing potentially statistical issues (confounding factors for groups) and losing power due to the group structure (heterogeneity of groups). Due to the reduction of selection bias in the Zelen designs and the cmRCT design, the study population is more diverse and representative [10,25,34]. Likewise, well-conducted observational cohort and case–control studies also include a broad representation of the population and have shown to provide the same level of internal validity as RCTs [13–15].

In the pre-consent conversation in RCTs, all patients are informed about the experimental treatment with 50% chance of not receiving the treatment. Although it is beneficial to provide all patients upfront information, clinical care often seems to deviate from this since patients are rarely told about treatment options that their physicians cannot guarantee to provide [40]. However, for many years it is considered ethically required to complete counseling before randomization [10]. Furthermore, randomization implies that patients are allocated and cannot choose between treatment options due to randomization, and therefore imposes ethical challenges.

Despite improvement in recruitment and external generalizability in Zelen’s single consent design, some ethical concerns have been raised [34,41]. The participants who are randomly assigned to the control group without giving IC, are unaware of being in a trial. This is resolved in Zelen’s double consent and cmRCT designs to receive IC from all the participants, although the participants in the control group for the cmRCT may not know at what moment in time the intervention is being implemented.

A cohort study design is often performed when a RCT would be impossible or unethical [42]. Compared with an RCT, observational studies have less ethical concerns since participants have 100% certainty of the treatment due to the absence of randomization. Treatment is allocated by caregiver. In some retrospective cohort studies (e.g., large cohorts where asking consent is not practicable, provided assurance that the patient’s privacy will not be compromised) IC for trial participation is not always necessary [43].

An advantage of an RCT is that randomization reduces bias and confounding, both known and unknown [44]. Common pitfalls in statistical analyses of RCTs are managing missing data, applying the wrong statistical test, misreporting of results of the statistical analysis, and ad hoc reporting of subgroup analysis [45]. Noncompliance issues with the new treatment are typically addressed by an intention-to-treat approach. In pragmatic trials, additional (bias) issues may arise due to the cluster structure of randomization and researchers have proposed to use a per-protocol analysis to be able to estimate the effect of treatment instead of the treatment assignment [46].

The possibility of a high number of participants rejecting the intervention when they were randomly sampled for the new treatment raises a potential bias issue with the cmRCT design. In the double Zelen design, crossing over in the double consent design reduces noncompliance with the participants allocated to standard of care, but eliminates the effect of randomization, and will lead to loss to follow-up if the experimental treatment is preferable. This biases the true treatment effect compared with the single consent design. To solve these potential bias issues of the cmRCT and double Zelen design, an intention-to-treat analysis is recommended. However, research regarding trials that used the double consent design showed that 28% of the trials did not use an intention-to-treat analysis [34]. When looking at cohort designs, the analysis of data from cohort studies is often complex due to the large numbers of possible confounding variables [35], but cohort designs may offer a wider variety of collected variables and may be more representative to the population of interest.

Calculation of an exact sample size for medical research is imperative [47,48] in particular for clinical trials. The sample size is typically calculated to achieve a minimal effect size that is considered clinically relevant. Not having the correct sample size is considered unethical [48]. Not including enough participants may risk a failure of the medical study to demonstrate the research goals. For instance, a true clinical benefit of the new treatment may not be detected due to lack-of-power of a too small sample size, while in prospective cohort studies specific associations may not be detected either. On the other hand, including too many participants may cause too much unnecessary burden on participants. This implies that too many participants may have been treated with a less beneficial treatment (depending on which treatment is clinically best), whereas a smaller trial could already have shown which treatment would be best. Also in cohorts, participant burden could be high due to the large number of variables that are being collected. Nevertheless, it should be noted that literature shows that studies are typically underpowered, because initial information is often too limited or too promising [49]. A disadvantage of clinical trials is that there are often difficulties to achieve the planned sample size due to lower than anticipated accrual rates [50]. In some cases, the clinical trial needs to be terminated before the final sample size can be achieved. In prospective cohort studies, participation rates are also declining [51,52]. However, within the cmRCT design, sample sizes may be attained more quickly due to the continuous inclusion of participants in the cohort.

Due to RCT being the gold standard in current clinical research practices, clinicians are familiar and conversant with the logistics of the study design. As a result, the study tasks are easier adapted into clinical practice and are carried out quicker. On the other hand, the majority of RCTs initially have a large sample size to achieve the intended treatment effect. This is accompanied by a long recruitment period and high costs [5,39]. It has been shown that only 55% of the studies reach their recruitment target [53]. In addition, 45% of trials had to request for extension to reach recruitment targets [53]. Due to the long recruitment period and follow-up time of a RCT, it takes years to implement a new treatment. As a result, RCTs are less able to quickly follow clinical innovations [5]. During RCTs, treatment guidelines are sometimes modified and it is likely for the new intervention to be overtaken by a competing intervention or by a technical improved version of itself [6]. Due to time and/or monetary constraints, RCTs may also rely on surrogate markers because of feasibility, and those do not always correlate well with the outcome of interest [5].

Within the cmRCT design, numbers required for RCTs are attained more quickly because of the continuous inclusion of participants in the cohort. Also, the expenses of organizing and carrying out RCTs can be greatly decreased by centralizing the data process. Furthermore, it allows for multiple interventions to be tested simultaneously or sequentially within the same cohort [10]. The challenges of this design are the significant organizational efforts required to manage a large ongoing cohort and simultaneously coordinating multiple RCTs. Additionally, researchers, clinicians and participants in the intervention group(s) may find it difficult to comply with the multiple consent procedures.

The logistics of the Zelen design could also be doubtful. The goal of the Zelen design was to improve clinicians’ and patients’ trial participation and to minimize bias [8]. A review of trials using the Zelen design by Adamson et al. [34] described two studies with different effects on recruitment rates. Due to the poor study qualities, these results were considered questionable [34].

Data collection in a prospective cohort could also be challenging, due to the time-consuming, expensive and complex nature caused by confounders, selection-bias and loss-to-follow-up [35,54]. A retrospective cohort can be used as study design to shorten the time to complete the study and is therefore cheaper in comparison to a prospective cohort. However, additional problems such as recall bias and missing data may be associated with the use of a retrospective cohort [35,54].

The objective is to include all eligible patients in a prospective cohort that is used for both clinical evidence of standard care and intervention study at the same time. From the eligible patients, a random sampling procedure is implemented that samples patients for the new treatment. By employing this random sampling technique, prior consent is not required, unlike the standard randomization process used in RCTs. As ‘random allocation of all’ and something that is ‘done’ to all patients in RCTs, randomization is typically understood to require prior consent of the patient. The decision to receive treatment A or B in the study lies with the randomization procedure. In the CIRSS, participants in the prospective cohort (i.e., agree to sharing data) are asked if they are willing to receive treatment B when they fall in the sample. If they decline treatment B, they will receive treatment A and remain part of the study. Obviously, patients that have not been sampled, do not need IC for not being sampled. Additionally, after the random sampling procedure, IC of all sampled patients is asked. They have the full right to decline the new treatment and receive standard care within or outside of the cohort. Subsequently, the nonsampled group receives information about standard care as is done in normal practice. An overview of the key features of our study design is provided in Table 2.

We anticipate that more patients will consent to participate in the CIRSS (due to a better recruitment strategy) in comparison to – for example – RCTs [10]. There is less exclusion criteria and all cohort participants will play a role in the analysis of the study, resembling the cmRCT design. Therefore, this would most likely lead to a better representation of the general population (external validation) [10,55]. Additionally, standard care in the nonsampled group is not affected by the possibility of the use of the new treatment option and will therefore better resemble routine standard care [10]. The sampling approach aligns very much with the randomization and therefore mimics a clinical trial (internal validity), although the sampled patients are aware of the experimental setting but the control group is not.

There are some major ethical advantages to this study design. First, by including the retrospective cohort as part of the control group, the intervention can be made available to more patients in the prospective cohort. This will lead to less burden to the patients (i.e., a smaller number prospective participants needed to reach the sample size) and a possible quicker implementation of new interventions or treatments. Second, the IC procedure is patient-centered, also contributing to as little stress as possible on the patient and caregiver. Additionally, randomly sampled patients are free to decline the intervention and have the opportunity to still take part in the study while receiving standard treatment. The sampling approach being fully random also gives every patient the same chance of being part of the intervention.

However, this CIRSS with historical controls poses a few ethical quandaries. The notion that the control group is unaware of the intervention (group) may be the leading issue, as discussed for Zelen’s single consent design, but 50 years has passed after his introduction of postrandomization and new insights, discoveries and advancements in the ethical domain may be needed. One of the compelling arguments for the uninformed control group is that this group offers a more accurate representation of reality (current clinical care). In addition, since not everyone will be sampled for the intervention, it can be more ethical to spare them from unneeded information and avoid putting them through possibly emotional distress from hearing they will not receive the intervention [28,29]. Additionally, this could avoid any potential strain on the caregiver–patient relationship, as reported in RCTs [8,29]. However, the discovery of the existence of an intervention (group) within the CIRSS by the patients in the prospective control group could lead to another ethical dilemma. The healthcare professional may find themselves in a difficult position as a result. Prior to the start of the study, this must be properly considered and its risk must be estimated.

In the CIRSS, participants who receive the experimental treatment are typically randomly sampled from the cohort and this splits the cohort in two representative sub cohorts (of possibly different size). Like in RCTs, this also addresses the issue of confounding, in particular when all sampled participants accept to undergo the experimental treatment. Selection bias in the CIRSS may occur when the sampled participants choose the control treatment, but this seems of a similar bias problem as the selection bias that comes from noncompliance [22]. Noncompliant participants are often a nonrandom group of participants. Additionally, the selection bias due to the refusals of the experimental treatment is compensated by an unbiased estimate of the clinical effect for control treatment obtained in the nonsampled participants. These nonsampled participants (and the historical controls) use the control treatment without knowing about the existence of other experimental treatments and are therefore unaffected by their knowledge of being present in a clinical trial. This provides an opportunity to investigate the potential of selection bias by comparing the clinical outcome of the refusals with clinical outcome of the nonsampled patients, but it requires some caution since historical controls may also introduce a temporal bias due to time-related factors (e.g., different patient population, seasonal effects, other innovations that affect the clinical outcome) compared with the nonsampled participants.

The possible biases obtained in CIRSS are not unsolvable, since they can be corrected by making use of causal inference [56,57] and hybrid sampling [58] approaches specifically developed for cohort studies. Indeed, the analysis of the CIRSS with historical controls will be analyzed as a single cohort study, with one treatment group and three different control groups (a proper control, a historical control, and a group of refusals). Potential imbalances and confounders may be corrected with inverse probability weighing approaches, assuming that the possible selection and temporal biases can be corrected with the collected participant information. Thus to correct for biases, a per-protocol analysis is proposed, following the advice for pragmatic trials from causal inference researchers [46], instead of an intention-to-treat analysis. In this respect, the CIRSS also deviates from the cmRCT, where an intention-to-treat analysis is suggested [59,60], although researchers are criticizing for this approach for cmRCTs as well [58,61]. More specifically, two weighting approaches are used to correct for possible biases that may occur due to the group of patients who are offered the new treatment B but receive current treatment A (either by choice or specific circumstances), referred to as the group of refusals in CIRSS and the retrospective and prospective nature of the study, referred to as a possible temporal bias or cohort effect. Weights (based on propensity score analyses using the observed information on participants unrelated to clinical outcomes) are created to correct for imbalances in patient characteristics between the four treatment groups. First a propensity score is determined between the retrospective and prospective patients to eliminate the cohort effect from temporal differences. This will lead to an inverse probability weight for each patient in the full study. Then secondly, the calculated weights for the group of refusals are corrected by the observed patient information from the nonsampled patients in the CIRSS (i.e., the proper control group) using inverse weighting probability that is common in the field of hybrid sampling [62]. This makes the group of refusals representative in characteristics to the nonsampled control group (creating one representative control group). After establishing the weights for all patient in the full cohort study (possibly rescaled to make the weights add up to the size of the full cohort study), the clinical outcomes can be analyzed by standard weighted (logistic) regression approaches where the intervention is compared with the combined control group. Similar causal inference analyses approaches (instrumental variables) have been described for cmRCT designs, where the same type of selection may occur by the design [22,59].

Clinical trials (both superiority and noninferiority trials) require a proper sample size calculation for a well-defined effect size, since the goal is to demonstrate a clinically relevant benefit of the new treatment. Similarly, CIRSS requires a sample size calculation when the study must empirically establish a relevant benefit for e.g., a new treatment compared with an existing treatment. The common approach is that the sample size calculation follows the proposed statistical analysis for the primary outcome. However, since the statistical analysis for CIRSS is rather complex due to the proposed data analytic approaches to accommodate potential biases. Thus, an exact sample size calculation for CIRSS, when a well-defined clinical effect is provided, is not so straightforward and requires additional research. Therefore, sample size calculation should at least be conducted under certain ideal situations, where it is assumed that selection biases are not present. The sample size calculation would then follow existing approaches for sample size calculations. Note that literature shows that covariate adjusted analyses (like the causal inference approaches proposed) in RTC’s may also lead to an increase in power [63].

Well-conducted observational cohort and case–control studies can provide the same level of internal validity as RCTs [13–15]. Furthermore, by adding a retrospective cohort to reinforce the prospective cohort, it reduces the number of prospective patients who need to receive standard care. This relatively increases the number of patients who could benefit from the new treatment and fasten the enrolment of the required number of patients. Additionally, it could lead to a faster implementation of new treatments, preventing the new intervention to be overtaken by a competing intervention of by a technically improved version of itself. Moreover, there could be financial benefits from higher effectiveness, which are especially interesting for specific patient populations and therapies, such as expensive interventions. An applied example of this study design, including operational aspects, including infrastructure, data integration and clinical implementation, are detailed in the NIEM-II study protocol [64]. Prior to the start of a study with CIRSS design, it is recommended to inform and train care givers about the IC procedure (approaching every potential participant, separate counseling for intervention group and control group and ensure that the control group remains unaware of the intervention).