Why clinical trials may not help patients make treatment decisions: results from focus group discussions with 22 patients
Publication: Journal of Comparative Effectiveness Research
Abstract
Aim: Despite broad interest in advancing personalized medicine, most evidence is currently derived from average results of clinical trials that may obscure heterogeneity of trial participants. Little is known currently about how patients view heterogeneity in trials and whether they can participate in methodological discussions about this concept. Materials & methods: In structured discussions with three focus groups involving 22 participants, we assessed how representatives of patient communities have used research to guide individual treatment decisions. Discussion themes were organized into a framework describing patient decision-making in four steps: decisions patients make in the course of care; information used to make decisions; sources for information; and quality of information. Results/conclusion: Patients prioritize information that reflects their own characteristics, preferences and values. They struggle applying clinical research to their own case.
Current approaches to evidence-based medicine are based upon a fundamental flaw of logic, whereby it is presumed clinicians can draw inferences for individuals based on average group effects from clinical research. Using the overall results of a randomized controlled trial (RCT) to predict the outcomes for an individual patient under alternative treatments remains an imperfect approach because individual patients have many characteristics that might affect the likelihood of an outcome, including both the benefits and risks of therapy [1]. There has been growing interest in understanding how a treatment’s effect can vary across patients – a concept described as heterogeneity of treatment effect (HTE), which is central to the research agenda for both personalized (or precision) medicine and comparative effectiveness research [2,3].
If evidence-based medicine is to become more personalized and patient-centered, patients who participate in research must be recognized as having multiple characteristics that are correlated with likelihood of response to treatment. To accomplish this, we have proposed an approach to RCT design and analysis that prioritizes predictive approaches to HTE, where the main goal is to develop models that can be used to predict which of two or more treatments will be better for a particular individual, given their unique characteristics [1,4–7]. Others have given methodological attention to applying HTE in research design, analysis and interpretation [8–13].
The concept of HTE has the potential to improve the patient-centeredness of clinical trial data, but currently little is known about how patients themselves view this concept and whether they can participate in methodological discussions about it. The field of HTE is still in its infancy, and the perspectives of key stakeholders – in particular patients and patient advocates – are essential to determine best practices for better personalizing evidence from RCTs. Patient perspectives should inform best practices in HTE, from methodological development to how risk models and predictions tools are implemented in clinical practice.
We therefore aimed to learn how representatives of several patient communities have used research to guide their own individual decision making around treatment. We also recruited and prepared a small number of patient representatives to participate in a 2018 National Academy of Medicine (NAM) symposium on this topic. The goal of the symposium was to establish an agenda for aligning patient-centered research with patient-centered care.
Materials & methods
We held webinar-enabled group discussions from April to August 2017 with representatives of three patient research networks. The networks included ARthritis Partnership with Comparative Effectiveness Researchers (AR-PoWER); the Health eHeart Alliance; and iConquerMS. Each network is or was funded through the Patient-Centered Outcomes Research Institute (PCORI) to improve patient involvement in development of research data. Participants in our discussions were recruited by leaders of the patient networks. Each discussion consisted of approximately 45 min of nontechnical presentation on the topic of HTE and approximately 45 min of guided group discussion (facilitated by TWC and DMK). The presentation and discussion covered the following five topics [14]: experience using clinical information in decisions; confidence applying average results to their decisions; factors influencing treatment decisions; understanding of the concept of HTE and useful, relevant data; and questions about applying HTE to clinical decisions.
Discussions were audio-recorded and transcribed. Transcripts were deidentified and audio recordings and identifiers were destroyed. Each transcript was assessed with a codebook derived deductively from our structured interview guide, then codes were modified inductively until saturated themes could be identified and organized into a framework describing patient decision-making. The analytic framework identifies four steps: decisions patients make in the course of their care; information they use to make those decisions; sources for that information; and quality of the information. Transcripts were coded using NVivo software [15] to relate comments made by participants to the analytic framework. Major themes in each stage are presented in the results.
We then identified patients to report on major themes from our discussions at the 2018 NAM symposium on HTE [16]. We used two criteria to select participants: the frequency with which each participant was able to articulate clear ideas during the webinars and the degree to which each participant had articulated key themes during focus group discussions. The goal of the symposium was to engage a wide set of stakeholders – including patients, researchers, physicians, funders, payers, regulators and others – in setting a methodological research agenda on HTE.
This work was deemed exempt from review by the Tufts Medical Center and Tufts University Health Sciences Institutional Review Board.
Results
In this section we report descriptive information on focus group participants, focus group discussion themes and logic model, and the NAM Symposium.
Focus group participants
A total of 22 patient representatives participated in three webinars. Webinar One included seven participants affiliated with AR-PoWER whose medical history included psoriatic arthritis, hip revisions, hip replacements and osteoarthritis. Webinar Two included six participants affiliated with the Health eHeart Alliance, a network of patients who expressed interest in heart-related research as of the time of signing up for membership. Webinar Three included nine participants, of which eight were affiliated with iConquerMS and one with the Health eHeart Alliance. Members of iConquerMS who participated in our focus group live with multiple sclerosis. Fifteen participants (68%) were female. Participants lived in 15 states across USA, including New England, the Mid-Atlantic, the Midwest and the West Coast.
Focus group discussion themes & logic model
Figure 1 depicts themes emerging in each of the analytic framework categories. The diagram sorts discussion themes into three categories (sources, information and decisions) and reflects a distinction between themes relating to peer reviewed information (in shaded boxes) and information from other sources (in unshaded boxes). Shaded boxes depict themes related to peer-reviewed research, the focus of our agenda-setting work for HTE. These include information about outcomes of treatment and the relationship between group evidence and the individual’s clinical case or characteristics. They include sources of information that are likely to be informed by peer reviewed evidence, including (some) online research, clinician judgment and some information presented by patient organizations. The unshaded boxes depict themes that are not always or usually informed by-peer reviewed research: information sourced from family members, friends and media and information relating to patient values and preferences or to their resources for medical care.
Figure 1. Major themes in patient discussions of individual clinical decision making.
Patient discussions were coded and organized into a framework describing patient decision-making for treatment decisions. The analytic framework identified four major steps: decisions patients make in the course of their care; information they use to make those decisions; sources they go to for that information; and quality of the information. Boxes depict themes emerging in each of the analytic framework steps. Grey-shaded boxes depict themes related to peer-reviewed research, including information about outcomes of treatment and the relationship between group evidence and the individual’s clinical case or characteristics. The unshaded boxes depict themes that are not always or usually informed by peer reviewed research: information sourced from family members, friends and media and information relating to patient values and preferences or to their resources for medical care.
Treatment decisions
Participants described three types of treatment decisions with which they had experience: whether they should start a new treatment; whether they should start one treatment or another (e.g., medication A vs medication B; medication vs surgical procedure); and whether they should continue with their current treatment.
One participant’s experience illustrates the dilemma some patients face concerning the first type of decision, whether and when to initiate treatment: “Every doctor I went to said, ‘You need a hip replacement, but you are too young.’ And I was like, ‘Well, so then what is next?’ And nobody had an answer.”
A participant with multiple sclerosis described her challenge with the second type of decision, when choosing among two medication options presented by her doctor: “The results from my MRI [were not] typical to what he had seen before, so he really did not have any basis to say, ‘Okay, we’re going to put you on [prescription A], or we’re going to put you on [prescription B].’ He just kind of went with, ‘Let’s try this.’” Another participant with MS did her own research when making a similar choice: “I spent a lot of time researching, even though [my doctor] gave me two options. Part of the reason I did go initially with [the recommended treatment option] was the research out there, in the studies that indicated that it was effective, more so than [the other treatment option].” Another participant said,
[T]he doctor in the hospital asked me […] ‘Okay, which blood thinner do you want to take? Here are the advantages of this one. Here are the advantages of this one.’ […] I said, ‘Well, which one is better for me in my situation?’ And he gave me some averages. […] I really do not feel comfortable choosing an average, and in the end I did. I just went with what works best for most people.
Another participant described a third type of decision when trying to figure out whether a medication is working and if it should be stopped: “Outside of the medications that help relieve the actual MS symptoms… I have a really hard time saying that a certain medication is helping or not helping.”
Information that matters
Participants described the factors they consider when making treatment decisions. These factors can be grouped in three general themes, each of which is represented by questions a patient might ask over the course of their care.
The first theme deals with the extent to which evidence about a treatment’s efficacy can be assumed to apply to an individual. It relates to patient-level characteristics, including demographics, genetics and clinical factors, and thus are relevant to HTE. When considering the case of “Jane,” a hypothetical patient presented in the technical portion of the webinar (see Box 1), participants discussed a number of Jane’s supposed characteristics, including her age, gender, BMI, genetics, comorbidities, lifestyle, medical history and family history. Participants recognized, however, that all of these characteristics are not always addressed in the results of trial and other clinical research. One participant commented, “First you want to know, in terms of the efficacy based on who you are, what is going to work best.” Another said,
We do not know, does the medicine work differently based on gender? Or did the medicine work better in age? Or did the medicine work better in terms of BMI? Or is there a genetic component? […] disease is so multifactorial […] I’d want to know more information, like which treatment works best for patients like me?
The second category includes patients’ values and preferences for care, such as the urgency of treatment, burden of treatment and quality of life. The third relates to a patient’s resources relative to their access to care, including their insurance coverage and the cost of treatment. One participant said, “But what if you are on disability, like I am, and you cannot afford the coverage?” Another said,
It did become kind of a case of, ‘Do you mind giving yourself injections every single day or every other day or once a week?’ And I had no other information on which one was actually best. I did not like injections at all, so I figured, ‘Okay, once a week, let that be it.
Consider a scenario: Jane had a stroke one year ago. Jane is a 45 year old female, with no other major problems like diabetes or high blood pressure. Jane and her doctor are deciding on the best treatment to avoid a future stroke. Research says that on average, treatment A is better than treatment B. Jane wants to know, “Should I start Treatment A?; Am I similar to the “average” person from the research study?; and Which treatment works best in patients like me?”
A third category relates to whether the patient has resources that will support a patient’s access to care. One participant, for instance, described making decisions on the basis of their health insurance coverage: “for me, it came down to my insurance company, and what it was going to cost me out of pocket.”
Sources of information
Participants also mentioned the sources they use to gather information, including their primary care physicians, specialists, patient organizations, online research, family, friends and social media. A key theme that emerged in this part of the discussion was the importance of going to trusted, high quality sources of information to collect the information they need. Physicians, especially primary care doctors, were seen as reliable sources of information. One participant said, “If that is what my doctor recommended to me, then I would probably do it.” Another said,
If it was the primary care doctor, I would take his recommendation and follow it because he knows me in all my quirkiness. I would probably research it on the internet, but I would still follow his advice because [he is] the one that is familiar with you.
Even when using additional sources of information, they still consider their doctor to be the best source. Another participant expressed similar thoughts: “you […] have to follow your doctor’s advice because they are the ones in the know.” One, however, expressed frustration with needing to rely so heavily on his physician’s advice, lamenting the absence of information that the layperson can use. This participant said,
I find that most of the research that is available is incomprehensible to the layperson. I had a very difficult time deciphering what is actually being said, and so I get a little overwhelmed and then I just sort of defer to what my doctor is telling me.
Quality of information
Participants were aware of and often expressed frustration with the absence of information in research or other sources of information about patients who share their characteristics. Some described the frustration with research findings based on exclusions of patients like them – their youthful or older age, minority race or ethnicity, or gender – from trials. One participant said, “Really, most studies unless you are taking all comers in a study, and identify that, you have already preselected a population, which will likely differ from the population at large, anyway.” Another participant said,
Now, if you use the case of the woman who had the stroke, basically she had a stroke, but she is a healthy stroke patient, since she had no other problems. And thus, how does she compare to a diabetic with a stroke, another patient with diabetic hypertension, and other risk factors, previous MI, etc.?
Others understood that patients with their characteristics may have been included in research, but findings couched in averages might obscure how those patients do on treatment. One participant said, “unless you power it for subgroup analyses within those, it can be very problematic.”
Many described working around these problems by taking strategies that are not ideal: some visited multiple providers until they could understand how existing might be applied to their case. For instance, the same participant who was told she was too young for a hip replacement kept seeing providers until she found one who could explain why her youthful age caused concern for a hip replacement. After seeing other providers, she had a conversation with one who could answer this question:
And I did ask, Well, what was the main contraindication?” And they said, “Well, you might need a revision.” And I was like, “Well, I could get hit by a bus tomorrow.” … until I got to a doctor who knew colleagues who did revisions, could I really have an open discussion, and then he said, “You need a hip replacement.” I said, “Yes, I know.” And he is like, “What do you want to do? Oh, and he put it back in my hands, and we talked about it together, then I could make my decision.
Others took an approach that mirrors the N-of-1 experimental treatment design, treating their own course of care as experimental until a satisfactory treatment was found:
I [was] just going along with medication that I was taking that was keeping me comfortable, but I really had no quality of life. And I did a lot of research before I decided to go on my biologic… my doctor just came right out, and asked me before I went on it, Do you want to continue with status quo?” And I really had to think about that. I thought, “No, I want a chance for my life to be better. So, I took the plunge.
Other participants described delaying decisions when there was not enough information, a choice that can cause friction in healthcare settings with timelines to keep:
when I am in the hospital, maybe it is a matter of time, or a doctor that does not know me, but I felt like he just wanted me to answer real quick, and it just makes me fear for patients who do not know to ask questions […] And I had to say, I need some time. I know you are on a tight schedule, but I am not ready to tell you what I need yet. I need to do some research. And he kind of looked at me like I was crazy.
The NAM symposium
Two focus group participants were invited to represent patient views at the 2018 NAM symposium on HTE. The participants were a clinician with over 30 years of experience applying trial data in a clinical setting and a PhD researcher with experience developing trial protocols, recruiting participants and interpreting results. While these are not typical patients, their principal activities today are involvement in leadership in their patient communities. Their background in clinical medicine and research only enhanced their understanding of the methodological issues involved in advancing use of HTE.
The participants presented their personal stories about using clinical research and other sources of information to guide their own care decisions, and then summarized key themes from the focus group discussions. Their participation was well received by the clinicians, researchers and industry and policy leaders in attendance. A conference evaluation survey contained the question, “What was the most valuable part or content of the conference?” Of respondents who completed this free-text field (n = 23), seven mentioned their appreciation of including different viewpoints from a variety of stakeholders.
Conclusion
We found that patients prioritize using personalized information in treatment decisions, such as their own characteristics, preferences and values. We also found that the patient perspective is both relevant and motivational in the formation of a research agenda for HTE. Results from three focus groups suggest that patients shared insight that is relevant to the challenges of applying clinical information in decisions about their own care. The discussion we held returned frequently to the dilemma of applying average results from clinical research to the individual. Patient advocates who participated in the discussions described many ways in which they struggle with the dilemma of how to apply clinical research to their own care.
This work demonstrated that patient-centered care goes well beyond questions of preferences and access to care and includes concerns about the patient-centeredness of evidence. An overarching theme in the discussions included the uncertain relationship between average results described in clinical research and participants’ own individual cases. Participants were concerned that trials exclude patients that represented their unique case, and that findings couched in averages obscure how patients like them might fare under treatment alternatives. These findings support the notion that patients are prepared to participate in discussions about HTE and to make valued contributions to shaping an agenda for HTE methods, in spite of the technical complexity of the research.
The patient perspective was also motivational to clinical, policy and research professionals who work on this topic. At the NAM symposium, presentations and discussions referred to the patient presentations frequently and evaluations reflected an appreciation of their involvement. Invited speakers and discussants addressed each step of an agenda for advancing HTE in clinical care, ranging from outstanding methodological questions to best practices for implementing models at the point of care and on payment policies that support the effective targeting of treatments. Patient stakeholder comments were incorporated and solicited in many of these discussions. The inclusion of patient perspectives along each step of translation of HTE concepts into actual practice is needed to ensure that new tools are developed with the end users in mind.
Our work indicates that patients want more precise answers about how a given treatment is likely to work for them, given their unique individual characteristics. A one-size-fits-all approach to treating a medical condition based on average responses from clinical trials is increasingly recognized as suboptimal by researchers and clinicians; patients included in this work share this frustration. Achieving the goal of better individualizing evidence for clinical decision making in actual practice will require deliberate coordination among a wide range of stakeholders with the ultimate goal of serving the patient. It is important for leaders in the field to come up with effective ways to communicate concepts of HTE that will help clinicians, patients and other stakeholders understand the nuances of the work.
Understanding HTE can transform medical care by increasing the likelihood that patients will benefit from the treatments that are offered to them and by contributing to the goal of avoiding harmful or wasteful treatment choices. The tailoring of treatments to individuals based upon heterogeneity of their clinical characteristics and their personal preferences is a priority for patients. To ensure that available information is most useful for the individual patient, patients should be engaged in all phases of research efforts to move past ‘one-size-fits-all’ approaches to decision making.
•
We assessed how patients use research to guide treatment decisions.
•
Participants described making decisions about when to start treatment, which treatment to start and when to stop treatment.
•
Patients understand that effects of treatment may be heterogeneous.
•
Patients understand that their individual characteristics and other factors may influence treatment effects.
•
Patients prioritize information that reflects their own characteristics, preferences and values, but struggle applying peer-reviewed research to their own case.
Acknowledgments
The authors wish to thank members of the AR-PoWER, Health e-Heart and iConquerMS Patient Powered Research Networks for their participation in this research. They also wish to thank B Nowell for advising on the design and implementation of this project.
Financial & competing interests disclosure
Support for this publication was provided by the Patient-Centered Outcomes Research Institute® (PCORI®), through two awards: a Patient-Centered Outcomes Research Institute (PCORI) Eugene Washington PCORI Engagement Award (1900-TMC) and the Predictive Analytics Resource Center (SA.Tufts.PARC.OCSCO.2018.01.25). The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.
No writing assistance was utilized in the production of this manuscript.
Ethical conduct of research
The authors state that they have obtained appropriate institutional review board approval or have followed the principles outlined in the Declaration of Helsinki for all human or animal experimental investigations. In addition, for investigations involving human subjects, informed consent has been obtained from the participants involved.
References
1.
Kent DM, Steyerberg E, van Klaveren D. Personalized evidence-based medicine: predictive approaches to heterogeneous treatment effects. BMJ 363, k4245 (2018).
2.
Kovalchik SA, Varadhan R, Weiss CO. Assessing heterogeneity of treatment effect in a clinical trial with the proportional interactions model. Stat. Med. 32(28), 4906–4923 (2013).
3.
Varadhan R, Segal JB, Boyd CM, Wu AW, Weiss CO. A framework for the analysis of heterogeneity of treatment effect in patient-centered outcomes research. J. Clin. Epidemiol. 66(8), 818–825 (2013).
4.
Kent DM, Rothwell PM, Ioannidis JP, Altman DG, Hayward RA. Assessing and reporting heterogeneity in treatment effects in clinical trials: a proposal. Trials 11, 85 (2010).
5.
Kent DM, Paulus JK, van Klaveren D et al. The predictive approaches to treatment effect heterogeneity (PATH) statement. Ann. Intern. Med. 172, 35–45 (2020).
6.
Rothwell PM. Can overall results of clinical trials be applied to all patients? Lancet 345(8965), 1616–1619 (1995).
7.
Kravitz RL, Duan N, Braslow J. Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages. Milbank Q. 82(4), 661–687 (2004).
8.
Kent DM, Hayward RA. Limitations of applying summary results of clinical trials to individual patients: the need for risk stratification. JAMA 298(10), 1209–1212 (2007).
9.
Rothwell PM, Mehta Z, Howard SC, Gutnikov SA, Warlow CP. Treating individuals 3: from subgroups to individuals: general principles and the example of carotid endarterectomy. Lancet 365(9455), 256–265 (2005).
10.
Kovalchik SA, Tammemagi M, Berg CD et al. Targeting of low-dose CT screening according to the risk of lung-cancer death. N. Engl. J. Med. 369(3), 245–254 (2013).
11.
Yeh RW, Secemsky EA, Kereiakes DJ et al. Development and validation of a prediction rule for benefit and harm of dual antiplatelet therapy beyond 1 year after percutaneous coronary intervention. JAMA 315(16), 1735–1749 (2016).
12.
Claggett B, Tian L, Castagno D, Wei LJ. Treatment selections using risk–benefit profiles based on data from comparative randomized clinical trials with multiple endpoints. Biostatistics 16(1), 60–72 (2015).
13.
Farooq V, Van Klaveren D, Steyerberg EW et al. Anatomical and clinical characteristics to guide decision making between coronary artery bypass surgery and percutaneous coronary intervention for individual patients: development and validation of SYNTAX score II. Lancet 381(9867), 639–650 (2013).
14.
Patient Centered Outcomes Research Institute. Conversations about using group data to treat individual. https://www.pcori.org/sites/default/files/Webinar-slides-April2017.pdf
15.
Nvivo. Qualitative Data Analysis Software. QSR International Pty Ltd. (10), (2012). www.qsrinternational.com/nvivo-qualitative-data-analysis-software/home
16.
Kent D, Paulus J, Ahmed M, Whicher D (Eds). Caring for the Individual Patient: Understanding Heterogeneous Treatment Effects. National Academy of Medicine, WA, USA (2019).
Information & Authors
Information
Published In
Pages: 651 - 658
PubMed: 32633549
Copyright
© 2020 Future Medicine Ltd.
History
Received: 17 February 2020
Accepted: 21 April 2020
Published online: 7 July 2020
Keywords:
Topics
Authors
Metrics & Citations
Metrics
Article Usage
Article usage data only available from February 2023. Historical article usage data, showing the number of article downloads, is available upon request.
Citations
How to Cite
Why clinical trials may not help patients make treatment decisions: results from focus group discussions with 22 patients. (2020) Journal of Comparative Effectiveness Research. DOI: 10.2217/cer-2020-0033
Export citation
Select the citation format you wish to export for this article or chapter.