Optimizing electronic capture of patient-reported outcome measures in oncology clinical trials: lessons learned from a qualitative study
Publication: Journal of Comparative Effectiveness Research
Abstract
Aim: To understand the impact of anticancer treatment on oncology patients’ ability to use electronic solutions for completing patient-reported outcomes (ePRO). Materials & methods: Semi-structured interviews were conducted with seven individuals who had experienced a cancer diagnosis and treatment. Results: Participants reported that the following would impact the ability to interact with an ePRO solution: peripheral neuropathy of the hands (4/7), fatigue and/or concentration and memory issues (6/7), where they are in a treatment cycle (5/7). Approaches to improve usability included: larger, well-spaced buttons to deal with finger numbness, the ability to pause a survey and complete at a later point and presenting the recall period with every question to reduce reliance on memory. Conclusion: Symptoms associated with cancers and anticancer treatments can impact the use of technologies. The recommendations for optimizing the electronic implementation of patient-reported outcome instruments in this population provides the potential to improve data quality in oncology trials and places patient needs at the forefront to ensure ‘fit-for-purpose’ solutions.
Patient-reported outcomes (PROs) are now routinely used as endpoints in oncology clinical trials to elicit the patient perspective, particularly related to symptoms and health-related quality of life (HRQoL) [1] and for supplementing objective clinical endpoints. The importance of utilizing PROs in drug development has also been emphasized by regulatory agencies such as the US FDA [2], and they are increasingly being used in labeling claims for oncology treatments. Between 2012 and 2016, 49 oncology drugs were approved by the FDA and/or EMA for a total of 64 indications; with 45 indications (70.3%) including PRO data in the regulatory submission dossier. Twenty-one of the 45 indications included PRO data on the labeling approved by the EMA [3]. However, both the FDA and EMA identified missing PRO data as problematic for the interpretation of efficacy [3], and the EMA specifically recognizes that collecting PROs from patients with advanced and progressive disease such as cancer, can be more problematic and challenging due to deteriorating health and/or cognitive difficulties [4], which can lead to increased missing data.
With the increasing use of PROs as trial endpoints, comes an increasing drive to capture PROs electronically (ePROs) due to the benefits offered. Electronic collection enables improved data quality due to inbuilt branching (eliminating conflicting entries) and edit checks to eliminate invalid or missing responses, and the ability for data to be reported at defined timepoints outside the clinic environment [5]. When capturing PROs outside of the clinic, electronic capture can offer improved data integrity, including transparent attributability and contemporaneousness due to preprogrammed completion windows, reminders and time/date stamped entries, reducing reporting gaps and enabling more sensitive detection of change. This is particularly relevant in oncology where health status is dynamic throughout treatment and recall of health status is often influenced by current status [6]. Thus, only measuring outcomes at site visits that occur at the beginning of a treatment cycle can be misleading; for example, in relation to side effects, this assessment timepoint is likely to occur after recovery from the previous cycle and before the expected time of greatest toxicity of the next cycle of treatment, [7] and therefore not provide an accurate reflection of the patient perspective. Furthermore, drug administration schedules differ (e.g., intravenous [IV] vs oral treatment) and so this can make comparisons between different classes of drugs within one trial challenging. Thus, measuring PROs at home offers an opportunity to capture more comprehensive and accurate patient perspective data.
Furthermore, for studies with at-home PRO assessments (e.g., daily ediaries and surveys/questionnaires) as primary or secondary end points that are going to be used for labeling claims, the FDA request evidence to demonstrate how the timing of measures (or events) specified in the protocol has been enforced [6]. ePRO solutions provide a time-stamped audit trail to corroborate this, offering a solution against the infamous ‘parking lot effect’ (completion of paper diaries in the parking lot just prior to a clinic visit) [5].
However, the usability of electronic solutions to capture PROs may be influenced uniquely by the impact of the cancer and its treatment. For example, chemotherapy-induced peripheral neuropathy (CIPN) has been reported in the literature, with up to 90% of oncology patients experiencing it at some stage during or after treatment [8,9]. Furthermore, CIPN can persist and some individuals may not recover full dexterity [8,10]. Cancer-related fatigue is also common, can be persistent, and experienced as physical, emotional and/or cognitive tiredness or exhaustion [11]. Cancer Research UK [12] states that it can affect between 25–99% of those undergoing treatment. Individuals can also experience cognitive impairments in multiple domains, including verbal memory, visual spatial skills, attention and psychomotor function (such as fine motor dexterity) [13]. Studies have shown that this is a highly prevalent impact of the disease and treatment, with up to 30% of people experiencing these problems prior to treatment, up to 75% during treatment, and up to 35% experiencing persistent cognitive difficulties post-treatment [14]. Such cognitive impairments resulting from anticancer treatments have been colloquially referred to as ‘chemobrain’ and more recently ‘cancer-associated cognitive change’ or ‘cancer-treatment-associated cognitive change’, as it is not confined to chemotherapy [14–18]. Such symptoms can pose particular challenges for clinical trials where ePROs are completed outside the clinic environment and site-staff are not present to provide guidance and ensure completion.
Given the wealth of benefits offered by electronic measurement, it is imperative to comprehensively understand whether device and software functionality for collecting PROs electronically are ‘fit-for-purpose’ in this disease population. While there have been studies conducted that look at the feasibility and acceptability of electronic recording of PROs [19], to our knowledge, there has not been a qualitative study in oncology patients exploring how the disease and symptoms associated with its treatment can impact willingness and ability to use technology for recording ePROs. Qualitative research offers a unique opportunity to understand patient experience in greater depth, which is especially pertinent in new and emerging research areas, and findings can be harnessed to inform future quantitative study designs. The objectives of the current study were to conduct semi-structured interviews to: 1) qualitatively investigate the challenges experienced by oncology patients that effect electronic device use for recording PROs; 2) use this qualitative data to inform recommendations for optimising implementation of ePRO instruments in oncology clinical trials.
Materials & methods
This research adheres to the recommendations outlined in the Standards for Reporting Qualitative Research [20].
The interviews detailed in this manuscript were conducted in the context of a broader usability testing project carried out in the UK. The overarching aim of this project was to investigate specific user needs and usability considerations of oncology patients who participate in clinical trials when using Signant Health’s electronic Clinical Outcome Assessment (eCOA) software solutions on two devices (a tablet and a provisioned mobile device), to understand how well patients can interact with the software solutions (ediaries, questionnaires and different response scale types) and what problems they experience, subsequently facilitating opportunities to make the solutions more user-friendly for this patient population. Here, we only report on the themes related to how the disease and symptoms associated with its treatment impact technology use. As this project formed a software usability testing evaluation, ethical approval was not required. All participants provided informed consent and signed a confidentiality form.
Sample
Participants were recruited using an external recruitment agency and underwent a telephone screening. To be eligible to participate, individuals were required to have a cancer diagnosis and be undergoing or have undergone treatment, and provide informed consent. Participants were also required to be primary English speakers with at least a basic level of digital experience (assessed via a screening questionnaire). If individuals had participated in any market research in the last 6 months or worked in web design, IT, market research, media or broadcasting, then they were not eligible to participate. Participants were offered a monetary incentive for participation. We aimed to recruit eight participants (which was deemed appropriate to meet the needs of usability testing) aged 40–70 years (with representation in the following age groups: 40–55, 55–65 and 65–70), a balance of males and females, and a range of oncology types and stages of treatment. The intention was that this would increase the likelihood that participants interviewed would provide a range of views and experiences.
Interview methods
Prior to conducting the interviews, the study research team, composed of user experience and eCOA specialists, developed a semi-structured interview guide. This detailed the most important questions to address during the interviews, while allowing participants to elaborate on key issues of specific relevance to their experience.
One-to-one semi-structured interviews based on this predefined interview guide were carried out in February 2019 at a UX research lab in London, UK by a qualitative researcher independent from the solution developer commissioning this research. Interviews were 90 min in duration and were video recorded. During the interview sessions, participants were given a tablet and mobile device containing Signant Health’s TrialMax eCOA software solution and asked to interact with the technology as if they were participating in a clinical trial. Sessions were streamed to an observation room where field notes were also taken. No repeat interviews were conducted.
Qualitative data analysis
Video recordings of the interviews were analyzed by an eCOA scientific researcher (FM) who was independent from those who designed or conducted the interviews. Interview recordings underwent manual thematic analysis following the recommended phases: familiarization, generating initial codes, searching for themes, reviewing themes, and defining and naming themes [21,22]. To increase reflexivity, members of the wider scientific team (all possessing research qualifications) contributed to analysis by providing review and sense checking of the derived themes.
Results
Participant characteristics
See Table 1 for participant characteristics.
| Characteristics | n (7) |
|---|---|
| Sex – Male – Female | 3 4 |
| Age (years) – Mean – Range | 54.75 30–68 |
| Highest education level achieved – Left education at 16 years old – Left education at 18 years old – Undergraduate – Postgraduate | 2 2 2 1 |
| Employment status – Part time – Full time – Retired | 3 3 1 |
| Oncology diagnosis – Breast† – Prostate – Colon/bowel | 3 2 2 |
| Digital access – Smartphone only – Tablet only – Both | 4 0 3 |
| Digital experience – Intermediate – Experienced | 3 4 |
†
Including one metastatic breast cancer.
Eight participants took part in the interviews, one of whom was excluded from analysis as during the interview they reported not yet having started treatment and so the information they provided was not applicable to the research question (their demographic information is not reported here). Thus, the current study analysis included seven individuals (4 females and 3 males) aged 30–68 years with a cancer diagnosis who had undergone or were currently undergoing treatment. Participants had been diagnosed with a range of cancer types and their time since diagnosis ranged from 3 months to 8 years. All participants reported having access to either a smartphone, tablet or both, and all described themselves as having at least an ‘intermediate’ experience with technology.
Emerging themes
Overall, participants had a positive attitude toward the use of technology to complete assessments and expressed (unprompted) a preference for electronic over paper completion. Furthermore, mobile devices were reported as a helpful tool to stay connected when confined to the home during periods of illness and/or treatment.
Themes that emerged in analyses with implications for device use were: peripheral neuropathy of the hands, tiredness and fatigue, difficulties maintaining focus and concentration, and varying health status based on timepoint in the treatment cycle. See Tables 2 and 3 for supporting quotes.
| Emerging themes impacting device use | Supporting quotes |
|---|---|
| Peripheral neuropathy of the hands | “I kind of had no feeling in my hands…it was a numbness and a stiffness” “When you've just got no energy and you’re starting to get the mobility problems…more the numbness, not so much the pain…sometimes you just cannot use your hands…that can last for a day or two…it wouldn’t be a problem though…it would still be possible…you’re not so bad that there is nothing you can do that whole 24 hour period” “I had numbness, I still have numbness…At the beginning, I think the very first session, it did affect me, but I do have numbness now still…but it doesn’t affect my holding” “The numbness in the fingers makes precise actions very difficult and frustrating.” “I’d find all precise interactions very difficult and frustrating, if not impossible” “Even if you do have problems, you can still tap” “I prefer to use a phone…sometimes you don’t want to do all the writing and it’s more easy to tap” |
| Fatigue and/or concentration issues | “Fatigue hits you out of nowhere” “You are so fatigued” “I just felt very tired, exhausted…I don’t think it would be quite a case of switching off, but it would certainly be a significant reduction” “Since I’ve had cancer, my reactions and things have slowed down…it slows you down…it’s the illness and the treatment” “Sometimes your concentration is not too good” “You are slower, because you can ask me questions and it might take me a second or two to click in to respond to you…at the first or second session” “I did find it difficult to concentrate cause it does make you very tired…it probably reduced my interaction with the phone” “The reading and concentration was impacted by the tiredness…it was pretty much throughout [the day]…pretty tired most of the time” “Going back to the tiredness…I am mentally more sluggish in the evening, now I don’t know if that’s age or medication…or just tiredness…thinking of the timing of this, that’s why maybe it’s better to have the beginning and the end of the day, because some people are night people and they are better at night” |
| Timepoint in treatment cycle | “On the first session I was very ill, you have it every two weeks the chemo, but after a certain period you get to adapt to it…I could still text I just couldn’t really hold a conversation” “Days 3–6 were my ‘dark’ days and I did not leave my bed, speak to anyone or even eat any food.” “Unless you you’ve got someone else who you can say ‘can you press the button for me?’…sometimes you can be in great pain….no no that wouldn’t cause any great problems…it would be alright…the chemo affects everyone differently….I did have a bit of problems with it but that was just in the early days if you had told me to do this everyday it would be like ‘oh no’, but I think I would have found some way of doing it though, so no, no it wouldn’t be a hardship” |
| Theme | Supporting quotes for solutions to improve usability |
|---|---|
| Peripheral neuropathy of the hands | “Why are these so small when there is space to make them bigger?” (in reference to some of the selection boxes on a scale) “The bigger the buttons, the better” |
| Fatigue and/or concentration issues | “If you can do something like this on your own time that would be better’ “It’s quite prohibitive actually to have such a short time window…but by actually setting the rules like that, you are actually giving a degree of importance to the survey…that’s a positive” “Sometimes you can’t do everything all at once…sometimes you might feel okay to start doing it and then all of a sudden…urgh…so it would be nice to know you could go back to it and save it, that would help” “If you are not in the right frame of mind…that is very important” (when asked about being able to save questionnaires and go back to them) |
| Timepoint in treatment cycle | “Maybe at different times, maybe I woke up in the morning, I didn’t want to get out of bed, I was ill, I had a bad night, but in the afternoon at 4 o’clock maybe I felt a bit better” “Maybe you can save what you’ve done and go back into it, that would be good…somebody could be sick” “I think there should be something for if you can’t fill it in, cause sometimes you don’t feel like doing anything that day…chemo is normally in the mornings… sometimes you might not feel like doing anything in the morning, sometimes you want to do something in the evening…I think you should be able to go in when you can…sometimes you cannot do things in set times cause you don’t have the energy…the chemo takes it out of you and you don’t want to do anything until night time, and sometimes you’re all buoyed up before your chemo and you can do it in the morning…your health is up-and-down” “Save it and go back to it…if you are not a well person like I wasn’t” |
| Caregiver assistance | “No, I’d want it done myself” “Yeah that would be a help actually…if you could give it to someone else, if you had to” |
| Voice assistant | “How many mistakes does it make? It makes mistakes…I don’t trust it…If you don’t read back the text, you’ve sent something which is nonsense” “Even if you have no energy, you still talk, you might be a bit slow…that could be a help…especially if you cannot use anything but you can still talk” |
| Stylus | “I can’t stand them…maybe the first or second session, maybe that would have helped me…I wouldn’t have kept it though” “I might like it…I’d probably opt for a stylus…I’m not strong about it but I would probably if I had an option…to be more accurate” “That could help, but if you had problems holding it wouldn’t do anything for you but it’s good to have, but sometimes you cannot hold anything anyway…even if you are hurting you can still pick up to hold…so yeah it would help…I think it would be easier” |
| Use of own device | “What I would prefer…is to have an app on my own phone” “I found it easier to use a laptop” |
Peripheral neuropathy of the hands
57% (4/7) of participants reported experiencing peripheral neuropathy of the hands that would impact their interaction with the devices used for recording ePROs. Descriptions of how this manifested included a lack of feeling in the hands, numbness, pins and needles, and stiffness (See Table 2). When treatment is complete, the peripheral neuropathy does not necessarily subside, and dexterity issues can remain long term. One participant reported that they used their smartphone less due to treatment affecting their hands and fingers, and another described not being able to use their hands straightaway in the morning.
However, participants did not see this as a barrier to completing the PRO instruments electronically and viewed using electronic reporting as superior to paper. With one participant saying it is “110% easier”. Furthermore, one participant reported difficulty writing during treatment and another reported difficulty turning pages due to numbness.
Regarding the implications of peripheral neuropathy in the hands for specific response scale types associated with common PRO instruments, there was no overall consensus on whether participants would prefer a numeric rating scale (NRS) or a sliding scale (visual analogue scale/VAS with a sliding marker). Some participants commented that a scale with larger buttons would be preferable (see Table 3) and that numbness would make it harder to use a sliding scale.
Fatigue and/or concentration issues
86% (6/7) of participants reported fatigue and/or concentration issues that may impact their ability to complete PRO instruments (of note, this would not be unique to electronic implementation). This could last throughout the day, for the duration of the treatment period, and for some time after treatment was complete. Participants did not report issues reading per se, but rather that they were tired and fatigued which meant they sometimes lacked the ability to interact mentally with reading materials and it could affect their concentration. One participant reported that they could probably only do a few questions at a time due to the fatigue. However, despite a reduced interaction with technology, another reported that even at the worst time they would still be able to use their phone. Importantly, no readability issues with the information on the devices were reported.
When asked about whether they had experienced ‘chemobrain’, only 29% said, ‘yes’ (2/7). Specifically, one participant described suffering short-term memory loss which could mean they would require more frequent reminders of the recall period for questions if there were multiple questions relating to this. They also reported heavily relying on images during chemotherapy instead of only written information. Another said they may not remember training instructions as a result of ‘chemobrain’.
Timepoint in treatment cycle
71% (5/7) of interviewees detailed that the timepoint in their treatment cycle would influence their health status and perceived ability to efficiently complete ePROs, with the early stages of treatment cycles reported as when they were feeling their worst (see Table 2). For example, one participant commented that they did not send post/mail during their first round of chemotherapy. Another said that they were fine on the day of chemotherapy but the second day they were weak and then they slept for two days, so they may find it difficult to complete instruments during this time. However, most participants said it is likely that they could find some time during the day when they would be able to complete ePROs.
Suggested solutions to improve usability
Encouragingly, despite the side effects of the treatment that could affect their interaction with technology, participants did not feel this would deter or inhibit them from completing PRO instruments on the devices.
You can be bad, but not so bad that you can’t use nothing at all…I would make the effort”“It's not difficult”“You’d make time in the day to do it”“I would still do it
However, there were some suggested approaches to improve usability directly related to the identified themes. These included: larger, well-spaced buttons to deal with finger numbness, the ability to pause a survey mid-way and complete at a later point, and presenting the recall period with every question to reduce reliance on memory. Reminders would also be important, and if they had paused in-between questionnaires it would be important to prompt them to return and complete the remaining ones. See Table 3 for supporting quotes.
Participants were also asked about some potential solutions to assist them with completing the ePROs (see Table 3). One such solution was to have caregivers assisting with completion; this was generally not a preferred solution, with one participant not wishing to worry their partner by letting them know how bad they are feeling. One participant, however, did feel that caregiver assistance could be a beneficial option at times. When asked about the use of a voice assistant, some were open to this idea but most tended not to see this as preferable as it could be annoying. There were mixed views about using a stylus, with most feeling they would not like it, although the option to have it would be appreciated. Finally, some participants mentioned that they would like to have an app on their own phone as opposed to a provisioned device which could just be another thing to remember; they may not take a provisioned device everywhere with them and so at certain times of the day they may miss the time window to complete the ediary (See Table 3).
Discussion
Participants in the current study reported that the following anticancer treatment related factors would impact their completion of ePRO assessments in a clinical trial: peripheral neuropathy of the hands, experiencing significant tiredness and fatigue which can also impair concentration, and the timepoint in their treatment cycle. The identified themes do not act independently of one another: peripheral neuropathy can be a direct side effect of anticancer treatments, but could also be an indirect symptom resulting from cognitive impairments that affect psychomotor function [9]; fatigue can significantly impact cognition and concentration [13]; and peripheral neuropathy and tiredness and/or fatigue, as well as difficulties with concentration may be more likely to affect individuals at certain times of the day, as well as be influenced by where they are in the treatment cycle. Every patient is different, and a one-size fits all approach is unlikely to be suitable for all. Therefore, identifying potential solutions to optimize implementation of ePROs for as many individuals as possible is desirable, and being flexible in the approach is likely to have the most beneficial effect.
Participants reported that the CIPN they experienced would not be an insurmountable barrier to them completing PRO instruments electronically, and they saw electronic reporting and ‘tapping’ as superior to writing or turning pages. However, participants commented that larger buttons, the functionality to choose between tapping or sliding an anchor (where individuals would move the anchor up-and-down by maintaining contact with the screen) on a visual analogue scale, and the option of a stylus could assist completion. With knowledge of the high prevalence of CIPN, the absence of an effective treatment [9,23], and a move toward recording PROs electronically, it is important to take this into consideration when designing electronic solutions.
In line with the literature documenting the commonality of cancer-related fatigue, the participants interviewed reported that their cancer and treatment had led to substantial tiredness, fatigue and/or concentration issues that may impact their ability to complete PRO instruments, and could sometimes lead to a reduced ability to interact mentally with reading materials. It is important to note that this would have an impact on PRO instruments in general and not be specific to electronic implementation, and participants did not report readability issues with the information presented on the devices they were shown.
Some participants reported that they would still be able to use their phone even when they were experiencing their fatigue at its worst, and others said it is likely that they would only be able to complete a few questions at a time. Participants also reported that the impact of their symptoms varied during the treatment cycles, with the beginning of the cycle tending to be the time when they were at their worst, which is in line with other studies that have shown the first few days following treatment to be when symptoms (e.g., fatigue) are at their peak [24]. Encouragingly, participants in our study reported that they would be able to find some time in the day when they could complete the ePROs. However, at such times, they may feel unable to complete PROs on an at-home ediary within a restricted time window and require more flexibility in both the time of day and the time window that they can complete them. Further, such symptoms can occur suddenly, emphasizing the importance of flexibility in the daily schedule of ediary completion.
It is recognized that enabling flexibility in the time window to complete measures and allowing participants to pause and return to a survey may be seen as problematic to trial sponsors, as it could impact the contemporaneous nature of the data which is used to ensure the validity of group comparisons. However, it is important to weigh this up against the higher likelihood of missing data if this is not an option. In their guidance on the evaluation of anticancer medicinal products, the EMA recognize that PRO measures should be administered [4]:
when it is feasible to expect high levels of completion by the individual
Some participants also described memory deficits and so it would be important that the recall period for questions is reiterated if multiple questions relate to that recall period across screens. Reminders for ediary completion would also be important, especially if the individual was not ready to complete it at the beginning of the time window. Additionally, simple solutions to reset forgotten PINS and access to quick ‘how to’ guides would be important due to memory issues that can be experienced.
When considering ways to improve ease of participation in clinical trials with at-home electronic data collection, some participants commented that they may like the option to complete ePROs on their own devices (phone, laptop) – known as ‘Bring Your Own Device’ (BYOD). Participants will be familiar with their own devices and so this option could increase engagement and compliance, potentially reducing missing data [25]. Further, measurement equivalence between electronic formats has been demonstrated [25,26], and so having participants complete the same assessment on different electronic devices would not impact the data being collected. Voice-assisted technology and caregiver completion tended not to be desirable solutions; some participants reported not wanting their ‘carer’ to know how sick they were really feeling, and so it is important to consider possible issues regarding ‘truthfulness’ of answers provided in this scenario [27].
Encouragingly, participants reported that the symptoms described would not deter them from participating in clinical trials and they had a positive attitude toward the use of technology to complete assessments. Completing measures electronically was preferable to paper completion and deemed more suitable given some of the impairments that can result from treatment. Nevertheless, our findings do suggest ways that ePRO implementation in oncology clinical trials can be optimized, and given the increasing drive for measuring outcomes that are important to the patient, applying this learning as we design oncology-specific solutions is imperative. Based on our findings we recommend consideration of the implementation best practices listed in Table 4 when using ePROs in this therapeutic population.
| Recommendation | |
|---|---|
| a) | Utilize larger, well-spaced buttons. |
| b) | Enable functionality to choose between tapping or sliding an anchor (where individuals would move the anchor up-and-down by maintaining contact with the screen) on a visual analogue scale |
| c) | Present recall period along with the full question text and response options to reduce the reliance on participants needing to remember the question recall period from previous screens. |
| d) | Enable some flexibility in the time window that participants complete the survey/questionnaire. |
| e) | Allowing ‘rest’ intervals between questionnaires so that participants can return to remaining questionnaires to complete at a later point (the time window should be adequately detailed in the trial protocol). |
| f) | Provide a stylus so that participants have this option. |
| g) | If suitable (taking into account trial design), implement PRO measures on the participant’s own device via an app or web-based solution (BYOD). |
| h) | Automated completion reminders. |
| i) | Be sensitive that participants may prefer to not complete PROs, rather than asking for caregiver assistance. |
| j) | Provide simple approaches for participants to reset their own PIN if forgotten, and to access simple quick reference guides. |
BYOD: Bring Your Own Device; PRO: Patient-reported outcome.
It is important to note these recommendations are ‘where possible’, as not all instrument authors will grant permission for changes to the presentation of scales when migrated from paper to electronic format. However, increasing accessibility is imperative and it is hoped that instrument authors will be aware of the challenges faced by oncology patients (given many of the scales used will be oncology-specific) and willing to facilitate their optimization to ensure they are fit-for-purpose.
Along with the increase in implementing PRO instruments electronically in clinical trials, there has also been an increase in trials incorporating measurements from wearable devices and other sensors [28]. These can be used to monitor a range of metrics, including sleep (duration and efficiency), activity, heartrate and blood pressure. The passive, nonobtrusive nature of their measurement offers a particularly attractive option for certain patient populations who are at high risk of treatment-related side effects (such as oncology), providing substantial benefit by reducing participant burden [29]. That is not to say that they would replace PROs, rather wearables could compliment these [30], offering some indication of health status when participants are too unwell to complete PRO instruments, such as tracking impact and recovery profiles associated with each treatment cycle. Such objective data could also help validate and corroborate the data from PRO measures [30]. Furthermore, the data obtained can be very informative about the impact of treatment during the early stages of a treatment cycle, which can be masked by the end-of-cycle site-based assessment when health status may have improved [30]. Studies have shown that in oncology patients step count is associated with likelihood of hospitalization, adverse events and mortality, and a strong predictor of treatment-related toxicity [30]. Studies investigating sleep patterns demonstrated that large discrepancies between self-report and actigraphy-based capture of habitual sleep patterns highlight the value of wearables when investigating such variables in relation to the efficacy and effectiveness of oncology treatments [31].
Adoption of these solutions in clinical trials is sometimes associated with uncertainty by researchers on whether regulators will accept sensor/wearables data in regulatory decision-making and drug labeling claims [28]. However, as PROs are increasingly used in drug approval applications, it is likely that data from wearables and sensors will also grow [28]. Future studies exploring the feasibility and acceptability of these solutions in this patient population would be valuable.
To our knowledge, this is the first study in oncology exploring how the disease and symptoms associated with its treatment can impact device use for recording ePRO data. Participants represented a broad range in age and cancer types and varying times since diagnosis and treatment length. We recognize the limitations of our study, including the small sample size (as is common in qualitative studies); however, participants were consistent in the issues they discussed and reported. Additionally, we acknowledge that the issues raised in these interviews may not reflect the issues experienced by all people with cancer and that these interviews were part of a broader usability study – a context which can encourage participants to be overly critical, and over think potential issues which may otherwise not have been reported. Nevertheless, our findings are in line with previous studies in the literature. Future studies in larger samples will be beneficial to strengthen the findings from this study.
Conclusion
Electronic capture of PROs offer considerable benefit for oncology patients in clinical trials but our findings highlight that peripheral neuropathy of the hands, tiredness, fatigue and/or concentration impairments experienced as a result of the disease and side effects of treatment would have specific implications for their use of technology for completing ePROs. While the identification of these symptoms is not new, the patient perspective on how they would impact their use of technology and strategies for optimizing usability is, and to our knowledge has not been explored. The insight gained in this research can be harnessed to ensure electronic solutions are optimal and ‘fit-for-purpose’, and the lessons learned used to inform best practice recommendations for implementation of ePROs in this population, placing patient needs at the forefront. We encourage clinical trial sponsors to take these into consideration when designing and implementing ePRO solutions in oncology studies.
•
There is an increasing drive in clinical research to measure outcomes that are important to the patient, and electronic implementation of PROs (ePROs) offer considerable benefit for oncology patients in clinical trials. However, symptoms associated with the disease and treatment can impact their use of technologies for recording PROs.
•
To our knowledge, this is the first study in oncology exploring how the disease and symptoms associated with its treatment can impact device use for recording ePRO data.
•
Participants reported that during the course of the disease they experienced chemotherapy-induced peripheral neuropathy of the hands, tiredness, fatigue and/or concentration impairments, and varying health status based on where they were in the treatment cycle, which would impact their device use.
•
Participants had a positive attitude toward the use of technology to complete assessments and completing measures electronically was preferable to paper completion.
•
We recommend consideration of the following when implementing ePRO measures in this therapeutic population: utilizing larger, well-spaced buttons; enabling functionality to choose between tapping or sliding an anchor (where individuals would move the anchor up-and-down by maintaining contact with the screen) on a visual analogue scale; presenting the recall period together with the question text and all response options to reduce the reliance on participants needing to remember the question recall period from previous screens; enabling some flexibility in the time window that participants complete the survey; allowing ‘rest’ intervals between questionnaires so participants can return to remaining questionnaires to complete at a later point; providing a stylus so that participants have this option; if suitable, implementing PRO measures on the participant’s own device via an app or web-based solution ‘Bring Your Own Device’; automated completion reminders; being sensitive that participants may prefer to not complete PROs, rather than asking for caregiver assistance; and provide simple approaches for participants to reset their own PIN if forgotten, and to access simple quick reference guides.
•
In combination with the previous literature demonstrating significant side effects of anticancer treatments, the findings from this research can facilitate the development of patient-centered solutions that account for the unique challenges experienced by oncology patients, and ensure electronic solutions are optimal and ‘fit-for-purpose’. The best practice recommendations for implementation of ePROs in this population provide the potential to improve data quality and places patient needs at the forefront.
Author contributions
B Sanderson and B Byrom contributed to the conception and design of the study. FD Mowlem analyzed the data and wrote the manuscript. All authors contributed to interpretation of the findings, reviewed and edited drafts of the manuscript, and approved the final version.
Acknowledgments
The authors extended their thanks to all the participants who gave up their time to take part in these interviews and provided valuable insights.
Financial & competing interests disclosure
When conducting this work, all authors were employed by Signant Health. Otherwise, the authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.
No writing assistance was utilized in the production of this manuscript.
References
1.
Basch E, Abernethy AP, Mullins CD et al. Recommendations for incorporating patient-reported outcomes into clinical comparative effectiveness research in adult oncology. J. Clin. Oncol. 30(34), 4249–4255 (2012).
2.
Fiero MH, Roydhouse JK, Vallejo J, King-Kallimanis BL, Kluetz PG, Sridhara R. US Food and Drug Administration review of statistical analysis of patient-reported outcomes in lung cancer clinical trials approved between January, 2008, and December, 2017. Lancet Oncol. 20(10), e582–e589 (2019).
3.
Gnanasakthy A, Barrett A, Evans E, D'Alessio D, Romano C, De M. A review of patient-reported outcomes labeling for oncology drugs approved by the FDA and the EMA (2012–2016). Value Heal. 22(2), 203–209 (2019).
4.
EMA. EMEA Appendix 2 to the guideline on the evaluation of anticancer medicinal products in man. 44(April), 1–18 (2014).
5.
Stone AA, Shiffman S, Schwartz JE, Broderick JE, Hufford MR. Patient non-compliance with paper diaries. Br. Med. J. 324(7347), 1193–1194 (2002).
6.
FDA. Guidance for industry Patient Reported Outcome Measures: Use in medical product development to support labeling claims. 1–39 (2009). www.fda.gov/media/77832/download
7.
Giesinger JM, Wintner LM, Zabernigg A et al. Assessing quality of life on the day of chemotherapy administration underestimates patients’ true symptom burden. BMC Cancer 14(1), 1–7 (2014).
8.
Kelley MR, Fehrenbacher JC. Challenges and opportunities identifying therapeutic targets for chemotherapy-induced peripheral neuropathy resulting from oxidative DNA damage. Neural Regen. Res. 12(1), 72–74 (2017).
9.
Zajaczkowską R, Kocot-Kępska M, Leppert W, Wrzosek A, Mika J, Wordliczek J. Mechanisms of chemotherapy-induced peripheral neuropathy. Int. J. Mol. Sci. 20(6), 1451 (2019).
10.
Brown TJ, Sedhom R, Gupta A. Chemotherapy-induced peripheral neuropathy. JAMA Oncol. 5(5), 750 (2019).
11.
Ebede CC, Jang Y, Escalante CP. Cancer-related fatigue in cancer surviorship. Med. Clin. North Am. 101(6), 1085–1097 (2017).
12.
CRUK. Fatigue and cancer drugs (2020). www.cancerresearchuk.org/about-cancer/cancer-in-general/treatment/cancer-drugs/side-effects/fatigue
13.
Pendergrass JC, Targum ST, Harrison JE. Cognitive impairment associated with cancer: a brief review. Innov. Clin. Neurosci. 15(1-2), 36–44 (2018).
14.
Janelsins MC, Kesler SR, Ahles TA, Morrow GR. Prevalence, mechanisms, and management of cancer-related cognitive impairment. Int. Rev. Psychiatry 26(1), 102–113 (2014).
15.
Gutmann DH. Clearing the fog surrounding chemobrain. Cell 176(1-2), 2–4 (2019).
16.
Hurria A, Somlo G, Ahles T. Renaming ‘chemobrain’. Cancer Invest. 25(6), 373–377 (2007).
17.
Moore HCF. An overview of chemotherapy-related cognitive dysfunction, or ‘chemobrain’. Oncol. Willist. Park 28, 797–804 (2014).
18.
Ahles TA, Saykin A. Cognitive effects of standard-dose chemotherapy in patients with cancer. Cancer Invest. 19(8), 812–820 (2001).
19.
Abernethy AP, Herndon JE, Wheeler JL et al. Feasibility and acceptability to patients of a longitudinal system for evaluating cancer-related symptoms and quality of life: pilot study of an e/tablet data-collection system in academic oncology. J. Pain Symptom Manage. 37(6), 1027–1038 (2009).
20.
O'Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Acad. Med. 89(9), 1245–1251 (2014).
21.
Nowell LS, Norris JM, White DE, Moules NJ. Thematic analysis: striving to meet the trustworthiness criteria. Int. J. Qual. Methods 16(1), 1–13 (2017).
22.
Braun V, Clarke V. Using thematic analysis in psychology. Qual. Res. Psychol. 3(2), 77–101 (2006).
23.
Staff NP, Grisold A, Grisold W, Windebank AJ. Chemotherapy-induced peripheral neuropathy: a current review. Ann. Neurol. 81(6), 772–781 (2017).
24.
de Jong N, Kester AD, Schouten HC, Abu-Saad HH, Courtens AM. Course of fatigue between two cycles of adjuvant chemotherapy in breast cancer patients. Cancer Nurs. 29(6), 467–477 (2006).
25.
Byrom B, Gwaltney C, Slagle A, Gnanasakthy A, Muehlhausen W. Measurement equivalence of patient-reported outcome measures migrated to electronic formats: a review of evidence and recommendations for clinical trials and bring your own device. Ther. Innov. Regul. Sci. 53(4), 426–430 (2018).
26.
Byrom B, Doll H, Muehlhausen W et al. Measurement equivalence of patient-reported outcome measure response scale types collected using bring your own device compared to paper and a provisioned device: results of a randomized equivalence trial. Value Heal. 21(5), 581–589 (2017).
27.
Abma JC, Chandra A, Mosher WD, Peterson LS, Piccinino LJ. Fertility, family planning, and women's health: new data from the 1995 National Survey of family growth. Vital Heal. Stat. 19, 1–114 (1997).
28.
Cox SM, Lane A, Volchenboum SL. Use of wearable, mobile, and sensor technology in cancer clinical trials. JCO Clin. Cancer Informatics (2), 1–11 (2018).
29.
Purswani JM, Ohri N, Champ C. Tracking steps in oncology: the time is now. Cancer Manag. Res. 10, 2439–2447 (2018).
30.
Gresham G, Hendifar AE, Spiegel B et al. Wearable activity monitors to assess performance status and predict clinical outcomes in advanced cancer patients. NPJ Digit. Med. 1(1), 1–8 (2018).
31.
Moore CM, Schmiefe SJ, Matthews EE. Survivors: discrepancy in selected sleep parameters. Behav. Sleep Med. 13(6), 472–490 (2015).
Information & Authors
Information
Published In
Pages: 1195 - 1204
PubMed: 33274651
Copyright
© 2020 Future Medicine Ltd.
History
Received: 8 July 2020
Accepted: 28 September 2020
Published online: 4 December 2020
Keywords:
Topics
Authors
Metrics & Citations
Metrics
Article Usage
Article usage data only available from February 2023. Historical article usage data, showing the number of article downloads, is available upon request.
Citations
How to Cite
Optimizing electronic capture of patient-reported outcome measures in oncology clinical trials: lessons learned from a qualitative study. (2020) Journal of Comparative Effectiveness Research. DOI: 10.2217/cer-2020-0143
Export citation
Select the citation format you wish to export for this article or chapter.
Citing Literature
- Florence D. Mowlem, Paul O’Donohoe, Accessible data collection methods in clinical trials: do current best practices for the implementation of electronic patient-reported outcome measures (ePROMs) meet accessibility standards?, Journal of Patient-Reported Outcomes, 10.1186/s41687-026-01039-8, 10, 1, (2026).
- Michelo Banda, Sian Bladon, Mariam Al-Attar, Roberto Cahuantzi, David A Jenkins, William G Dixon, Sabine N van der Veer, Current methods for analyzing time-series patient-generated health data to assess treatment response: a scoping review, Journal of the American Medical Informatics Association, 10.1093/jamia/ocag027, 33, 5, (1065-1076), (2026).
- Hosna Salmani, Somayeh Nasiri, Mahdi Alemrajabi, Maryam Ahmadi, Electronic patient-reported outcome systems and capabilities in cancer care: a systematic review, Frontiers in Digital Health, 10.3389/fdgth.2025.1560533, 7, (2025).
- Johanne Dam Lyhne, Allan ‘Ben’ Smith, Tina Birgitte Wisbech Carstensen, Lisa Beatty, Adeola Bamgboje-Ayodele, Britt Klein, Lars Henrik Jensen, Lisbeth Frostholm, Adapting a Self-Guided eHealth Intervention Into a Tailored Therapist-Guided eHealth Intervention for Survivors of Colorectal Cancer, JMIR Cancer, 10.2196/63486, 11, (e63486-e63486), (2025).
- Hosna Salmani, Somayeh Nasiri, Mahdi Alemrajabi, Maryam Ahmadi, Advancing patient-centered cancer care: a systematic review of electronic patient-reported outcome measures, Frontiers in Rehabilitation Sciences, 10.3389/fresc.2024.1427712, 5, (2024).
- Hosna Salmani, Somayeh Nasiri, Maryam Ahmadi, The advantages, disadvantages, threats, and opportunities of electronic patient-reported outcome systems in cancer: A systematic review, DIGITAL HEALTH, 10.1177/20552076241257146, 10, (2024).
- Florence D. Mowlem, Celeste A. Elash, Kelly M. Dumais, Estelle Haenel, Paul O’Donohoe, Jennifer Olt, Alexandra V. Kalpadakis-Smith, Ben James, Grazia Balestrieri, Kayci Becker, Melissa C. Newara, Scottie Kern, Best Practices for the Electronic Implementation and Migration of Patient-Reported Outcome Measures, Value in Health, 10.1016/j.jval.2023.10.007, 27, 1, (79-94), (2024).
- Anna Bolzani, Sophie Kupf, Farina Hodiamont, Isabel Burner-Fritsch, Claudia Bausewein, Christina Ramsenthaler, Measurement equivalence of the paper-based and electronic version of the Integrated Palliative care Outcome Scale (IPOS): A randomised crossover trial, Palliative Medicine, 10.1177/02692163231157871, 37, 5, (760-770), (2023).
- J. Moore, N. Goodson, P. Wicks, J. Reites, What role can decentralized trial designs play to improve rare disease studies?, Orphanet Journal of Rare Diseases, 10.1186/s13023-022-02388-5, 17, 1, (2022).
- Florence D. Mowlem, Pamela Tenaerts, Chad Gwaltney, Ingrid Oakley-Girvan, Regulatory Acceptance of Patient-Reported Outcome (PRO) Data from Bring-Your-Own-Device (BYOD) Solutions to Support Medical Product Labeling Claims, Therapeutic Innovation & Regulatory Science, 10.1007/s43441-022-00412-1, 56, 4, (531-535), (2022).