All together now: findings from a PCORI workshop to align patient-reported outcomes in the electronic health record

In November 2013, the Patient-Centered Outcomes Research Institute (PCORI) and its Methodology Committee hosted a PRO Infrastructure Workshop (Atlanta, GA, USA) to understand current practices integrating PROs into EHR systems, and to generate broad stakeholder priorities to advance and sustain use. This meeting had three main goals: a landscape review, panel presentations and stakeholder discussion groups.

Patients, clinicians, researchers, healthcare system leaders and policymakers were invited to present on current use, identify common implementation barriers, share examples of clinical, research and quality improvement uses, and discuss strategies and priorities for increasing collection and use of PROs in EHR systems. A vision was articulated for integrating PRO measures within EHRs and for creating a larger national data infrastructure for medical care and public health. At the local level, this infrastructure would permit concurrent research and evaluation to be conducted more routinely within a real-world delivery system. It would also allow the care of individual patients to benefit from the knowledge and experience of those previously treated. This infrastructure would be fractal, locally used but also aggregating up to the regional and national levels. At the national level, the overall structure would support widespread capability for the US healthcare system to learn from practice, quality improvement and research [5,9,10].

Based on a pre-meeting landscape review of the field [11], we designed workshop presentations and discussion groups to learn about the range of current integration trends. Below are highlights and the key messages from the workshop, with a focus on how PROs most often collected for clinical practice applications can be leveraged for broader quality improvement and research applications. Following that is a summary of progress on the resulting agenda to date.

Workshop participants included in-person and online attendees (see Table 1 for description of participants). Panel presentations focusing on specific examples of PRO integration in the EHR were followed by full group discussion, interspersed with small group breakout sessions, with an opportunity for all attendees to rotate through breakout sessions on three topics: patient and PRO data, clinical use of PRO data and research and quality measurement. The small group breakout sessions proceeded through semi-structured discussion to address barriers and facilitators of PRO integration in the EHR and to submit recommendations for concrete actions for prioritization through consensus in the final session with all attendees. Data were analyzed and guided by basic principles of grounded theory [12]. Our thematic content analysis began with line-by-line examination of the transcripts. Two of the authors independently identified provisional categories and themes (RE Jensen & AW Wu). Initial theories were compared against newly identified themes, synthesis from the discussion, and author notes, to ensure validity. Nominal grouping was used to reach consensus regarding final categorization (Boxes 1 & 2).

We found that current integration efforts focus on informing and improving clinical care across different types of illness and healthcare settings. Very few examples were identified of PROs being pulled from EHRs for research-specific purposes. We identified four predominant EHR/PRO system designs: physician use during a patient visit; quality assessment and improvement; population-based risk screening and clinical and health services research. Each design illustrates the potential of collecting PROs and integrating results into the EHR. Current efforts are tailored to specific contexts, resulting in a wide range of target populations, assessment modalities, data collection schedules and external PRO reporting. This degree of specificity limits widespread collection across different systems. While goals and methods are similar, better understanding of commonalities and a core set of cross-cutting standards is necessary to provide a platform for widespread adoption.

Leading examples in patient-level symptom tracking tend to be for specific diseases that benefit from long-term monitoring, such as rheumatoid arthritis [13–15], HIV/AIDS [16] and cancer patients undergoing chemotherapy [17]. In these cases, PRO data modified treatment. Risk-screening applications focused on linking high-risk scores to targeted interventions. For example, at Kaiser Permanente Colorado a positive previsit PRO screen triggers standardized order sets to direct testing, treatment and referrals for geriatric patients.

Quality improvement efforts on symptom identification and management were identified at all healthcare levels: clinics, practices and populations. For example, Minnesota Community Measurement tracks and publicly reports PRO-based depression screenings and 6-month remission rates at the clinic level from over 80,000 patients annually [18]. In these cases, PRO measurement provides information on the PRO screening use, symptom prevalence and management.

The final workshop goal was to identify and understand stakeholder perspectives and their interest in promoting PRO EHR integration. Healthcare payers, such as Medicare, are looking to demonstrate value in healthcare beyond mortality and costs, which PRO-based quality of care indicators could provide. Healthcare plans, specifically integrated healthcare delivery systems such as Kaiser Permanente and Group Health Cooperative, are looking to leverage their existing clinical electronic data collection infrastructure. Clinicians and researchers were identified as ‘early adopters’, as patient-level PRO collection and use is most commonly driven by academic medical centers, often for use in specific clinics or conditions funded through research grants. Finally, patients themselves are important stakeholders, demonstrating increased interest in understanding their symptoms and functional issues, while current PRO data capture options and interpretation can be provided directly to the patient [19], aligning with growing patient expectations for transparency and access to medical information.

Stakeholders met in small groups to identify barriers and successful strategies to inform a research agenda: patient concerns, individual clinical care and large scale evaluation (quality improvement and research applications). Differences in needs, interests and expertise resulted in a range of opinions, but each of the groups was successful at achieving consensus. Below is a discussion of the main elements from each topic (Box 1) and actionable next steps (Box 2).

Discussions regarding patient concerns focused on PRO collection. This action requires engaged patients, as they must be willing to complete assessments and provide responses that accurately reflect their current health. Reasonably complete and accurate data are needed for the timely identification of new concerns and to prevent biased assessment of care quality at the population level due to nonresponse.

Patient burden and the amount of time patients invest to complete PRO assessments were also important considerations. Research indicates that patients report high satisfaction completing assessments if they think data are used to inform their care [20]. However, it is possible that patients with different levels of functioning may value the usefulness of PRO data differently. Understanding how best to promote relevant PRO assessments while identifying the maintaining content relevance and low burden may help address these concerns.

Beyond promoting patient engagement, ethical issues related to the potential harms, respect for persons and privacy concerns related to PRO collection were identified. Just as a patient nonresponse to a PRO screen may delay issue identification, poor integration of PRO collection into patient care may disrupt or delay care. Additionally, PRO collection can be annoying to patients, if it feels redundant or repetitive with other information collected in the provision of care.

Finally, workshop participants raised concerns over the privacy and protection of their PRO data. Currently, many institutions require a blanket consent process, where patients give general consent for use of all of their data, clinical and patient-reported. To date, no published work has explored the extent to which sensitive PRO data is restricted within the EHR (e.g., assessments of a patient’s satisfaction with care), or outside the EHR (e.g., disclosure of depression severity to family members). Providing clear assurances of data privacy, clinical benefit and recognizing patient burden may help address these concerns and help promote patient involvement.

Themes from these small group sessions focus on improving the individual-level patient experience by increasing the clarity and relevance of clinical PRO assessment. Due to the limited information available and strong interest in ethical and privacy issues surrounding EHR integration, a targeted recommendation was identified as a necessary and immediate priority.

Clinical use focused on how PRO collection could be best integrated into every-day clinical care. A key issue voiced by workshop participants was the current established dichotomy between clinical and PRO-based measures in care settings. Many participants felt the benefits of PRO integration are not currently well-communicated and PROs often feel irrelevant and unimportant relative to clinical laboratory values. Clinicians also need guidance for interpreting PRO data, to ensure the data are translated into clinical actions as appropriate. A provider may see an ‘anxiety’ score of 60, but not know if it is a concerning score or if follow-up is necessary.

Many possible solutions were identified. One was to move beyond symptoms and function by including patient goals, treatment preferences and experience of care. Financial incentives were also highlighted as a way to promote and disseminate use. Establishing reimbursements for PRO screening or achieving quality of care benchmarks provides actionable patient follow-up while supporting quality improvement.

The final topic discussed by workshop participants was the necessity of minimizing workflow disruption. Participants agreed that workflow is often unique to each clinic, and general PRO standardization guidelines would not be effective. However, many agreed that ‘success stories’ of early PRO system adopters, and establishing meaningful PRO-based evaluation outcomes demonstrating institutional value and/or cost savings, would encourage adoption.

Recommendations for PRO integration in the clinical setting focused on promoting the value of PRO assessment in routine care. For clinicians, recommendations centered on clinical situations where PROs provide immediately actionable information. For system-level stakeholders, recommendations look to establish PRO value with respect to quality-of-care evaluation.

Participant discussions examining the promotion of large-scale research and evaluation focused on overcoming one key barrier: that the majority of current efforts to integrate PROs in the EHR have focused on demonstrating clinical relevance at the individual patient-level. Therefore the patient visit has been identified as the focal point for engagement, whereas population-level efforts may require collection and evaluation outside the clinical encounter. Opportunities identified focused particularly on demonstrating how PRO information in large-scale quality measurement and improvement extend beyond the clinical visit. Research and quality improvement recommendations focused on the necessity and complexity of large-scale PRO surveillance. Workgroups identified and prioritized a wide range of methodology, content, infrastructure and dissemination goals necessary to promote and sustain successful surveillance efforts.

Stakeholders also recommended that current collaborations and data networks be enlisted as contributors to standardize PRO integration across systems. PCORNet was specifically highlighted as a resource to promote best practices in population-level PRO use. PCORNet is a PCORI-funded effort which has created a national, highly representative, patient-centered network to support more efficient clinical and observational research [21]. Even in development, PCORNet accelerated the adoption of routine PRO measurement within EHRs by convening a national PRO taskforce charged with identifying a common core PRO data collection.

A meeting theme across all groups was that PROs are not ‘one size fits all’. PROs comprise a broad range of information, including hundreds of symptoms, functional issues and topics that relate to quality of life. Effective measure deployment from this armamentarium differs based on context, as different PRO information will be clinically relevant and actionable at different times. More research is necessary to better understand how individual patient goals and preferences can tailor PRO selection for maximum, immediate and long-term patient benefit. For population-level PRO use, clinicians and health system representatives must also weigh in on matching measures to specific goals.

The EHR is a natural vehicle to collect and disseminate PRO data across a broad variety of clinical situations. However, current projects integrating PRO measures in EHRs are often designed for specific populations, clinical applications and quality improvement, which limits opportunities for expansion and scaling up across settings. Based on this workshop, immediate priorities were identified in three areas to promote PRO integration: education, research and infrastructure (Box 2). First, for education, guidance for providers [22], institutions [23] and healthcare systems are limited [24,25], often (but not always [22]) focusing on PRO selection and score interpretation. Expanding efforts to address the complexities of workflow and administration is necessary. Second, for research, directed methodological research to address gaps in PRO conceptualization, measurement and implementation can provide the information needed to promote interpretability and relevance. Third, for infrastructure, additional tools and resources are needed to support broad infrastructure development and demonstrate the value of PROs to major stakeholders. Together these elements best leverage EHR integration for widespread PRO adoption and regular use.

Priorities identified by workshop participants were incorporated into PCORI Funding Announcement development and, as a result, approximately 10% of the methods priority area portfolio reflects projects based on the priorities identified from this convening of diverse stakeholders. In addition, in follow-up to the 2013 PRO EHR meeting and in response to these immediate priorities, PCORI is now funding an effort to develop a ‘Users’ Guide for Integrating Patient-Reported Outcomes in Electronic Health Records' (PCORI contract #MC-0013; PI: Snyder). This Users’ Guide will review the considerations for including PROs in EHRs, offering a range of alternatives rather than one ‘right’ answer, thereby allowing clinics and institutions to determine what will work best in their environments. This effort will also identify key research questions for the field. A public meeting presenting the Users’ Guide will offer an opportunity for participants to discuss voluntarily implementing PRO integration in consistent ways to enable data pooling.

Diverse stakeholders were convened to discuss examples of successful integration of PROs into the EHR to support clinical care, health system- or population-level monitoring, quality management and research. Through group discussion and guided workshop sessions, barriers to use of PRO data through the EHR were identified, allowing for creation of an agenda for future work. PCORI has begun to address this agenda through funding solicitations based on the priorities identified at the workshop, through PCORNet and through a focused contract to create a guide for PRO integration in the EHR applicable across care settings.

Involving multiple stakeholder groups in the problem identification and articulation of priorities is a first step toward ensuring future work is responsive to the differing needs across the care delivery spectrum. Patient and clinician engagement addresses feasibility and relevance of PRO data collection. Health and IT professionals, as well as researchers, are necessary to address system interoperability while ensuring data accuracy. Healthcare plans, professional groups, accreditation organizations and the government can provide incentives to integrate, report and evaluate PROs as part of routine care. PCORI and other influential groups, such as the NIH Collaboratory, are well-positioned to guide this effort [26]. Joint interdisciplinary participation in this effort to standardize the patient voice within the EHR will ensure PROs are actionable, positioned to improve care quality and the patient healthcare experience at both the patient and system-level.

AW Wu, CF Snyder and RE Jensen have received funding from the Patient-Centered Outcomes Research Institute. RE Jensen has received funding from the National Cancer Institute (P30CA051008). Jensen has received funding from the National Center for Research Resources, National Center for Advancing Translational Sciences, NIH (KL2TR000102). The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.

No writing assistance was utilized in the production of this manuscript.