Letter in Reply – Quality in qualitative evidence: new best practice principles from NICE’s real-world evidence framework

We agree that the two examples cited in our editorial for qualitative evidence considered in HST19 [2] and HST27 [3] reflect distinct context and methodologies. The intention of our editorial was not to compare these examples directly or suggest one as an example of best practice, but rather to illustrate the varying roles of qualitative evidence in supporting NICE decision-making. Each example was chosen to highlight different scenarios in which qualitative evidence has featured in appraisal committee discussion. The presented examples provided insights on outcomes valued by patients (HST19) or lived experiences of patients that could not be captured by trial outcomes (HST27).

We acknowledge differences in organizational capacity for generating this evidence and in turn indicate the need for consistent guidance and supportive frameworks for generating qualitative evidence.

The information for the editorial’s chosen examples of qualitative evidence in NICE technology evaluations is derived from publicly available NICE appraisal documentation. For HST27, our reference to the committee’s discussion aimed to reflect published considerations and conclusion that patient and carer testimonies were persuasive and significantly influenced the interpretation of the clinical trial and cost-effectiveness evidence. Specifically, it was noted that given the lack of scientifically robust tools to measure overall effects of the condition, qualitative information provided nuanced understanding of the burden of the disease. The committee also note that data, if collected systematically and analyzed using formal analytical techniques, would provide more robust evidence. This is a common concern for qualitative evidence noted by experts in the area [4,5]. Thus, further highlighting the need for best practice recommendations.

We fully recognize resource and capacity constraints for many patient organizations to conduct high quality qualitative research. With this as an important consideration, our editorial clearly notes the challenges involved and highlights the need for greater support and capacity building. The recent update to NICE RWE framework [6] on qualitative evidence was developed partly in response to stakeholder feedback identifying a lack of guidance on qualitative evidence to support NICE evaluations. The framework recommendations, summarized in the editorial, are intended as a best practice guide and do not set minimum standards. They are aimed at enhancing transparency, credibility and usability of qualitative evidence that is submitted by evidence developers including patient organizations.

We share the opinion that development and use of patient-generated evidence must be fair, meaningful and not tokenistic. Our editorial acknowledges the evolving role of qualitative evidence in HTA decision-making and contends that robust development of such evidence is necessary to strengthen its credibility and influence. In both examples cited, the patient-generated evidence had a significant impact on the committee’s interpretation of the evidence and resulted in changes to the plausible cost-effectiveness estimates used for decision-making.

NICE maintains engagement with patients and stakeholders to ensure their perspectives are meaningfully and appropriately incorporated into its decision-making, particularly in HST evaluations. NICE’s in-house People and Communities team (PaCT) meets with patient experts and organizations throughout the course of the evaluations to ensure that they remain engaged, understand the process and can contribute in the most meaningful way possible. NICE, along with several voluntary and community sector organizations (VCSs), also co-leads a dedicated voluntary and community sector network. This meets on a quarterly basis online and is supported by a monthly training and education sessions on subjects that are at the forefront of VCS network priorities.

We appreciate the author’s reflections and agree that patient evidence must be valued and practically supported in the context of health technology evaluations. Our editorial aimed to contribute to that conversation by clarifying how NICE is developing best practice recommendations to signal what good looks like and encourage transparency as part of its broader commitment to rigorous, inclusive and accountable decision-making.

The authors received no financial and/or material support for this research or the creation of this work.

The authors have no competing interests or relevant affiliations with any organization or entity with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.

No funded writing assistance was utilized in the production of this manuscript.

This work is licensed under the Attribution-NonCommercial-NoDerivatives 4.0 Unported License. To view a copy of this license, visit https://creativecommons.org/licenses/by-nc-nd/4.0/