Skip to main content
Search
Search
Advanced Search
Skip main navigation
The Evidence Base Post

CADTH joint final guidance on reporting real-world evidence studies in Canada

  • Joanne Walker

CADTH, in association with Health Canada and INESSS, has produced the final guidance for regulatory and health technology assessment agencies in Canada to advance the integration of real-world evidence in healthcare decision making.

Here we provide a background on the development of the final guidance and summarize the key recommendations made.

Growing need for RWE guidance

CADTH (Canadian Agency for Drugs and Technologies in Health), along with Health Canada and the Institut national d’excellence en santé et en services sociaux (INESSS), has announced the publication of a new guidance document, called ‘Guidance for Reporting Real-World Evidence.’ The final guidance, published on 16th May 2023, provides key considerations for sponsors to provide transparent and consistent reporting of real-world evidence (RWE) studies to effectively inform regulatory and health technology assessment (HTA) decision-making in Canada. Its release comes following an extensive stakeholder engagement and consultation period and coincides with the 2023 CADTH Symposium. The new guidance reflects the agencies’ recognition of the growing importance of RWE in healthcare and follows a number of other initiatives released globally, including the US FDA Real-World Evidence (RWE) Program, UK’s NICE real-world evidence framework and China’s National Medical Products Administration (NMPA) interim ‘Guidance for Real-World Data Used to Generate Real-World Evidences’.

Commenting on the guidance and speaking to The Evidence Base, Karen Facey (University of Edinburgh and RWE4Decisions) explained:

“I’m really excited to see this comprehensive but practical guidance from CADTH on how to report RWE studies that are pivotal to HTA. This will be a valuable tool not just for health technology developers and data analysts, but also for HTA bodies in other countries. I look forward to seeing how the considerations and recommendations can be applied to other jurisdictions in Europe.”

Development phase

To create the 66-page guidance document, CADTH and Health Canada established the RWE Steering Committee, including regulators, HTA organizations, methodologists, and pan-Canadian data holders, tasked with developing a strategic framework to implement RWE in healthcare decision making as well as guiding and supporting stakeholders to facilitate the optimal integration of RWE. The draft guidance was opened for public consultation in November 2022 and generated feedback from more than 50 organizations, the majority of which came from the pharmaceutical industry, patient communities and commercial RWE service providers. The resulting final guidance has been revised and modified to take into consideration many of the aspects raised during the stakeholder feedback process.

Goals and objectives

The goal of the guidance is to provide a ‘comprehensive, credible and fit-for-purpose reporting guidance that aims to harmonize current RWE principles for Canadian HTA agencies and regulators whilst maintaining alignment with international standards’. To achieve this, CADTH, Health Canada and INESSS set three goals to:

  1. Ensure regulatory and HTA agencies have sufficient information to evaluate a study for its appropriateness of use to inform decision-making.
  2. Provide core reporting standards for RWE studies that align with global standards.
  3. Prioritize transparency in reporting while accounting for practical challenges related to RWD and RWE.

They also set a further two specific objectives to:

  1. Identify existing global guidance on principles and standards for reporting on RWE studies to create an initial draft of Canadian RWE reporting standards that align with international standards.
  2. Establish consensus on items to be included in the core reporting standards for Canadian RWE studies through engagement with national and international experts in RWD and RWE.

Guidance recommendations

The guidance sets out 12 main recommendations that lay the foundation for reporting of RWE studies. Each recommendation is briefly summarized below; readers are encouraged to read the full guidance document for a comprehensive description of each recommendation and their detailed requirements.

Research Questions and Study Design – to include clear reporting on all aspects of the RWE study including aim, rationale, methodology, role of study team including patient partners, ethics, governance and funding.

Setting and Context – to clarify the setting in which the data are collected, the study period, any missing data, and circumstances for using RWE from sources outside Canada.

Data Specifications, Access, Cleaning Methods, and Linkage – to provide transparent information on the provenance of the RWD sources and their limitations, including the access to the database population, data cleaning methods, characteristics of data quality and methods and characteristics of any data linkage.

Data Sources, Data Dictionary, and Variables – to establish clear reporting of the methods used to define study variables, such as details of and justification for the specific database used, the data extraction criteria, sources of data for each variable of interest and a data dictionary.

Participants – to understand the generalizability of the study population to the real-world target population by incorporating details of the inclusion/exclusion criteria, study population characteristics, number of participants and any missing data.

Exposure Definitions and Comparators – to explain the exposure being studied (limited to drug and medical device exposure in the guidance) by clarifying the requirements for a patient to be classed as exposed, the data sources for this information, details of the comparator (where applicable) and how any changes in use of the exposure and comparator over time may affect study results.

Outcomes – to ensure the outcomes studied are relevant to the study question by including the study outcomes and definitions, references on the validity of outcome definitions, relevance of the study outcomes to real-world practice and considerations of outcome misclassification and the accuracy of outcomes timing in relation to exposure to the treatment(s) of interest.

Bias, Confounding, and Effect Modifiers or Subgroup Effects – to overcome any issues that may arise from bias and confounding by explaining how potential sources of bias could influence the outcome of the study and how these were overcome, any confounders factors, the methods used to conduct sensitivity analyses and any known or hypothesized effect modifiers.

Statistical Methods – to ensure transparency in the statistical methods used to generate results, details of the software used, statistical methods, statistical code and thresholds of statistical significance are amongst some of the considerations that must be given.

Study Findings – to facilitate accurate and transparent reporting of the study results relative to the study objectives by summarizing main analyses for each objective, providing details of each outcome and avoiding selective reporting of results.

Interpretation and Generalizability – to provide an interpretation of the study results and their applicability to the general Canadian patient population by discussing the primary, secondary, and exploratory study finding, including adjusted and unadjusted analyses.

Limitations – to include details of the limitations of the study in terms of sources of and missing data, bias and confounding, sample size limitations, clinically meaningful interpretation of the results and whether any of these factors influences the credibility of the study results.

Next steps

In the document, CADTH, Health Canada and INESSS explain the guidance comes at a time when RWD and RWE in decision making is at a turning point and continously changing and evolving. They anticipate the guidance will be a ‘living document’ to be updated, revised and extended as the fields evolve over time.

Guidance for Reporting Real-World Evidence lays the foundation for the use of RWE in regulatory approval and Health Technology Assessment (HTA) in Canada, starting with the principles for reporting of RWE studies. CADTH, Health Canada, and INESSS intend to use the guidance as appropriate for their individual needs, aligned to the principles outlined in the document. This initiative forms the foundation for transparent reporting of RWE studies in Canada and facilitates appraisal of RWE for the purpose of supporting decision-making.