Skip to main content
Free access
29 May 2018

Multi-stakeholder engagement in health services research

Multi-stakeholder engagement (Box 1) in the area of health services research ensures that the patient's voice is heard and that stakeholders such as community leaders, clinicians and health advocates are brought to the table. Multi-stakeholder engagement brings new perspectives into the decision-making process in the health and research arenas. This article presents a discussion of the phenomenon, considerations for convening a multi-stakeholder advisory group, engaging stakeholders and issues to consider to maximize the productivity of a multi-stakeholder advisory body.

Multi-stakeholder engagement: getting it right

Multi-stakeholder engagement as a concept is not new, but it is now being used in creative ways to ensure that the voice of the patient is included and heard in the research setting. Also, when key decisions need to be made regarding research, a variety of stakeholders are at the table in a fashion not seen in the past. In this way, all parties involved in the healthcare processes are able to make informed healthcare decisions [3]. This addresses the issue of missed opportunities to create meaningful decision-making in the current health model. The formation of multi-stakeholder advisory groups, committees or boards (used interchangeably in this article) is a key part of the bidirectional learning process in the healthcare research setting (Box 2).

What is a multi-stakeholder advisory group?

A multi-stakeholder advisory group in patient-centered outcomes research should be comprised of anyone who can influence outcomes or the patient decision on health, such as clinicians, payers, patients, caregivers, professional organizations or advocacy groups. The advisory group integrates a spectrum of perspectives influencing a health outcome. Benefits from multi-stakeholder advisory groups include obtaining insight from participants with different backgrounds and perspectives [4–9].

How to select a multi-stakeholder advisory committee?

In selecting members of a multi-stakeholder advisory group or committee, one should consider who would be interested in the topic of study, who would have expertise in the topic or who would be affected by the topic. One should reflect on what you are trying to accomplish and whose perspectives would be invaluable. Ensure there is, at minimum, representation from patients, patient groups, healthcare professionals and payers. Consider who might be needed to help disseminate results and who will help implement the study.

Assuring balance & diversity on the multi-stakeholder advisory committee

In order to assure balance and diversity on a multi-stakeholder advisory committee, you should ensure that enough patients are represented – the gold standard is usually 25% of the advisory group. It is important to think about having a diversity of experiences, perspectives, values and preferences. Where feasible, ensure a good representation of age, gender, sexual orientation, race, ethnicity, language and religion. It might also be valuable to consider different stages or variants of the clinical condition (e.g., stages of cancer or type of diabetes or the cause of pneumonia). The diversity on the committee should mirror the demographics of the patient population in the study. Bear in mind that a diversity of clinician specialty may be needed.
To further assure balance and diversity, reach out to communities impacted by the health issue under study in selecting advisory group members. It can be helpful, if committee size permits, to involve patients who have had positive experiences with treatment and those with negative experiences. Make sure every voice is heard by using a moderator. It is important to make accommodations for people with special needs.
Recently, I was involved in assembling a stakeholder advisory board for an effort to establish what is being called a ‘Learning Healthcare Community’. Diversity on the board was important and was considered in terms of race, provider/recipient of care, community leadership and familiarity with the region under study. Guidance from such a diverse body has proven extremely useful in working with the West Baltimore community that will be served by the project.

Roles & shared decision-making

Make sure that everyone on the advisory group knows what is expected of them (share ground rules) upfront for the decision-making purpose. From the start, everyone should know their role or job description so they know what is expected. Keep in mind that roles may change during the course of the research study.
Start every meeting with an overview of aims and objectives, ‘why we are here?’. There should be as close to consensus as possible in the group when making decisions. Support the majority view and ensure the view of the minority is also heard. Consider that it may not always be possible to implement recommendations of the group; so always provide feedback regarding why a decision was made in a particular way. Everyone should have the opportunity to voice his or her view in the decision-making process.
Remind participants that they represent one of many perspectives at the table. Get a sense of each participants’ strengths and engage them through those strengths. Make sure everyone is involved.

Orienting members

The first meeting of an advisory committee should be an orientation to cover roles and responsibilities. Have an orientation phone call with members who are joining midstream, prior to their coming on board with members who have already had orientation. The patients can have a separate orientation prior to the first advisory committee meeting, if they are not used to working in a research setting.

Responding to turnover

Recognize that turnover can be a challenge or a welcome opportunity for change. Investigate whether turnover is a result of the project not going well. If a patient on your committee steps down, replace him or her with another patient. Get input from stakeholders to recommend a new member of the group.

Authentic engagement of stakeholders

Ensure transparency within the group. Remind everyone to be respectful. Be clear on motives for bringing a stakeholder on board. It is important for researchers to be humble and have an open mind (be ready to actively listen and act upon the participant contribution). Speak in language everyone can understand (i.e., use layman terms) and allow people to ask questions. Ask if everyone in the group is up to speed with the discussion. Get to know people outside of the group setting and make it a point to meet them on their turf.
Show stakeholders that you value their input. Engage stakeholders before critical decisions are made. Be respectful of people's time – do not meet about topics that you could have handled by email. Let participants know how their input has influenced the project or process. Give them feedback on how you carried out their recommendations from previous meetings or not. Create an atmosphere in which stakeholders can feel comfortable.

Updating members in-between meetings

Updating members between official meetings provides a space to disseminate updates and better utilize meeting and/or phone time for bidirectional conversation. Use newsletters, a listserv, phone calls and emails preferably without attachments (attachments are less likely to be read). If feedback is needed between meetings, use surveys or polls.

Potential benefits of multi-stakeholder engagement

Multi-stakeholder engagement realigns healthcare research with needs of various parties: clinicians, patients, policymakers, and payers [2]. The contributions of relevant perspectives make for a better study outcome. Helpful views are brought to the table, with more input into decisions. Multi-stakeholder engagement helps with recruitment of participants for the study. You get buy-in from different stakeholders and ultimately less pushback. It allows you to gain support of community partners, if that were who you engaged (Box 3).

Assessing the impact of multi-stakeholder engagement

The numerous benefits of multi-stakeholder engagement should be tracked – document everything! Keep track of what would have been done and what was done based on stakeholder input. Track how often stakeholders participate and if they give feedback. In assessing the impact, make sure you are following principles of effective evaluation. Measure impact on things, the multi-stakeholder committee was charged with accomplishing. Document baseline assumptions and compare with future results. Track recommendations and dissemination. Survey stakeholders and ask how they think you are doing. Consider developing a standard evaluation questionnaire to answer the following questions:
Were voices heard?
Was facilitation effective?
Was there any change in the study based on stakeholder recommendations?
Did the study get completed on schedule?

Administrative & financial issues of multi-stakeholder engagement

It is helpful to have a point of contact to handle in-person meetings. When scheduling the meetings, take care in coordinating schedules with varying time zones. At the meetings, make sure there are options for varied modes of communication for stakeholders (emailed comments, phone call-in, in-person meeting, etc.).
Budget for the time of the stakeholders. Compensate patients as well as other participants. Some participants may opt out of payment, especially if it would cause an actual or apparent conflict of interest with their employer. It is common to offer all participants US$500 a day for each of four meeting days a year. Think of staff resources. Budget approximately US$30,000 for each in-person stakeholder meeting to cover all expenses including travel.

Some common challenges

Challenges encountered in multi-stakeholder engagement include the difficulty in putting together a truly diverse group. For example, in the ‘Learning Healthcare Community’ stakeholder advisory board mentioned above, although we had a very diverse board it was pointed out that we lacked the voice of the youth in the community.
Another stumbling block is that often the stakeholders are employed full-time with another organization and have time constraints hampering their involvement on a board. Taking actions such as having the stakeholders pick their own meeting dates can be helpful in addressing this matter.
Also, it can be quite costly holding in-person meetings of the board. So it is helpful to make good use of emails, webinars, surveys and other tools so that only crucial topics are handled in-person and you can minimize the frequency of in-person meetings.


It is important not to underestimate the value of multi-stakeholder engagement in healthcare research. Selecting a diverse team of stakeholders, orienting them in their responsibilities and making sure that all input is valued are keys to ensuring that all are heard, inspired and empowered to transform the conduct of research. Stakeholder advisory groups have the potential to minimize waste of time and resources in research. All-in-all multi-stakeholder engagement lends itself to better science and improved health of the public.

Future perspective

The author envisions a future of healthcare and research in which patients and other stakeholders are engaged throughout the life cycle of healthcare or service provision to inform the process and optimize services provided therein.
Box 1. What is multi-stakeholder engagement?
Stakeholder: an individual or group [in health service research] who is responsible for or affected by health-and healthcare-related decisions that can be informed by research evidence [1].
Engagement: a bidirectional relationship between stakeholder and researcher that results in informed decision-making about the prioritization, conduct and use of research.
Stakeholder engagement [2]: an iterative process of actively soliciting the knowledge, experience, judgment and values of individuals selected to represent a broad range of direct interests in a particular issue for the dual purpose of creating a shared understanding and making relevant, transparent and effective decisions.
Box 2. Who is a research stakeholder?
‘‘A stakeholder for a research study is simply anyone who cares about the outcomes or processes of the study and is going to use the evidence that comes out of the study to inform healthcare a stakeholder board is simply a collection of those different perspectives that advises the study team throughout the course of study’’
Ellen Tambor, Senior Research Manager, Center for Medical Technology Policy
Box 3. The value of stakeholder engagement.
Stakeholder engagement may improve the relevance of research questions, increase the transparency of research activities and accelerate the adoption of evidence into practice [1].


The author gives special thanks to CD Mullins and the PATIENTS Program team (University of Maryland School of Pharmacy), E Tambor (CMPT) and D Lavallee (University of Washington) for their contributions to this article.

Financial & competing interests disclosure

The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.
No writing assistance was utilized in the production of this manuscript.


Concannon TW, Fuster M, Saunders T et al. A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research. J. Gen. Intern. Med. 29(12), 1692–1701 (2014).
Deverka PA, Lavallee DC, Desai PJ et al. Stakeholder participation in comparative effectiveness research: defining a framework for effective engagement. J. Comp. Eff. Res. 1(2), 181–194 (2012).
Mullins C, Abdulhalim AM, Lavallee DC. Continuous patient engagement in comparative effectiveness research. JAMA 307(15), 1587–1588 (2012).
Kauffman KS, Dosreis S, Ross M, Barnet B, Onukwugha E, Mullins CD. Engaging hard-to-reach patients in patient-centered outcomes research. J. Comp. Eff. Res. 2(3), 313–324 (2013).
Mullins CD, Vandigo JE, Zheng J, Wicks P. Patient-centeredness in the design of clinical trials. Value Health 17(4), 471–475 (2014).
Chen J, Mullins CD, Novak P, Thomas SB. Personalized strategies to activate and empower patients in health care and reduce health disparities. Health Educ. Behav. 43(1), 25–34 (2016).
Mullins CD, Onukwugha E, Cooke JL, Hussain A, Baquet CR. The potential impact of comparative effectiveness research on the health of minority populations. Health Affairs 29(11), 2098–2104 (2010).
Sofolahan-Oladeinde Y, Mullins CD, Baquet CR. Using community-based participatory research in patient-centered outcomes research to address health disparities in under-represented communities. J. Comp. Eff. Res. 4(5), 515–523 (2015).
Mullins CD, Wingate LT, Edwards HA, Tofade T, Wutoh A. Transitioning from learning healthcare systems to learning healthcare communities. J. Comp. Eff. Res. (2018) (Epub ahead of print).

Information & Authors


Published In


Received: 16 March 2018
Accepted: 6 April 2018
Published online: 29 May 2018


  1. health services research
  2. multi-stakeholder engagement
  3. patient-centered outcomes research
  4. PCOR
  5. stakeholder advisory board
  6. stakeholder advisory group



E Olayinka Ladeji [email protected]
1Pharmaceutical Health Services Research Department, School of Pharmacy, University of Maryland, Baltimore 220 Arch St., Saratoga Building, Baltimore, MD 21201, USA


*Author for correspondence: [email protected]

Metrics & Citations


Article Usage

Article usage data only available from February 2023. Historical article usage data, showing the number of article downloads, is available upon request.

Downloaded 88 times


How to Cite

Multi-stakeholder engagement in health services research. (2018) Journal of Comparative Effectiveness Research. DOI: 10.2217/cer-2018-0026

Export citation

Select the citation format you wish to export for this article or chapter.

Citing Literature

  • Preparing newborn screening for the future: a collaborative stakeholder engagement exploring challenges and opportunities to modernizing the newborn screening system, BMC Pediatrics, 10.1186/s12887-021-03035-x, 22, 1, (2022).
  • The impact of panel composition and topic on stakeholder perspectives: Generating hypotheses from online maternal and child health modified‐Delphi panels, Health Expectations, 10.1111/hex.13420, 25, 2, (732-743), (2022).
  • Patients’ early engagement in research proposal development (PEER-PD): patients guiding the proposal writing, Journal of Comparative Effectiveness Research, 10.2217/cer-2018-0129, 8, 6, (441-453), (2019).

View Options

View options


View PDF

Get Access

Restore your content access

Enter your email address to restore your content access:

Note: This functionality works only for purchases done as a guest. If you already have an account, log in to access the content to which you are entitled.







Copy the content Link

Share on social media