Positive deviance approaches to improving vaccination coverage rates within healthcare systems: a systematic review

We searched English language articles in Medline, Embase, Cochrane Library, CINAHL and PsycINFO without any date restrictions on 4 October 2017. We also searched the references of included studies to find additional studies. An outline of the search strategy for PubMed is shown in the protocol, which is attached as an appendix. It was modified and adapted for various databases. Additional searches were carried out by manually reviewing the reference lists of included studies.

We carried out a systematic review according to a prespecified protocol. The protocol was registered at PROSPERO (CRD42017078221). Details of the search are available from the protocol. We followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist for reporting the results [7].

We included studies of any design (e.g., qualitative, quantitative and mixed method) without any restrictions on study design, if the study has reported on the use of positive deviance approaches for vaccination among children and adults. We excluded narrative reviews and studies that did not use the positive deviance approach or address the outcome of vaccination.

We used a broad definition of the positive deviance approach, which relied on the definition in the primary studies. Positive deviance approaches may have different names (e.g., bright star) and all such studies were eligible for inclusion. We did not limit ourselves to high performers but also evaluated studies that reported on low-performing providers and health systems.

We conducted a qualitative synthesis of data to identify potentially effective strategies for improving vaccination coverage using the positive deviance approach. We also attempted to determine the quality of such studies.

Studies were selected for inclusion based on the review of titles and abstracts that were exported into a web-based data management repository (www.rayyan.qcri.org) [8]. Articles, titles and abstracts were screened by two authors independently for potentially eligible articles. Any discordance between the authors over the eligibility of any article were resolved by reference to the full text, and all discrepancies were resolved by agreement after rechecking the source papers, further discussion among the reviewers, and full consensus prior to inclusion. Two reviewers were independently involved in all stages of study selection, data extraction and quality assessment. We extracted data on all relevant outcomes for each of the research questions, including that relating to study design, the location of the study, the study characteristics and the stages of the positive deviance approach. We extracted data on the major themes onto an iterative framework that listed the various strategies used to improve vaccination rates as a part of positive deviance approach. We also evaluated data on the results of the risk of bias assessment.

We used the 16-item QATSDD tool [9], scored on a Likert scale of 1–4 to evaluate the risk of bias. This tool shows good reliability and validity in the quality assessment when studies included in the systematic review have diverse study designs.

All data were extracted into a prespecified Excel data sheet. We compiled all strategies for improving vaccination coverage and created an organizing framework of strategies using thematic analysis. We did not conduct a separate analysis of strategies identified by high performers to low performers because the primary studies did not report the data to enable such comparisons. We extracted the strategies as reported in the results section of each study and compiled a list of all strategies identified with examples. The core strategies identified represent a compilation of successful strategies identified in studies that included both high- and low-performing sites and high-performing sites only.

We screened 241 citations and identified eight studies that were eligible for inclusion [10–17]. The process of study selection is shown in Figure 1. The baseline characteristics of the studies are shown in Table 1. Five studies were conducted in the USA [10,11,15–17], two studies in Africa [12,13] and one study in the UK [14]. Similarly, differences in vaccine characteristics (number of doses required, seasonality, response to epidemic, availability) also resulted in differences between studies. The settings varied, from evaluating programs in practice sites [14,17], public vaccination clinics [10], to evaluating programs at district levels [12], local health departments [11] and health plans [15]. These studies evaluated a range of vaccines including H1N1 vaccination among school children [10,11], Diphtheria pertussis and tetanus vaccination [12,13], influenza vaccination [14], human papilloma virus (HPV) vaccination [15] and vaccination for children diptheria pertussis tetanus (DTP)/diptheria tetanus acellular pertussis (DTaP), polio, measles mumps rubella (MMR), Hib, HepB [17]. All studies identified positive deviants and generated hypotheses about how such deviants succeed, although not all studies used an objective performance measure to identify high performers [10–17]. We did not identify any studies that tested hypotheses quantitatively or addressed the dissemination stage of the positive deviance approach.

The quality assessment of the included studies using the QATSDD tool [9] was variable with incomplete reporting of several data elements as shown in Table 2. Most studies stated their aims and objectives and described the research setting. Only four studies provided a rationale for sample size [10,11,13,14]. Details of the recruitment process were only available from four studies [14–17]. These reflect a limitation in the reporting of studies rather than a judgment of its quality. Although most studies outlined a process of identifying positive deviants, this was not sufficiently detailed. Two studies described this in more detail but selected positive deviants using a subjective approach. Most studies did not clearly describe the process of selecting positive deviants or include detailed questionnaires or interview guides [10,11]. Only the Naimoli study measured routine vaccination coverage that is considered one of the best approaches to measure effectiveness of a vaccination strategy.

The studies were conducted in heterogeneous settings with differences in contextual factors, system characteristics, vaccines of interest and the population targeted. However, despite these differences, several broad themes emerged. We characterized these themes as strategies and factors related to the process of vaccine delivery, supporting structural elements and contextual factors. We present a model of the potential relationship between these elements in Figure 2.

Key elements that characterize the process of vaccine delivery among high performers were found consistently across studies and can be categorized as: planning, raising awareness of vaccination, vaccine delivery and vaccine program evaluation as shown in Table 3. Many studies described collaborative planning with stakeholders and community partners as important to achieving successful vaccination programs. Approaches to increasing awareness and acceptance of vaccination among the target population included the need for tailored, targeted and deliberate messaging designed to overcome barriers to vaccination. Targeted communication is intended for a subgroup of participants based on participant characteristics shared by the group; whereas tailored communication is intended to reach one specific individual [18]. In the context of vaccine delivery, targeting refers to increasing the immunization level of the population by delivering vaccines to groups at the highest risk.

Several studies described the benefits of engaging a variety of people in delivering this message, and particularly the value of including trusted and familiar individuals such as community health workers or well-known organizations to make the message more persuasive. Finally, most studies identified innovative and multimodal approaches to delivering vaccination-related messages and reminders to the target population as important features of high performers. Strategies to optimize the delivery of the vaccine used by high performers include tailoring clinic hours and locations to the needs of the target population, and policies to make vaccination a default option. Examples of this include use of standing orders and providing vaccination ‘opportunistically’ at any visit. Half of the studies described the use of regular program evaluation that often included efforts to identify and track unvaccinated individuals to inform future planning.

Structural elements such as staffing and vaccine supply are necessary, but not enough to distinguish improvers, according to LaFond et al. [12]. Such elements could include appropriately trained staff [12], dedicated staff toward vaccination campaigns [16] and/or enough staff or volunteers to ensure vaccination [10]. Other resources include ensuring the cold chain supply, an adequate and reliable vaccine supply, transportation and financing [12].

Identifying contextual factors and mechanisms (strategies) that lead to success is an overarching goal of positive deviance. While we did not parse out each strategy and theme and the context in which it worked well, the setting and context for each study is shown in Table 1. Three studies [10–12] used a realist evaluation to learn about the combinations of context and mechanisms that led select schools, health departments or districts to perform well. This enabled them to not only describe which approaches worked well but also in which contexts they did so. The remaining studies reviewed took different real-world approaches to doing that and achieved variable results in identifying interactions between context and strategies.

There were several contextual factors, including community engagement that were identified as important strategies in the included studies. Community engagement was an important factor noted by LaFond et al., who engaged a cadre of community health workers to ensure vaccine access for the community [11]. Klaiman et al. engaged the community to reach vulnerable populations and ensured that community providers supported the setting up and taking down of clinics [10]. Efforts at community engagement included establishing relationships with school authorities to set up clinics [11].

The role of leadership and organizational climate was identified as important to high-performing status in several studies [10,12,17]. There were several characteristics of leaders that ensured success. These included leadership styles which prioritized vaccination [14,16,17]. Some leaders administered the vaccine themselves, participated in skits promoting the vaccines and empowered vaccination campaigns via town hall meetings and larger events. The leaders at high-performing sites were noted to be tenacious, and a single leader was responsible for performance, as compared with fragmented leadership with no clear lead at low-performing sites [14]. Wishner et al. noted that there were a greater number of leaders with dominant managerial styles at above-average performing sites [17]. These leaders provided a clear vision and mobilized people to achieve this vision.

Two studies identified organization flexibility as an important feature of high performers. For example, Klaiman et al. noted the importance of flexibility in decision-making, staffing and clinic implementation [11]. Wishner et al. also noted that organizational climates that fostered a higher sense of responsibility among employees were associated with higher performance among small practices [17]. One study reported on team characteristics of high performers where team members worked together closely, had frequent communications, established close working relationships and were empowered to find solutions [16].

We found that a small but diverse group of positive deviance studies have been used to identify effective strategies for facilitating the uptake of vaccines. These studies focused on settings ranging from the clinic, community, public health organizations, health plans and national policy settings. Despite this diversity, we identified several core strategies and approaches that appear to support vaccine uptake in real-world settings. Such strategies may be helpful to organizations seeking to improve vaccination rates.

One of the key findings from these studies was the importance of tailoring and targeting in both messaging and delivery of vaccines as well as the importance of tracking delivery of vaccines. Targeted communication is intended for a subgroup of participants based on participant characteristics shared by the group; whereas tailored communication is intended to reach one specific individual. In the context of vaccine delivery, targeting also refers to increasing the immunization level of the population by delivering vaccines to groups at highest risk.

Our findings suggest the importance of patient and provider education, reminders, feedback loops, community collaborations, immunization registries and use of the medical home – a comprehensive patient or family-centered approach to primary care. Although the specific strategy used will depend on the local context, the findings from our review can be used to inform planning and implementation of efforts to improve vaccination. These findings are consistent with the understanding that there is rarely a ‘one-size-fits-all’ solution to complex problems such as increasing vaccination rates.

There is a striking similarity on the types of strategies to support vaccination identified via the positive deviance approach and those proven to be effective in interventional studies. For example, our systematic review identified strategies comparable to those identified in a systematic review of 183 studies of interventions to improve vaccination among children, adolescents and adults in the USA [19]. That study categorized interventions under three headings: increasing community demand, enhancing access to vaccination services and provider-based interventions. There was evidence to support that community demand can be increased through client reminder or recall, the use of educational interventions, vaccination requirements for child care, school or college attendance. There was also evidence to support enhancing access to vaccination services through reduction of out of pocket costs, expanding access to healthcare settings and home visits and vaccinations in schools. Strong evidence supports the use of provider-based interventions including reminders to providers, assessment of and feedback to providers, and standing orders for vaccinations in adults. There is a striking similarity in the types of strategies to support vaccination identified via the positive deviance approach and those proven to be effective in interventional studies. This suggests that many of the strategies identified by studying high performers may in fact be effective despite the lack of quantitative hypothesis testing in these studies and supports the approach of positive deviance as an efficient means to identify effective real-world strategies. However, it is possible that some of the strategies used by high performers were informed by evidence generated from randomized controlled trials.

The set of interventions that were successful in our review is largely like those recommended by the CDC guidelines because they were likely informed by these guidelines or the underlying evidence. For example, many of the processes of vaccine delivery used by high performers (Table 3) included approaches outlined in the CDC Task Force on Community Preventative Services, such as prompts and reminders and reducing logistical barriers to vaccination. However, our study also identified novel contextual factors that are hypothesized to contribute to successful vaccination coverage, such as community engagement, teamwork, organizational climate and leadership characteristics, illustrating the potential value of the positive deviance approach.

There are several strengths to the positive deviance approach. The strategies are practical, common sense and doable with the resources that most organizations have available. It is neither practical nor feasible to conduct a randomized controlled trial of each strategy or variation identified in these studies, but the approaches listed above provide a starting point for organizations to consider incorporating into their local approaches to vaccination. We also noted variation in the quality of studies which was anticipated. In addition, several of the studies lacked specificity or details on their approach. The process of identifying high performers via peers rather than by measures of actual performance is a limitation of some studies [10,11]. Very few studies involved multiple types of stakeholders (such as patients) or collect data at multiple levels. For example, the study on HPV vaccination collected data at only one level, interviewing medical directors and quality improvement staff of health plans [15]. We could not comment on the effectiveness of these strategies, or the value of dissemination since none of the studies quantitative conducted hypothesis testing or dissemination.

There are limitations to our review. Although we conducted a comprehensive search, it is possible that we may have missed few studies because the terminology used to describe positive deviants is somewhat inconsistent. We did not include non-English studies which may have biased our results. Few studies were conducted in the USA, which limits the generalizability of the findings from studies conducted in the UK or Africa to the USA where practice patterns are different. It is possible that we missed subsequent hypothesis testing studies, which may not be reported as such, or labeled as positive deviance studies. We cautiously consider these strategies to be potentially effective strategies for several reasons. Only one study reported vaccination as a measure of effectiveness. Furthermore, the findings from such qualitative studies should only be considered hypothesis generating and will require testing in larger samples to generate conclusive results on effectiveness.

Despite these limitations, there are several strengths to our review. We conducted a comprehensive search according to a prespecified protocol and included studies that addressed diverse approaches to vaccination. We were able to identify themes that are applicable in real-world settings and have face validity. These can be used to inform future strategies to improve vaccination.

Future studies should clearly define the approach used to identify positive deviants and should use objective measures of performance. Without an objective means of selecting high performers, the implications of the identified strategies are difficult to interpret and replicate. A recent study that sets forth criteria for assessing suitability of measures of hospital quality and safety as a good first step toward standardizing selection of indicators to use to select positive deviants could serve as a useful guide [20].

Success may be more broadly defined and conceptualized than simply high vaccination coverage rates when conducting such studies. It is possible that some of the parameters of ‘success’ identified in our review (minimizing disruption, costs, parental concerns) may be features of programs that effectively promote vaccination coverage.

Five studies included in our review included low performers [12–14,16,17]. Researchers should also consider including low performers to identify those characteristics that differentiate high performers from low performers, and also shed light on ineffective strategies.

 Future studies, including systematic reviews, could also evaluate whether effective strategies that improve vaccination in randomized controlled trials are similar or dissimilar from those identified using the positive deviance approach.