Factors associated with evidence-based decision-making among patients and providers

We contracted with Marketing Systems Group to conduct an online survey through the vendor Luth Research, which maintains an opt-in online panel of approximately 2 million people. Luth drew a sample of 1678 adults who self-reported having one or more of the four conditions most commonly reported in the Luth panel – hypertension, Type II diabetes, high cholesterol or fibromyalgia. Patients with chronic conditions are currently under medical care, and therefore CER is relevant for them. Luth sent invitation emails and up to three reminder emails to the sample. The invitations and reminders informed prospective respondents that participation in the survey was voluntary, survey responses would be kept confidential and survey participation would be anonymous; the survey instrument reiterated the assurances of confidentiality and anonymity. We fielded the online survey between November 2011 and January 2012, collecting data from 603 respondents; the response rate was 36%. The sample was 67% female; had a mean age of 52 years; and was predominantly white (86% white, 5% African–American, 4% Latino/a, 1% Asian and 5% multiple). Respondents were sent a US$20 check after completing the questionnaire.

The sample list for providers was leased from the vendor SK&A, a company that manages databases compiled from such sources as company and corporate directories, websites, state licensing information, telephone directories (white and yellow pages), and professional associations. SK&A staff conducts telephone verification calls to maximize the accuracy of the information. SK&A drew a sample of 5,670 physicians, nurse practitioners and physician assistants. We sent invitation letters by postal mail, invitation emails and up to three reminder emails to the sample. The invitations and reminders informed prospective respondents that participation in the survey was voluntary, survey responses would be kept confidential and survey participation would be anonymous; the survey instrument reiterated the assurances of confidentiality and anonymity. We fielded the online survey between November 2011 and January 2012, collecting data from 628 respondents; the response rate was 9%. The broader project also included a sample of healthcare administrators drawn by SK&A. Respondents were initially classified as patients, providers or healthcare administrators based on classification by the sample providers. 507 of those in the provider sample responded, resulting in a 9% response rate. However, respondents were realigned based on self-identification in the survey as healthcare administrators, providers (not also healthcare administrators) and patients (neither healthcare administrators nor providers). The final 628 providers in our analytic sample included 446 from the provider sample, 163 from the healthcare administrator sample and 19 from the patient sample. The sample was 42% female and had a mean age of 48 years. Respondents were sent a US$20 check after completing the questionnaire.

Because we collected the data for a broader project that examined differences among CER stakeholder groups, sample size was determined by a statistical power analysis of ability to detect differences between patients and providers. To enhance our ability to detect statistically significant effects, we conducted our power analysis using conservative assumptions, and therefore, did not need to conduct a power analysis for the current study's analyses. To have an 80% power to detect a difference of 0.083 at a 0.05 significance level for a variable that is a proportion of 0.5, the minimum sample sizes are 600 for patients and 553 for providers.

Because these were online surveys, we limited the sample frames to people for whom the sample vendors had an email address on file. After the data collection period ended, we calculated weights for the survey data to account for the probability of being selected into the sample. Weighting for the probability of selection accounts for the variation in selection probabilities in the sample. Because respondents may be different from nonrespondents on important variables of interest, weighting for nonresponse corrects for potential bias that may result. The probability of selection is the number of sampled cases divided by the sample frame size. The weight is calculated by dividing one over the probability of selection. For providers, the weight was calculated separately for each provider type because the sample frame differed for physicians, nurse practitioners and physician assistants. In addition to the weighting for selection probability, the sample weights also included poststratification adjustments for age, race and gender. Poststratification adjusts the sampling weights so that the joint distribution of a set of poststratifying variables matches the known population joint distribution. When the full joint distribution is not available, we use a raking (or iterative) procedure so that the final distribution of each weighting variable in the dataset matches the distribution of that variable in the population. Because of the relatively low response rates, we did not calculate weights for differential nonresponse. In addition, a distribution of chronic diseases in the US population is not available; therefore, although we poststratified both samples for age, race and gender, we could not poststratify the patient sample to reflect the age-race-gender distribution of patients with chronic diseases, which may be different from the distribution in the broader population.

In developing our survey instrument, we were guided by a conceptual framework that we developed, based on a generic KAB approach. The framework identified our domains – CER-related knowledge, self efficacy/skills, attitudes, opinions and behaviors – and key constructs within each domain. Our constructs were then refined based on feedback from a technical advisory panel composed of medical decision-making experts representing a diverse range of disciplines and professional affiliations.

We developed the English-language survey instrument following a review of existing literature and survey instruments, which yielded no existing items suitable or adaptable for our domains. Therefore, our survey items were newly developed but in a few cases informed by wording of phrases or terms in existing items. For many items, we developed parallel versions tailored to patients and providers; patient versions were designed to be appropriate for those with a lower reading level and avoided terminology familiar to providers but not to patients. Once we developed the draft survey instrument, our technical advisory panel provided focused feedback to address two challenges we had identified – ensuring that the parallel versions of items had the same meaning and ensuring that our knowledge items reflected consensus among experts and yet would result in a wide distribution of responses. We then cognitively tested the survey with seven individuals identified through leads from colleagues. The final instrument took 15–20 min for respondents to complete and received Office of Management and Budget approval (control number 201105-0990-004, expiration date 08/31/2014).

The patient version of the survey instrument assessed knowledge (one composite item), self efficacy/skills (one item), general attitudes (two items), specific opinions/beliefs (four items) and behaviors pertinent to CER (three items), as well as health status and demographics (education, age and gender). Figure 1 presents all of the variables used in the patient analysis.

The provider version of the survey instrument assessed knowledge (one composite item), self efficacy/skills (one item), general attitudes (two items), specific opinions/beliefs (four items) and behaviors pertinent to CER (two items), as well as demographics (age and gender). Figure 2 presents all of the variables used in the provider analysis.

We conducted data analysis using the IBM SPSS Statistics Version 22 software Complex Samples General Linear Model procedure, to apply the weighting for selection probability with poststratification adjustments for age, race and gender for the multivariate linear regressions. Separately for patients and providers, we regressed each behavior item on the remaining survey items using backward stepwise elimination, with the criterion for inclusion set at 0.05, listwise deletion of missing data, and two-tailed tests of statistical significance.

Although pairwise deletion of missing data would have retained more respondents in each regression model, using pairwise deletion in a multivariate analysis can be problematic, because different respondents contribute to different coefficients within the same model. Furthermore, the Ns in the results tables show that using listwise deletion did not greatly reduce the number of respondents included in each regression model, because relatively few respondents had any missing data.

We selected the backward elimination procedure specifically because it is well-suited to exploratory research, effectively eliminating the least informative variables and retaining the strongest variables in the regression model. However, the procedure is vulnerable to multicollinearity, resulting in unstable regression results. Therefore, we conducted checks to assess multicollinearity and stability of the results.

We also conducted standard checks to assess the extent to which the data met the nonautocorrelation, noncollinearity and normality assumptions of the linear regression procedure:

Tables 1, 2 and 3 present the regression results for each of the three behaviors (dependent variables). For all three behaviors, after eliminating variables not significant in the first step, all remaining variables were significant in the second step, so no further backward elimination was required. Among the three, the regressions show stronger associations for ‘use of information on different treatment approaches (R² = 0.20)’ and ‘use of research information to guide care decisions (R² = 0.23) than for bringing evidence to medical appointments (R² = 0.14)'.

All three behaviors are associated with patient perceptions, that is, some combination of self efficacy, attitudes and opinions.

Neither knowledge nor belief that CER will increase access to new treatments is associated with any of the three behaviors. These findings are not surprising, given that respondents had very low levels of CER-related awareness or knowledge, and therefore also likely did not have an informed belief regarding the impact of CER on access to new treatments.

Belief that doctors have the CER needed is associated with less frequent ‘use of research information to guide care decisions' and less frequent ‘bringing evidence to medical appointments’. That is, those who believe that doctors have the CER needed are less engaged, perhaps because they perceive evidence-based decision-making to be part of the physician's role rather than the patient's role.

For the most part, health status and demographics is not associated with any of the three behaviors; however, more educated respondents were more likely to ‘use research information to guide care decisions’, and older respondents were less likely to do so.

Tables 4 and 5 present the regression results for each of the two behaviors (dependent variables). For both behaviors, after eliminating variables not significant in the first step, all remaining variables were significant in the second step, so no further backward elimination was required. The regressions show strong associations for both ‘use of information on different treatment approaches' (R² = 0.36) and ‘use of CER to guide care recommendations' (R² = 0.29).

Both behaviors are associated with knowledge. Both are also associated with provider perceptions, that is, some combination of self efficacy, attitudes and opinions.

Neither belief that CER will reduce medical costs nor belief that CER will increase access to new treatments is associated with either behavior. In addition, neither age nor gender is associated with either behavior.