Using big data for quality assessment in oncology

Big data are powerful resources for assessing the safety of medical treatments on a large scale, which can reveal novel insights. In an observational study by Keating et al. [3] utilizing data from the Veterans Healthcare Administration, the authors found that prostate cancer patients treated with androgen-deprivation therapy (ADT) had higher incidences of diabetes and cardiovascular disease compared with patients not treated with ADT. Information on diabetes, coronary heart disease and myocardial function were ascertained using International Classification of Diseases, Ninth Revision (ICD-9) codes associated with inpatient and outpatient physician visits. This and other similar studies [4–6] have led to an increasing awareness of these potential effects of a common prostate cancer treatment.

The Surveillance, Epidemiology, and End Results (SEER) program of the National Cancer Institute (NCI) is a collection of population-based cancer registries across the USA, which provides patient information including: demographics, sex, age at diagnoses, tumor site and morphology, treatment and survival [7]. The population-based nature of this database makes it a useful tool for examining cancer incidence and mortality across the USA. A study by Darby et al. [8] used SEER data to assess long-term mortality from heart disease in 308,861 women who received radiotherapy for breast cancer. Comparing the risk of mortality from heart disease in women who received radiotherapy for right-sided breast cancer versus left-sided breast cancer (the latter group of patients more likely to receive incidental radiation dose to the heart), this study found that patients treated from 1973 to 1982 on the left side had a higher risk for cardiac mortality after treatment. This was one of the first large-scale studies to quantify the risk of cardiac sequelae due to incidental radiation doses to the heart from treatment of a common disease (breast cancer).

SEER linked with Medicare data (SEER-Medicare) additionally provide Medicare claims and diagnostic data for patients 65 years and older, and is another commonly used data resource to examine population-based patterns of care and patient outcomes. In addition to the information provided by SEER, linked SEER-Medicare data contain diagnostic and billing codes including tests, procedures, outpatient visits, hospital admissions and home healthcare for Medicare beneficiaries. Several studies summarized in Table 1 demonstrate how SEER-Medicare data can be used to assess safety. The first example, a study by Pinder et al. [9] assessed the incidence of congestive heart failure in 43,338 women following anthracycline chemotherapy for breast cancer. Congestive heart failure diagnoses were identified through claims in Medicare inpatient, outpatient and physician files. A second example, by Hershman et al. [10] determined the rate of cardiac toxicity in patients treated with doxorubicin-based chemotherapy for diffuse large B-cell lymphoma using methodology similar to the previous study. A study by Sheets et al. [11] compared the rates of gastrointestinal and urinary morbidity, erectile dysfunction and hip fractures associated with various types of radiation treatments for localized prostate cancer.

In 1999, the IOM published a report describing the discrepancy between the ideal cancer care system and that which was actually observed across the USA [33]. Among the list of described problems were underuse of cancer screening, lack of adherence to standards for diagnosis (inadequate biopsies, poor reporting of pathology studies), inadequate patient counseling regarding treatment options and underuse of adjuvant radiotherapy and chemotherapy following surgery in various cancers. Acknowledgement of these inconsistencies led to significant investments in research geared toward monitoring quality and clinical guideline adherence. Clinical practice guidelines are tools to help guide physicians and patients in diagnostic and treatment decisions. A commonly referenced set of guidelines in big data research are those created by the National Comprehensive Cancer Network (NCCN). Numerous published studies have utilized data from large cancer registries to examine patterns of care compared with ‘standards’ published in the NCCN guidelines. A few are summarized in Table 1 and below to illustrate.

In a study published by Gray et al. [12], the authors used the National Cancer Data Base (NCDB), a large cancer registry which includes ˜70% of incident cancer patients across the USA. This study examined how frequently patients with muscle-invasive bladder cancer received curative-intent treatments based on clinical practice standards set by the NCCN and European Association of Urology. This study showed that curative-intent treatment was significantly underused in elderly patients. This was attributable to older patients not being medically eligible for radical cystectomy, yet not being referred for radiotherapy which is another guideline-recommended treatment option. Another study, by Chen et al. [13], used SEER-Medicare data to examine the proportion of Medicare prostate cancer patients across the USA who received treatments consistent with the NCCN guidelines. Similar to the prior study, this one also demonstrated that a sizable proportion of patients received potentially inappropriate care. Specifically, this study found that for patients with high-risk prostate cancer, the most aggressive form of localized disease, guideline-concordance ranged from 66.6% (for patients aged 66–69 years) down to 51.9% (aged 75–79 years). Another example, by Chagpar et al. [14], used the NCDB to assess adherence to NCCN guidelines in colon cancer patients. Guideline adherence was based on whether the appropriate treatment had been recommended, independent of whether it was actually received. This study reported that patients with stage I colon cancer received guideline-concordant treatment recommendations more frequently (96%) than those with stage II (low risk 66%; high risk 36%), stage III (71%) and stage IV (73%) disease.

Large cancer registries in some individual US states and in other countries have also been used to examine patterns of care and guideline adherence. A study by van de Water et al. [15] assessed treatments received by breast cancer patients identified using The Netherlands Cancer Registry. Treatment was considered to be guideline-adherent based on Dutch recommendations for breast cancer management. Finally, Bristow et al. [16] assessed the frequency with which ovarian cancer patients identified from the California Cancer Registry received stage-appropriate surgical procedures and chemotherapy.

Big data can also be used to compare the effectiveness of different cancer treatments. A recent study used SEER-Medicare data to compare the outcomes of prostate cancer patients who received a robotic prostatectomy versus an older technique open prostatectomy [34]. For this comparison, big data were required because a randomized trial was not possible given the rapid diffusion of robotic surgery across the USA. This study showed that robotic versus open prostatectomy was associated with lower blood transfusion rates and slightly shorter hospital stays, but higher costs.

The complexity of the cancer care system can make it difficult for patients and their physicians to coordinate a comprehensive and patient-centered treatment plan. Patient-centered care should seek to involve patients, families and the treatment team including physicians in conversations to develop thorough treatment plans that integrate medical, emotional and social needs. Studies using big data can be used to assess indicators of patient-centeredness in oncologic care.

An example was a study published by Burge et al. [17], which utilized the Nova Scotia Cancer Registry to determine if greater family physician continuity of care provided to cancer patients at the end-of-life decreased emergency department utilization. It has been well documented that cancer patients generally prefer to spend their final days out of hospital [35–37], as trips to the emergency department can be emotionally, physically and financially burdensome. This study found that patients with low and moderate continuity of care were two- to four-times more likely to make visits to the emergency department compared with those with high continuity of care. In another study by Sharma et al. [18], the investigators used SEER-Medicare to quantify intensive care unit (ICU) usage during the last 6 months of life among 45,627 patients with advanced lung cancer. This study found that ICU usage during the last 6 months of life increased from 17.5% in 1993 to 24.7% in 2002 (p < 0.001); interestingly, there was also an increase in hospice usage during that same period (28.8–49.9%).

A study by Royce et al. [19] used data from the population-based National Health Interview Survey (NHIS) to examine the rates of prostate, breast, cervical and colorectal cancer screening in patients with limited life expectancy. The NHIS is an annual, large-scale survey conducted by the National Center for Health Statistics which is part of the Centers for Disease Control and Prevention that collects information on healthcare received by individuals across the USA; NHIS data are recognized as reflective of US population-based healthcare patterns. Although it is widely recognized that cancer screening in patients with limited life expectancy does not benefit (and likely harms) the patient [38], this study showed that a significant proportion of men and women with less than 5-year life expectancy across the USA received prostate (55%), breast (38%), colorectal (41%) and cervical cancer (31%) screening.

There is compelling evidence that delays between cancer diagnosis and treatment initiation are associated with poor patient outcomes [39–46]. It is therefore important to identify if cancer patients are receiving timely treatment, and big data resources are ideal for these studies.

In a study published by Fedewa et al. [20], the delay in receiving adjuvant chemotherapy for breast cancer was assessed using NCDB. Overall, 95.8% of women received adjuvant chemotherapy within 90 days following surgery, which is a commonly recommended time frame. Stokes et al. [21] used SEER-Medicare data to examine the time from prostate cancer diagnosis to initiating treatment. Somewhat surprisingly, this study showed that patients with high-risk cancer had a longer time from diagnosis to definitive treatment compared with low-risk cancer.

Studies have also assessed the impact of treatment delays on patient outcomes. For example, a study by Froud et al. [22] investigated the effect of time delay between breast-conserving surgery and adjuvant radiation therapy on local recurrence among 1962 women with breast cancer. Patients were identified from the population-based Breast Cancer Outcomes Unit Database which contains demographic, staging, treatment and outcome information for patients in the British Columbia Cancer Agency. This study found no difference in the rate of ipsilateral breast cancer recurrence on univariate or multivariate analysis comparing groups with 0–5, 6–8, 9–12 and ≥13 weeks between breast conserving surgery and initiation of radiotherapy.

Hollenbeck et al. [23] used SEER-Medicare data to determine the association between delays in bladder cancer diagnosis and mortality. The investigators identified 29,740 patients who had hematuria (the most common presenting symptom for bladder cancer) in the year preceding a bladder cancer diagnosis. These patients were stratified according to the time interval between hematuria and the bladder cancer diagnosis. The study found that patients who experienced a delay of 9 months or more between detection of hematuria and cancer diagnosis had higher mortality compared with patients diagnosed within 3 months. Another study by Hershman et al. [24] investigated the association between survival and time delay to adjuvant chemotherapy in colon cancer. This study found that a delay in the initiation of adjuvant chemotherapy by >3 months was associated with higher colon cancer-specific mortality and all-cause mortality.

The IOM estimates that US$765 billion of the healthcare budget – roughly 30 cents of every medical dollar – is wasted each year in the USA [47]. Big data can be used to assess cancer-specific medical costs in an attempt to understand the economic impact associated with cancer care. One such study by Warren et al. [25] accomplished this by using SEER-Medicare data to assess the cost of treatment for 28,916 elderly women with early-stage breast cancer in a fee-for-service setting. Although the initial cost of treatment was higher for women receiving breast-conserving surgery plus radiotherapy (BCT) versus modified radical mastectomy (MRM), there was no significant difference in long-term total care costs (US$77,106 for BCT vs US$75,897 for MRM). Another study by Yabroff et al. [26] also used SEER-Medicare data, and analyzed medical care costs of 718,907 cancer patients matched to 1,623,651 control subjects without cancer. This study found that the cost of cancer care varies widely by cancer type, with 5-year costs ranging from less than US$20,000 for breast cancer to over US$40,000 for brain, esophageal and ovarian cancers.

One way to assess potential waste in healthcare spending is to examine the adoption of newer, more expensive tests and treatments – specifically those that have not been proven to improve patient outcomes. Many studies have examined cancer care across the USA and consistently reported that newer and more costly technologies and treatments are often rapidly adopted for use before research has demonstrated their effectiveness [11,48–49]. In a study by Nguyen et al. [27] using SEER-Medicare data, the cost associated with adoption of newer technologies in prostate cancer led to an increase in Medicare spending of US$350 million annually. Waste in healthcare can also be assessed by calculating the costs associated with unnecessary tests. Falchook et al. [28] examined this question, assessing the use of unnecessary computed tomography (CT), MRI and bone scans during staging work-up in patients with low- and intermediate-risk prostate cancers. For these patients with early prostate cancer diagnoses, staging scans are not recommended by published practice guidelines [50]. This study found that unnecessary staging scans in these patients cost Medicare approximately US$11.3 million each year.

Access to quality healthcare has a significant impact on cancer outcomes. However, many cancer patients face barriers to receiving the care they need. Big data analyses have been central to demonstrating worrisome health disparities in oncology.

Several studies have demonstrated that certain patient groups are more likely to be diagnosed with more advanced cancers. A study by Fedewa et al., for example, used the National Cancer Data Base to compare the severity of prostate cancer at diagnosis by race/ethnicity and insurance status in 312,339 men [29]. Uninsured and Medicaid patients were more likely to have more aggressive cancers at diagnosis compared with privately insured patients. Similarly, more aggressive cancers were found in African–American, Hispanic and Asian men compared with Caucasians. Similar authors also used the National Cancer Data Base to examine the association between insurance status with cervical cancer stage [30]. Even after adjusting for race/ethnicity and other sociodemographic and clinical factors, this study found that uninsured, Medicaid and Medicare patients were more likely to be diagnosed with advanced-staged cervical cancers compared with patients with private insurance.

Many studies using big data have also demonstrated disparities in cancer treatment. Gross et al. [31] used SEER-Medicare data to show that African–American versus Caucasian patients were less likely to receive aggressive treatment for lung, breast, colon and prostate cancers. Du et al. [32] investigated the effects of socioeconomic factors on racial disparities in cancer treatment and survival. This study involved 13,234 cancer patients (including breast, colorectal, prostate, lung, cervical, ovarian, melanoma and bladder cancer) identified from the National Longitudinal Mortality Study-SEER linked database, and found that African–American patients with early-stage nonmetastatic cancer had worse 3-year overall survival compared with Caucasians. However, this difference in survival disappeared after adjusting for socioeconomic factors, including health insurance status, education level and income. This suggests that racial disparities in many common cancers may be partly mediated by socioeconomic factors.

Randomized controlled trials are considered the ‘gold standard’ for comparing efficacy of cancer treatments. Thus, to assess the ‘effective’ aspect of quality cancer care, there are important strengths of using clinical trials over of big data. The most important issue here is the ability of randomization to minimize confounding in measured and unmeasured factors among the comparison groups. While sophisticated analytic methodologies can be used to minimize confounding with big data analyses, residual confounding can remain which may affect the results from these studies [51,52].

On the other hand, there are also important strengths of using big data for comparative effectiveness. Because randomized trials take significant resources and time to complete, it is impractical to use trials to answer all (or even most) of the clinically relevant comparative effectiveness questions. There are other common situations when randomization is not possible, such as when a new treatment or technology is rapidly diffused into common practice to replace older options – as was the case of robotic surgery in prostate cancer [48]. In these situations, big data likely represent the best available method to compare the effectiveness of different options available to a cancer patient, and can also provide results faster than randomized trials; the latter is important so that patients and physicians can have data to guide their decisions before a technology has fully disseminated. Other important strengths of big data include large available sample sizes of patients, which provide sufficient power in these analyses and also subgroup analyses to determine if treatment benefits apply more to certain patient populations.

Another important consideration when evaluating observational studies including analyses of big data is the external validity. External validity is the extent to which the results of a study can be applied to the general population. The results of a study with high external validity will be able to be generalized beyond the study sample to other people and situations. The common threats to external validity include selecting small, homogenous patient samples limited to single geographic locations or institutions. The effect of these threats may be mitigated by using large registries that include diverse patient groups representative of the overall patient population. Population-based big data sources such as SEER and NHIS have external validity (generalizability) as a particular strength. In contrast, patients who enroll on randomized clinical trials are often highly selected and treatments and other clinical care usually provided in a controlled setting; therefore, results from trials may be more representative of ideal patients and outcomes but less generalizable to the overall population.

For the other IOM-described aspects of quality of care – safety, patient centeredness, timeliness, efficiency and equitability – big data resources represent the best way for assessment. However, a significant limitation of existing big data resources is the lack of detailed clinical information. For example, cancer recurrence data are not available in SEER or NCDB, and few big data sources have patient-reported quality of life. In addition, currently, big data resources which are available for research are at least 2–3 years behind current practice due to the time needed for data processing and quality assurance. After factoring in the additional time needed for data analysis and publication, the reported quality assessment results are typically at least 5 years old – and thus no longer reflective of ‘current’ practice or quality gaps.

Furthermore, while big data sources can be used to assess the quality of care, it is often unable to answer ‘why’, and lacks information on patient preferences. Therefore, in order to fully understand results from big data analyses, and then to create potential solutions to improve care, primary data collection and prospective studies are needed.

Current big data resources have already been shown to be powerful tools to assess the six dimensions of quality of care as defined by the IOM. However, as mentioned above, there are still significant limitations and challenges to the big data resources available today.

We envision a future where big data will be used to evaluate quality of care in a more timely way, and also to guide clinical decision-making in real-time based on individual patient characteristics and preferences [53]. The continued development and adoption of electronic health record (EHR) systems can help enable this vision to come true. We envision a future where large groups of patient data can be pooled – including detailed information about patient diagnoses and outcomes – from across institutions, so that each patient and their physician can find ‘patients like me’ to help with real-time clinical decision-making. The use of big data to support clinical care can occur at the time of initial treatment decision-making, during treatment for side effect management as well as during follow-up to predict long-term outcomes.

Going from where we are today to making this future vision a reality will not be easy. It will require an ability to combine vast amounts of data across different EHR systems. Currently, data captured in EHRs are incomplete and often not in discrete data elements; the latter important for data analysis. To allow pooling EHR data across a large number of institutions, a set of agreed upon discrete data elements would need to be built into the EHR systems. Furthermore, combining data from various institutions is often hindered by technical and legal hurdles.

The recent Big Data Workshop sponsored by the American Society for Radiation Oncology, American Association of Physicists in Medicine and the NCI specifically discussed this vision, the challenges and potential solutions. For this vision to become a reality, stakeholders including the professional societies, EHR vendors and funding agencies including NCI need to work together. For example, oncologists through professional organizations can contribute by defining core data elements necessary to collect for each patient, which may differ for each type of cancer. Proprietary EHRs can contribute by allowing data to be exportable into a universal format that can be used by all EHR systems, so that the data can be pooled across a large number of institutions, stored in a central repository and made available for clinical use and research. These efforts will require funding to accomplish. Despite these challenges, moving from the current status of using big data to retrospectively assess care quality to a future where big data can be used in real time to help patients receive high-quality care is an important and logical step. This vision is achievable but will require many people throughout oncology to work together.

The authors have no relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript. This includes employment, consultancies, honoraria, stock ownership or options, expert testimony, grants or patents received or pending, or royalties.

No writing assistance was utilized in the production of this manuscript.