Using Big Data for Big Research: MPOG, NACOR and other Anesthesia Registries
Richard P. Dutton, M.D., M.B.A.
Executive Director, Anesthesia Quality Institute
Chief Quality Officer, American Society of Anesthesiologists, Park Ridge, IL
Introduction
A silent revolution is under way in anesthesiology, one that will have a lasting impact on our patients and our practice. I refer to the research potential of ‘big data’ in anesthesiology, driven by the rapid uptake of Information Age technology in our offices, clinics and hospitals. Electronic healthcare records (EHRs) are changing the way we care for patients, the way we document and bill, and how we understand our practice. In the long run they will do much more: they will provide a fundamental ability to link clinical decision-making in the operating room with patient outcomes in a way that lets us learn from every patient encounter.
This article will provide a brief overview of existing large datasets describing anesthesia patients, procedures and outcomes. I will review their current contents and structure, their future development, and their long-term potential for comparative effectiveness and health services delivery research.
Existing Anesthesia Registries
Table 1 lists several sources for big data in anesthesia. Two large datasets supported by the federal government include information about anesthesia care. These are the National Inpatient Sample (NIS) and the Centers for Medicare and Medicaid Services (CMS) 5% and 100% data files. Both of these are populated with administrative data, generated to support Part A (hospital) and Part B (provider) billing, and then aggregated by government payers. The NIS is a statistically-balanced database constructed from information submitted by hospitals to state health agencies, and then passed to the Agency for Healthcare Research and Quality (AHRQ). NIS includes information about hospital inpatients, including those who undergo surgery. NIS is especially useful for looking at gross outcomes (in-hospital mortality, length of stay) and overall patient characteristics (age, sex, comorbidities). NIS includes International Classification of Diseases, 9th Edition (ICD-9) information for each patient.
In theory this includes a number of codes for postoperative complications, but in practice the accuracy of this data is questionable. Different hospitals in different states have widely differing incentives for finding this information in their medical records and reporting it accurately. Some hospitals have incentives for caring for sicker patients, and therefore seek out and document comorbidities more aggressively than those hospitals which do not; further, some complications are now associated with payment withholds or penalties, leading to an incentive to not find them, or to code them in other ways.
The CMS datasets are derived from clinical encounters submitted as claims for payment by facilities and providers, both inpatient and outpatient, and have some of the same shortcomings as the NIS. CMS data are also heavily skewed towards older patients, because of the eligibility requirements of Medicare. These data include both inpatient and outpatient surgical procedures and anesthetics. The 5% sample is readily available for anesthesia researchers, but includes no identifying information about each case, making it impossible to link records for the same patient for an anesthesiologist, a CRNA and a hospital. The 100% data are more comprehensive, but harder to gain access to. There is no anesthesia-specific information in the CMS data, other than the code of the procedure performed, and no information on long-term outcomes.
Administrative data are also available from several private insurance companies and aggregating agencies such as Premier. These datasets are strong in information on resource utilization, but less useful for assessing the specifics of clinical care. While the Premier database accurately reflects charges for medications used, for example, it can shed little light on why specific medications are chosen.
More specific to the practice of anesthesiology is the registry of the Multicenter Perioperative Outcomes Group (MPOG), built primarily to facilitate anesthesia research.1 MPOG is a collaborative effort of more than three dozen academic anesthesia departments, coordinated by the University of Michigan. The MPOG investigators developed a common format for capturing clinical information from Anesthesia Information Management Systems (AIMS), and they are working with their information technology vendors to map AIMS data and transmit it to the MPOG registry. To date, MPOG is collecting every-case data from about a dozen of the participating departments, representing five of the ten different AIMS commonly used in the United States. Efforts are underway to build reports in the remaining AIMS, so that within a few years any hospital with an AIMS will be able to contribute to MPOG. Data in MPOG are highly granular for the anesthetic encounter, capturing information on anesthesia procedures, medications, fluids, monitors and vital signs for every patient the department cares for. Any academic physician in any MPOG participant practice can apply to the Data Use Committee for permission to analyze the aggregated data.
Similar to MPOG, but including additional data from anesthesia billing systems and quality capture software, is the National Anesthesia Clinical Outcomes Registry (NACOR), developed and maintained by the non-profit Anesthesia Quality Institute (AQI).2,3 This organization was founded by the American Society of Anesthesiologists (ASA) to improve the quality of anesthesia practice throughout the United States. Like MPOG, NACOR collects data by direct transmission of digital information; in fact, NACOR uses the same formats developed by MPOG for collecting data from AIMS. Unlike MPOG, NACOR begins by harvesting information from anesthesia billing systems, which use relatively simple formats to capture basic data from every case. Participation in NACOR is open to any anesthesia practice in the United States, whether they have started using an AIMS or not. The purpose of NACOR is to provide local feedback to the anesthesia practice for regulatory compliance and quality improvement. NACOR is the largest specialty-specific registry in anesthesia, including data from more than 2,100 facilities, 20,000 providers and 250 practices (see Figure 1). Although not primarily intended for research, the AQI does publish a Participant User File (PUF) from NACOR each quarter, presenting a de-identified aggregate dataset for the use of academic researchers in AQI-participant practices. The most recent version of the PUF includes more than 13,000,000 cases collected in NACOR between January 1, 2010 and September 30, 2013. The NACOR PUF is heterogeneous in the data presented; all cases have billing information, about a quarter have quality outcome information (usually gathered at the time of PACU discharge) and only about 10% have detailed information from an AIMS. Because of its size and its national reach, NACOR is a good choice for descriptions of anesthesia practice in the U.S., and can serve as the backdrop for studies using more granular information collected at a single institution.
There are also a number of subspecialty registries available for specific niche practices in anesthesia. Largest is the Society for Ambulatory Anesthesia Clinical Outcomes Registry (SAMBA-SCOR), which focuses on process and outcomes in outpatient surgery. SAMBA-SCOR shares data with both MPOG and NACOR. All three registries use identical data formats, such that the same output files can be used for submitting data to all three. Other subspecialty registries include the Pediatric Regional Anesthesia Network database, the registry of the Malignant Hyperthermia Association of the United States, and the newly-launched anesthesia component of the Society for Thoracic Surgery national cardiac surgery registry. Each of these projects gathers granular information about a specific subset of anesthesia patients, and each is intended primarily for scientific research.
Future Developments
Although created by different stakeholders for different purposes, all databases containing anesthesia information face common challenges in recruiting participants, defining data elements, collecting case-by-case information, and analyzing and reporting the results. Older surgical registries have relied on an “eyeballs” methodology, in which a professional abstractor reviews medical records for specific pieces of information. This model generates good data in a consistent fashion, but it is expensive for the hospital to support and thus limited in the number of cases, patients and data elements that can be included. Modern registries, built to take advantage of the increasing use of EHR systems, are all seeking to have data move directly from the medical record to the registry, without requiring human abstraction. The difficulty with this model is the heterogeneity of electronic data today. Some elements, such as vital signs and medication doses, are relatively standard from one system to another, but other elements, such as outcomes and complications, are lacking in consistent definitions across practices and software vendors. In 2013 MPOG and the AQI combined to sponsor a conference on common measures and common definitions in anesthesia electronic records. The first DefCon included two dozen anesthesia quality management experts working with an equal number of EHR vendors to produce consensus definitions of key elements of the anesthesia record. Published on the AQI website, these definitions are serving to unify data for anesthesia research and quality management.4
Another goal for the future is to move from self-reported outcomes, which require the active participation of the clinician, to measures that can be passively calculated from the medical record. Hemodynamic instability, for example, will be calculated from the vital signs in the OR and PACU, as captured by the AIMS, rather than by subjective assessment of the provider. An exception to this principle, but still a necessary step for the future, will be increasing collection of outcomes reported by anesthesia patients. Such measures as adequacy of pain management, respect for privacy, quality of communication, and overall satisfaction with anesthesia care can only be gathered from the patient’s perspective. Several commercial systems have been launched to gather this kind of data, using tools such as automated voice-response systems, email, and text messaging. Data gathered in this fashion can be linked directly to digital information in the billing system or EHR, and can be transmitted automatically to registries such as NACOR.
Healthcare reform at the national level will drive an increasing need for ‘shared accountability’ measures, which assess overall outcomes from an entire process of care. Rather than reporting on specific processes at the individual level—such as the timely administration of perioperative antibiotics—shared accountability measures will look at meaningful outcomes such as mortality, major morbidity and hospital length of stay at the level of the entire perioperative team. Anesthesiologists, surgeons and hospitals will work together to define these measures and collect the necessary data. Large registries such as NACOR are an obvious source for parts of the data, especially if granular anesthesia process information can be combined with longterm outcomes collected in existing surgical registries. Linkage of data from one system to another will depend on accurate patient identification (while still protecting patient confidentiality) and common definitions of cases and risk factors. Shared accountability measures will be required for public reporting on the effectiveness of new healthcare organization and payment mechanisms, such as the perioperative surgical home.
There is also an opportunity to combine structured data in the medical record with narrative data about specific cases and events. An example would be a future state in which the anesthesia provider completes a quality capture form at the end of every case, indicating the absence of any major adverse event. In the rare case in which something unusual or unexpected happens, the EHR would shunt the reporter directly to an online incident report form that requests a narrative description of the event. This electronic data would be kept separate from the medical record, but would be available for local quality and risk management purposes, and for automatic transmission to national aggregators such as NACOR. The AQI is pioneering this approach in a few practices today, and is looking for other vendors and groups to work with. [Ed. – ABC is discussing with the AQI methods to include a narrative incident report feature in our EHR technology.]
Turning Big Data into Scientific Research
However the data are collected, the major challenge for any anesthesia registry is analysis and reporting. Clinicians are busy and distracted on a daily basis, and are bombarded with administrative and educational products. To effectively turn data into information, registries must find ways to creatively analyze what they collect and intuitively present it to their stakeholders. One mechanism for doing this is through publication of scientific papers. Journal articles maintain a consistent level of quality through peer review, are familiar to all physicians, reach a large audience both inside and outside the specialty, and are preserved for future reference through the National Library of Medicine. Writing scientific papers can be ‘crowd-sourced’ by providing access to the data to a large cadre of volunteer researchers. This model is being followed by both MPOG and the AQI, who make their data available with minimal barriers to any anesthesiologist at a participating institution. On a larger scale, this is also the model followed by the federal government with CMS data and the National Inpatient Sample.
Availability to researchers of Big Data from national-level registries is rapidly spawning a new breed of clinical scientist with skills in medical informatics, epidemiology and complex statistical methodologies. New grant mechanisms are arising to support these scientists, with funding from AHRQ, the Patient Centered Outcomes Research Institute (PCORI) and organizations within our own specialty. Both the Foundation for Anesthesia Education and Research and the Anesthesia Patient Safety Foundation now offer funding for healthcare delivery and comparative effectiveness research. These grants are directed towards young investigators, and the funding often includes specific provisions for training in operations research, healthcare economics, or safety and quality. A number of anesthesia training programs are following suit. The anesthesia departments at Yale, the University of Alabama at Birmingham, the University of Michigan, the University of Washington and the University of California at Irvine all have dedicated training for residents and fellows in patient safety, health policy research, and quality management. Other institutions, such as Vanderbilt and Mount Sinai, offer dedicated training in healthcare information technology. Academic anesthesiologists of the future will need to be experts in electronic data collection, aggregation, and interpretation; these skills will be just as important as anatomy, pharmacology and physiology were to generations past.
Conclusion
Consistent with our specialty’s long history of advancing patient safety, the evolution of Big Data registries in anesthesia is leading all of medicine into a new era. In the near future we will learn from every patient we care for, and will have objective evidence to guide decisions about drugs, monitors, and anesthesia techniques. This advancing knowledge will free us to take on ever more challenging patients and operations, and to take our place as leaders and facilitators of procedural care of all kinds.
1Kheterpal S, Healy D, Aziz MF, Shanks AM, Freundlich RE, Linton F, Martin LD, Linton J, Epps JL, Fernandez-Bustamante A, Jameson LC, Tremper T, Tremper KK; on behalf of the Multicenter Perioperative Outcomes Group (MPOG) Perioperative Clinical Research Committee. Incidence, Predictors, and Outcome of Difficult Mask Ventilation Combined with Difficult Laryngoscopy: A Report from the Multicenter Perioperative Outcomes Group. Anesthesiology. 2013 Sep 25. [Epub ahead of print]
2Dutton RP, Dukatz A. Quality improvement using automated data sources: the anesthesia quality institute. Anesthesiol Clin. 2011 29(3):439-54.
3Grissom TE, DuKatz A, Kordylewski H, Dutton RP. Bring out your data: The evolution of the National Anesthesia Clinical Outcomes Registry. International Journal of Computational Models and Algorithms in Medicine, 2011; 2: 51-69.
4http://www.aqihq.org/qualitymeasurementtools.aspx
Richard P. Dutton, MD, MBA is Executive Director of the Anesthesia Quality Institute (AQI). He also serves as Chief Quality Officer for the American Society of Anesthesiologists. Dr. Dutton is a Clinical Associate at the University of Chicago Department of Anesthesia and Critical Care. To contact Dr. Dutton or the AQI, visit www.aqihq.org.