Public reporting of health care data: A new frontier in quality improvement

The American College of Surgeons (ACS) historically and continuously has sought to promote the highest standards of surgical care. Hence, the ACS recognizes the importance of objectively collecting, analyzing, and reporting data regarding processes of care and clinical and patient outcomes in efforts to optimize quality. The public and the government are now expressing a greater demand for this data.1 This article addresses the role of public reporting as a means of informational transparency that aims to maximize the quality of deliverable surgical care.

Properly done, public reporting offers several potential benefits. It could reduce information asymmetry between both patients and providers and providers and payors; promote competition in the health care marketplace; apply pressure to reduce costs and improve quality; empower patients to be more active participants in their own care; and foster a culture of accountability, transparency, and efficiency.2 However, to be successful, public reporting must use a framework that has credibility for both those being evaluated and for those using the data.

Most current health care data collection and reporting modalities suffer from inherent limitations. For example, the claims data that commercial payors and the Centers for Medicare & Medicaid Services (CMS) currently use are designed primarily for billing and payment purposes and are not specifically tailored for quality measurement. Therefore, these claims data are ineffective for tracking many relevant clinical processes and outcomes. The three main drawbacks of administrative claims data pertain to: (1) documentation; (2) coding; and (3) attribution. With regard to documentation, it is often unclear from claims data which physician served as the primary surgeon or anesthesiologist and which served as the assistant.

The second issue is that coding is carried out by hospitals’ medical records staff who are not directly involved in the care of patients and is limited by the information entered into patients’ medical records. The issue with attribution is that no standardized methodology is available to appropriately ascribe patient episodes of care among the several providers who participate in the care. Furthermore, the crucial ability to risk-adjust at the individual patient level to account for differences in prior medical health status and other factors affecting procedural risk remains limited, despite the many algorithms that attempt to compensate for these deficiencies.

Whereas feedback from outcomes data to physicians and hospitals can be a powerful tool in quality improvement, an over-reliance on claims data, for the reasons listed previously, is potentially problematic. Ideally, surgical care should be assessed with clinical data using outcomes measures specifically designed for surgical quality improvement that are clinically relevant and risk-adjusted. These data may then be combined with episode-based and long-term resource use data that assess cost.

Ultimately, national reimbursement policies (such as Medicare’s hospital conditions of participation or value-based purchasing) will be driven by quality performance measures. How these measures are developed and reported to various interested groups will be crucial to optimizing quality. The ACS believes a strategy that addresses the needs of all pertinent stakeholders (patients, physicians, hospitals, and payors) is critical to successfully reporting health care data, while avoiding unproductive and potentially harmful regulatory and payment policies. The ACS remains committed to providing leadership in innovative quality assurance and the development of data-driven standards through programs such as the College’s National Surgical Quality Improvement Program (ACS NSQIP®), a hospital-based registry used by more than 400 hospitals. Through the use of clinical data, ACS NSQIP has provided surgeons with vital patient information that has led to real cost savings through improved care.3

Other efforts

Many physician, insurer, and governmental organizations have collaborated to establish guidelines for public reporting. Several are described in this article, but this list is by no means comprehensive.

AQA

The AQA alliance—originally known as the Ambulatory Care Quality Alliance—is a coalition started by the American Academy of Family Physicians (AAFP), the American College of Physicians (ACP), America’s Health Insurance Plans (AHIP), and the Agency for Healthcare Research and Quality (AHRQ). It is now a multi-stakeholder collaborative composed of more than 100 organizations representing physicians, clinicians, consumers, and health insurance plans. The AQA alliance has focused on establishing a consensus regarding a set of measures for assessing clinical performance that will be useful to payors, a multi-year strategy for rolling out measurements in the marketplace, a model for aggregating data, and a method for reporting useful data to providers, consumers, and purchasers. With regard to health care reform, the AQA aims to facilitate alignment between public and private efforts, promote best practices, address the gap between measurement and improvement, and provide guidance to the U.S. Department of Health and Human Services.4

The Commonwealth Fund

The Commonwealth Fund is a private foundation that was established in 1918 and supports independent research with the ultimate goal being to “promote a high performing health care system that achieves better access, improved quality, and greater efficiency, particularly for society’s most vulnerable populations, including low-income people, the uninsured, minority Americans, young children, and elderly adults.”5 Regarding public reporting, The Commonwealth Fund recognizes the powerful potential of quality improvement, but stipulates the following: information must be presented consistently and appropriately, multidisciplinary collaboration between numerous stakeholders is essential, research and concurrent evaluation should remain a prospective goal as well, and automated data collection is ideal.6

STS and ACCF

Public reporting of non-risk-adjusted coronary bypass mortality rates by Medicare was first undertaken in the 1980s, and this initiative provided the catalyst for the formation of the Society of Thoracic Surgeons’ (STS) Adult Cardiac Surgery Database. This database has grown and evolved since its initial formation into an audited database containing more than 4.5 million patient records with more than 1,000 adult cardiac surgical centers currently submitting data on each patient undergoing a cardiac surgical procedure. A “rating” system (one, two, or three stars)—based on risk-adjusted mortality rates and on a number of National Quality Forum (NQF)-approved process measures—has been developed. Each center participating in the database receives reports on its performance twice each year. Auditing is carried out upon data submission, and randomly by an external auditing entity (Iowa Foundation for Medical Care). In an agreement with Consumer Reports, over the last 24 months, database participants may voluntarily submit their STS rating for presentation in this publication.7-9 In addition, participating centers may also post their star ratings on the STS website.

New York State and Pennsylvania began reporting coronary artery bypass graft (CABG) mortality rates for surgeons and hospitals in the early 1990s. New York developed its own risk-adjustment system for both coronary bypass procedures and for percutaneous coronary interventions, and Pennsylvania reports unadjusted (raw) mortality rates. The American College of Cardiology Foundation (ACCF) has developed its own clinical registries for various subspecialty areas, the most prominent being an interventional catheterization database. The ACCF 2008 health policy statement on principles for public reporting of physician performance data state the following: the driving force behind such measures should be quality improvement, performance measures should be scientifically valid, reporting programs should be developed in partnership with physicians, disparate reporting programs should be standardized and uniform as best possible, reporting should occur at the appropriate level of accountability, and all reporting programs should include a formal process evaluating the program’s effect on quality and cost, including an assessment of potential unintended consequences.10

Stakeholders’ data needs in the public reporting schematic

Stakeholders’ data needs in the public reporting schematic

Stakeholder perspective

Given the dynamic interplay of the medical industry, public reporting necessarily involves a mix of various stakeholders. Successful reporting will require consideration of each party’s various needs (see table).

Patients

Because the driving force behind public reporting should be improvement of quality of deliverable care, the patient perspective remains essential. Information asymmetry between patients and providers has been a consistent barrier to increasing the role of patient choice in improving quality and decreasing costs. Public reporting constitutes a powerful intervention that can bridge this gap, but only if conducted appropriately. Therefore, patients should have access to information that meets the following criteria:

  • Understandable. Measures on processes and outcomes must be reported to patients in a manner that the lay population can easily interpret. Information in performance reports must not be misleading or confusing; it should be clear and accurate, otherwise it can exacerbate the already-prominent information asymmetry. Using performance reports to make health care choices is a very difficult cognitive task. Patients have a difficult time differentially weighing factors reported about physicians, and although they often believe they are weighing various factors, they often are not.11
  • Useful and relevant. Patients should be able to use public reporting data to make logical and informed decisions that guide their care-seeking behavior, including choice of physician(s) and/or hospital(s). Such measures should also offer reassurance that current care is safe, effective, affordable, and patient-centered as a form of positive reinforcement. Finally, comparative metrics that allow selection between providers and/or hospitals should be considered as well. The likelihood of differences, when identified as statistically important and/or clinically relevant, should be explained. Reports should provide guidance on making the information provided actionable for the patient.
  • Easily accessible. Patients should be able to find and view information with a public search, and various methods of access should be built into the design of public reporting protocols. Information technology has already been used to bridge information asymmetry in other industries (such as used car sales) and offers a powerful tool in the area of public reporting as well.

Physicians

Physician participation is crucial to the success of public reporting initiatives. Therefore, the following provider perspectives must be considered:

  • Physician-led. Due to the increasing complexity of specialized medicine, physicians delivering care within the specialty being reported should identify, maintain, and prospectively review quality metrics that evaluate processes and outcomes. These metrics should be created in an open fashion with provisions made for external comments into the rule-making process.
  • Quality-oriented, not punitive. Public reporting initiatives should be implemented primarily as quality improvement efforts that aim to improve systems and prevent systems failures; they should not serve as punitive indictments that mark individual and organizational culpability. Individualized confidential reports for physicians to understand where their care or outcomes differ from their peers should be considered for all systems, and be required for systems reporting on individual physicians. A major physician concern with public reporting of outcomes is that the information will be used punitively. Therefore, data acquisition practices, risk-adjustment techniques, and mechanisms for feedback of outcomes data should be developed through a multi-stakeholder consensus-based, surgeon-led process.
  • Protected. Physicians should balance the issues of maintaining patient confidentiality against promoting transparency of public reporting measures themselves. Current federal law does not protect information submitted to patient safety systems from discovery in legal proceedings, which discourages their use and limits their potential for future corrective measures. To be optimally effective, public reporting systems may require such protections, so that information provided by physicians remains privileged for judicial proceedings in civil matters or other disclosures.
  • Publicly accountable. Because disputes between physicians or between physicians and other agents will arise, public reporting should include mechanisms for timely review and appeal of any results, with public resolution of such issues.

Employers/purchasers

Publicly reported data are also valuable to employers as well as patients, as long as they meet the following criteria:

  • Aggregated. Employers should have access to aggregated information that they can use to select coverage options and providers for their employees. Protections to avoid the release of specific outcomes of certain patients within the employer group must be included in reporting in order to protect the privacy of the patients in the database. This information should include costs and quality outcomes of possible providers and hospitals, as well as benchmarks of service and quality standards.
  • Easily obtainable. This information should be presented in a manner that is understandable for purchasers and easily obtainable.

Payors/insurance companies

Substantial information asymmetry also exists between patients and insurers, and between insurers and in-network providers. Public reporting may potentially bridge this gap as well by meeting the following criteria:

  • Available. Insurers should be able to use available data to evaluate in-network providers on processes and outcomes.
  • Comparative. Insurers should be able to evaluate their own performance on service and quality against competitors and other relevant norms.
  • Tailored. Quality outcomes should be tailored to remain relevant to the insured population.
  • Effective. It is worth noting in this context that the most powerful information will likely result from a merger of clinical registry outcome data with insurance company data upon use. These merged data will allow the assessment of the effectiveness of treatment strategies and fulfill a major need in the U.S. health care system for more rational use and control of resources.

Hospitals

Publicly reported data will benefit hospitals as well by meeting the following criteria:

  • Comprehensive. Information submitted to reporting systems should be comprehensively analyzed to identify interventions that minimize the risk of unintended negative consequences.
  • Confidential. Confidentiality protections must be enacted for patients, health care professionals, and health care organizations to enable a culture of identifying and reducing errors.
  • Collaborative. Reporting systems should facilitate sharing of patient information across health care organizations and foster confidential collaboration across different reporting systems.

Government/policymakers

It is important to note that, at the current critical juncture in health care, public reporting can be used to guide future policymaking by meeting the following criteria:

  • Cumulative. Policymakers should have access to current and accurate aggregate information on providers, hospitals, and health plans.
  • Granular. Governing bodies should be allowed to monitor changes in the health care system, identify areas requiring closer examination, and promote a culture of self-reporting by monitoring groups.
  • Systems-based. Policymakers and other regulatory bodies should stress the importance of avoiding direct punitive actions against individual providers or groups, and, instead, should favor systems-wide corrective actions tailored for quality improvement.
  • Efficient. Policymakers should support the development of improved quality measures and promote coordination between groups to decrease duplication and other redundancies.
  • Innovative. Policymakers should foster cooperation and reward innovation to incentivize public reporting and implementation of improvements

Principles in public reporting

The following should serve as guiding principles for enacting accurate and effective public reporting:

Data characteristics

  • Quality improvement should remain the driving force behind public reporting initiatives.
  • Processes of care, and the clinical, patient, and outcome quality metrics, should be valid, timely, reliable, evidence-based, and appropriately risk-adjusted to account for medical, socio-cultural, and economic factors that affect patient care.
  • Data safeguards must be implemented to protect the privacy of patients and physicians.

Data collection

  • Expert physician involvement and oversight are required to create measures, analyses, outcomes, and public reports.
  • Data should be collected in a manner verifiable by independent clinicians and/or trained data abstraction experts.
  • Data collection facilities should have appropriate infrastructure, including staffing, specialists, and equipment.
  • Data collection should, itself, be cost-effective and, ideally, electronic.

Data analysis

  • Data analysis should be transparent, clearly defined, and independently reviewable by those being reported on, as well as statistically robust.
  • Periodic external peer review is vital to verifying analytic methodology.
  • Both processes and outcomes should be measured and benchmarked, with more emphasis placed on outcomes.

Reporting

  • For outcomes that are dependent on multidisciplinary performance of teams of physicians and other ancillary health care professionals, reporting is most appropriate at the organizational, as opposed to the individual physician, level.
  • Quality and resource use measures should be linked at both the physician and organizational levels to demonstrate overall value and effectiveness.
  • All reports, both electronic and paper-based, should include disclaimers regarding the limitations of physician performance assessment and the uses of such information for consumer choice and overall quality improvement.
  • All public reporting initiatives should include prospective and concurrent evaluations of their effect on cost, quality, and any unintended consequences.
  • Reporting on patient experience should use validated, reliable, and standardized tools.
  • Public reporting should include an appeals process for dispute resolution that is accessible by clinicians and consumers, with timely and responsive data adjudication and correction.

 

Unresolved issues

Despite the powerful benefits that public reporting offers and the increasing recognition of its utility, several issues remain unresolved, which should serve as guidelines for future inquiry.

First, an AQA alliance survey revealed that consumers want information on individual physicians, as well as larger groups and hospitals. While feasible, “concerns about sample size, attribution, and other technical issues challenge the ability to measure physicians at the individual level.”12 Therefore, process and outcomes metrics may need to be further designed to specifically analyze individual physicians, and this information needs to be reported in a fair and accurate manner to patients. Efforts to ensure equity and accuracy are particularly important when considering various lower volume services where statistical “noise” may yield results that are not reflective of the quality of care. For complex in-hospital procedures that require interaction among multiple providers from different specialties, a focus on a single physician may not provide an accurate reflection of the overall quality of care provided by a team of caregivers.

Second, in any public reporting schema there are trade-offs between the transparency required for success and confidentiality required to protect physicians from litigation and from unfair and invalid characterizations of their clinical practices. If public reporting data may be used in litigation, physicians are far less likely to comply, which may undercut public reporting initiatives. Ultimately, patients themselves may be on the losing side of this situation due to the lack of potential quality improvement. Therefore, a solution must be enacted that balances appropriate confidentiality and protection required for compliance from providers with transparency needed to fairly and accurately assess publicly reported information.

Third, public reporting poses numerous issues concerning data management. Various agencies could manage and audit the entire public reporting process, including professional societies, certifying medical boards, regulatory bodies, and third-party payors. This option, which bypasses control of the process, will invariably affect the financial incentive structure required to make public reporting sustainable in the long-term. Furthermore, the relationship between professional public-reporting registries and both the CMS and private insurance companies will need to be defined.

Conclusion

Public reporting involves the objective collection, robust analysis, and transparent reporting of health care data to patients, providers, insurers, hospitals, and policymakers. Such measures constitute a new frontier in quality improvement that may promote competition in the health care marketplace, empower patients to be more active participants in the handling of their own care, and foster a greater culture of accountability, transparency, and efficiency. In the organization’s continual quest to provide the highest standards of surgical care, the ACS remains committed to propelling public reporting forward as a market-based initiative that will drive quality improvement.


References

  1. Fung CH, Lim YW, Mattke S, Damberg C, Shekelle PG. Systematic review: The evidence that publishing patient care performance data improves quality of care. Ann Int Med. 2008;148(2):111-123.
  2. Marshall M, Shekelle P, Brook R, Leatherman S. Dying to Know: Public Release of Information About Quality of Health Care. London: Nuffield Trust; 2000.
  3. Ingraham AM, Richards KE, Hall BL, Ko CY. Quality improvement in surgery: The American College of Surgeons National Surgical Quality Improvement Program approach. Adv Surg. 2010;44:251-267.
  4. U.S. Department of Health and Human Services. Agency for Healthcare Research and Quality. The Ambulatory Care Quality Alliance: Improving clinical quality and consumer decisionmaking. Available at: http://www.ahrq.gov/qual/aqaback.htm. Accessed April 10, 2012.
  5. The Commonwealth Fund. Mission statement. Available at: http://www.commonwealthfund.org/About-Us/Mission-Statement.aspx. Accessed April 10, 2012.
  6. Colmers JM. Public reporting and transparency. The Commonwealth Fund. February 2007. Available at: http://www.commonwealthfund.org/Publications/Fund-Reports/2007/Feb/Public-Reporting-and-Transparency.aspx. Accessed April 10, 2012.
  7. Shahian DM, Edwards FH, Jacobs JP, Prager RL, Normand SL, Shewan CM, O’Brien SM, Peterson ED, Grover FL. Public reporting of cardiac surgery performance: Part 1: History, rationale, consequences. Ann Thorac Surg. 2011;92(9)(3 Suppl):S2-11.
  8. Shahian DM, Edwards FH, Jacobs JP, Prager RL, Normand SL, Shewan CM, O’Brien SM, Peterson ED, Grover FL. Public reporting of cardiac surgery performance: Part 2: Implementation. Ann Thorac Surg. 2011;92(9)(3 Suppl):S12-23.
  9. Shahian DM. Public reporting of cardiac surgery performance: Introduction. Ann Thorac Surg. 2011;92(3 Suppl): S1.
  10. Drozda JP, Jr. Hagan EP, Mirro MJ, Peterson ED, Wright JS, American College of Cardiology Foundation Writing Committee. ACCF 2008 health policy statement on principles for public reporting of physician performance data: A report of the American College of Cardiology Foundation Writing Committee to develop principles for public reporting of physician performance data. J Am Coll Cardiol. 2008;51(20):1993-2001.
  11. Hibbard JH, Peters EM. Supporting informed consumer health care decisions: Data presentation approaches that facilitate the use of information in choice. Ann Rev Pub Hlth. 2003;24:413-433.
  12. The AQA Alliance. AQA reporting workgroup paper: Results from a survey on physician reports. 2011. Available at www.aqaalliance.org/files/AQAReport10.03.11.doc. Accessed April 10, 2012.

Tagged as: , ,

Contact

Bulletin of the American College of Surgeons
633 N. Saint Clair St.
Chicago, IL 60611

Archives

Download the Bulletin App


Get it on Google Play