Distinguishing QI projects from human subjects research: Ethical and practical considerations

Since the 2000 publication of the Institute of Medicine’s report To Err Is Human: Building a Safer Health Care System and the 2001 report Crossing the Quality Chasm: A New Health System for the 21st Century, the focus on improving the quality of health care in the U.S. has grown sharper.^1,2 This trend continued with the release of the U.S. Department of Health and Human Services’ 2011 Report to Congress: National Strategy for Quality Improvement in Health Care.³

The emphasis on quality improvement has affected surgery, specifically with national attention on process measures, such as the Centers for Medicare & Medicaid Services’ (CMS) Surgical Care Improvement Project (SCIP), and rigorous outcome measurement initiatives, such as the American College of Surgeons National Surgical Quality Improvement Program^® (ACS NSQIP^®).^4,5 The momentum and focus on quality improvement (QI) continues to grow, and QI is now recognized as a major force shaping health care.

Perhaps the greatest strides toward improving the quality of surgical care are occurring at the institutional level, epitomizing the axiom to “think globally, but act locally.” For surgeons and surgical trainees on the front lines of patient care and involved in many QI projects, being well-versed in the language, methods, and tools of QI has become essential. Many institutions have initiated formal didactic and hands-on surgical QI projects as a part of postgraduate training.⁶ The Accreditation Council for Graduate Medical Education (ACGME) acknowledges that its core competencies, such as practice-based learning and improvement and systems-based practice, are well-aligned with participation in QI efforts at the local level.⁷ The ACGME has formally outlined a Clinical Learning Environment Review (CLER) program that encourages residencies to increase the emphasis on patient safety.

Public demand is driving these changes, which provide opportunities for sponsoring institutions to demonstrate leadership in patient safety, quality improvement, and reduction in health care disparities (see table above).⁸ The ACS has also recognized the value of formal quality improvement education. The College’s Quality In-Training Initiative (QITI) adapts ACS NSQIP methods and data for use in graduate surgical education, develops a national quality improvement curriculum, and creates a culture in surgical education that emphasizes quality consciousness.^9,10

These efforts have fostered a large number of QI projects and tremendous research opportunities for students, residents, and practicing clinicians. It is difficult to find a contemporary surgical meeting agenda or surgical journal table of contents that does not have a significant portion dedicated to QI. A search of the U.S. National Library of Medicine’s National Institute of Health PubMed.org website demonstrates a dramatic rise in the number of QI studies published over the last decade (see Figure 1). With more projects being designed, performed, and published, the line separating QI efforts from human subjects research (HSR) is often blurred. The role of institutional review boards (IRB) in QI efforts is, at times, unclear. Each project should be individually assessed, and if any questions arise, liaisons and experts from local groups—such as the IRB, QI officers, and experienced researchers—should be engaged for advice and review.

The purpose of this article is to help surgical investigators navigate the process of distinguishing between QI and HSR initiatives. It also serves as a guide for initial project design.

Development of the QI endeavor

Protecting safety and confidentiality of human subjects who participate in research activities is of paramount importance. The many policies and procedures currently designed to protect human subjects sometimes may seem at odds with the ability of researchers and institutions to rapidly develop and institute QI projects. Potentially tenuous distinctions are further challenged because many QI efforts have a control group, and the QI intervention is often disruptive, with minor changes in clinical practice potentially altering risks faced by the patient, especially in a high-stakes situation such as surgical care.

Yet the overlap of QI projects with direct patient care is often precisely why many clinicians find QI projects rewarding. The impetus for many QI projects is typically a clinical observation that spawns a query into contemporary management options or other aspects of practice, which, in turn, generates the concept that a small intervention or modification of the current treatment algorithm might improve the outcome for the patient. Navigating the process of QI takes various forms, involving different philosophies and methodologies, many of which have emerged from the manufacturing industry.¹¹

One example is the Six Sigma approach to QI, which calls for defining, measuring, analyzing, improving, controlling, and then repeating the cycle. Other examples include plan-do-study-act cycles, statistical process control, chart tracking, and Lean methodology principles. Typically, an institution chooses from these available QI tools and uses them across multiple departments and disciplines. A cadre of quality officers, comprising members of various management teams, is usually available to assist in the QI effort. Guidelines have been developed for reporting QI studies known as the Standards for Quality Improvement Reporting Excellence (SQUIRE).¹² These guidelines consist of a checklist of 19 items that address areas common to all scientific reporting but are modified to reflect the unique nature of medical quality improvement efforts. Checklist items include sections on methods, results, and discussion and are reported in terms of the lessons learned from the intervention in addition to the outcomes being measured. As the QI endeavor is formulated, the clinician is often left wondering where QI ends and HSR begins.

Defining research and human subjects

Some QI activities have both a research purpose and a QI goal, and in these cases, HSR regulations may apply. The first question to address is whether the QI endeavor is legitimate research. The Code of Federal Regulations (32 CFR 219.102[d]) defines research as “a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge.”¹³ Research is carried out to add to the profession’s understanding of surgical conditions and disease in contrast to altering or comparing established or already validated treatment options. The Code of Federal Regulations (32 CFR 219.102[f]) defines a human subject as “a living individual about whom an investigator conducting research obtains either data through intervention or interaction with the individual or identifiable private information.”¹³ QI is more often designed to study whether an accepted norm or behavior is being conducted locally and, if not, to modify the actions of the personnel involved so as to approach these norms. In general, QI is not designed to develop generalizable knowledge or to investigate items or approaches that are not considered a recognizable norm, but this situation is not always black and white.

Several key terms defining QI include the following:

• Interventions include both physical procedures by which data are gathered, such as blood draws or tumor samplings, as well as manipulation of the subject or the subject’s environment for research purposes.
• Interactions include communication or interpersonal contact between investigator and subject.
• Private information includes information about behavior that occurs in a context in which an individual can reasonably expect that no observation or recording is occurring, and information that has been provided for specific purposes by an individual and which the individual can reasonably expect will not be made public (for example, a medical history). Private information must be individually identifiable (for example, the identity of the subject is, or may readily be ascertained, by the investigator or may be associated with the information) to obtain the information to constitute research involving human subjects.¹³

Exemptions from HSR

Based on the information provided in this article, it is hard to imagine that many QI projects would qualify as anything other than HSR. However, there are several exemptions from full IRB evaluation. An example of a study that would be exempt from IRB review is an investigation designed to study the public benefit of a service program in terms of its efficacy or efficiency, possibly in the context of available alternatives. Another example is research designed to evaluate the taste and quality of food. More pertinent to projects within the patient-centered health sciences, research studies that use purely de-identified, preexisting data fall outside the regulatory definition of HSR. A key distinction is that this research does not include assembling a dataset that contains identified data and then stripping the database of the identifiers. Independent of the Code of Federal Regulations, investigators must also factor in compliance with the Health Insurance Portability and Accountability Act (HIPAA). HIPAA provides two avenues for de-identifying data:

• The safe-harbor method, which involves stripping specific data elements from datasets
• Removal of identifiers to the extent that a statistician concludes that the data cannot be reasonably re-associated with any particular individual

Certain national datasets, including information from the Agency for Healthcare Research and Quality’s Healthcare Costs and Utilization Project, the ACS NSQIP participant use file (PUF), and Medicare billing data are distributed in a de-identified manner. Therefore, studies using these preexisting and de-identified sources are typically exempt from formal IRB review. Institutions have developed individual policies and procedures that apply to intramural QI projects. If there is any question about whether IRB review is needed, local experts and IRB liaisons should be enlisted to review the protocol in question. Often, if the study involves minimal risk, the IRB may grant an expedited review.

Ethical considerations

Health care research requires that investigators maintain and abide by the highest ethical standards. The war crimes of World War II prompted development of the Nuremberg Code, which provides guiding principles for HSR, such as voluntary consent, beneficence, and properly formulated scientific inquiry.¹⁴ Subsequently, the Belmont Report, published by the U.S. Department of Health & Human Services, summarized ethical principles and expanded upon guidelines pertaining to respect for persons, beneficence, and justice.¹⁵ In 2013, the World Medical Association provided an update of the Declaration of Helsinki, which outlines guiding principles for medical research involving human subjects, including the following statement:

Physicians who combine medical research with medical care should involve their patients in research only to the extent that this is justified by its potential preventive, diagnostic or therapeutic value and if the physician has good reason to believe that participation in the research study will not adversely affect the health of the patients who serve as research subjects.¹⁶

With these guidelines in mind, similarities between QI and HSR must be acknowledged. Both are fundamentally generated by observation and an inquiry that generates a query, pursued in the form of data collection. Both initiatives often involve testing various solutions and identifying key interventions that merit further investigation and thought. Both QI and HSR should be conducted in a manner that shows respect for patients and minimizes patient risk. Some QI projects are similar to HSR in terms of providing informed participation or options for treatment pathways, and thus need to be reviewed by an IRB. The question of informed consent for participation in QI projects remains a topic of ongoing debate.¹⁷ Informed consent can be waived after review if the research in question:

• Poses “no more than minimal risk to the subjects.”
• “Will not adversely affect the rights and welfare of the subject.”
• “Could not practicably be carried out” otherwise.
• “Whenever appropriate, the subjects will be provided with additional pertinent information after participation.”¹³

Some institutions now have blanket hospital-wide admission consent forms that not only serve as permission for treatment, but also include QI activities as part of the consent.

Fundamental differences between QI and HSR

Although QI and HSR often overlap, there are several fundamental differences between the two. First, as already noted, HSR involves generating or contributing to generalizable knowledge, whereas QI attempts to improve a program or service or align current treatment with established best practices and evidence-based medicine. HSR often involves randomization of patients, whereas QI projects typically do not randomize to various treatment arms and more often subject the entire population to a system or policy change, often tracked over time. Whereas HSR generates findings that might affect future policies, QI findings intentionally address current standards or protocols and attempt to change the policies and standards for subsequent patients or encounters. HSR is rooted in identifying a subset of patients to study in the most controlled manner, often using strict inclusion and exclusion criteria to define the population of interest. Conversely, QI is typically all-encompassing for all patients who may have a disease, undergo a procedure, or interact with a specific aspect of the health care system. Exclusion of specific populations or creating exceptions from the algorithm may subvert the QI effort.

Patients participating in HSR are not guaranteed that they will benefit from participation. Significant efforts are made to minimize risk and harm; the benefits, however, are unknown and may be the impetus behind the study design.¹⁸ QI efforts are typically designed with a clear benefit in terms of safety, quality, efficiency, satisfaction, cost, or some other measurable outcome for the patient at hand or subsequent patients in the near-term.

Most HSR has a clearly defined study protocol with start and end dates and minimal alteration to study design. On the other hand, QI can be continuous and tracked over long periods of time, with organized response to trends, clear identification of outliers, and an expected evolution of the QI algorithm. Finally, with the intention of HSR being the development of generalizable knowledge, publication of findings is the norm and often expected regardless of study results.

QI efforts are often conducted for a health care system’s internal use and often do not result in external publication or presentation. However, when appropriate, QI projects can yield publications and presentations that disseminate protocols and pathways as well as share lessons that may benefit other institutions or programs interested in similar QI projects. Similarly, QI projects may provide benchmarks for care within one medical system or medical center that can be used to guide efforts in a larger scale or at other institutions.

The SQUIRE guidelines provide a format to report QI projects in a standard fashion with attention paid to key principles that are beneficial in the dissemination of QI findings. While some institutions view potential publication as grounds for formal protocol review, others do not; therefore, researchers and practitioners should become familiar with their own institutional approach to this distinction. A 2002 survey provided various scenarios to help distinguish QI from HSR and enlisted 100 quality officers, 94 institutional review board chairs, and 38 journal editors to aid in that goal.¹⁹ The work found some disagreement not only between the various categories of experts, but also between quality officers and IRB chairs from the same institutions. One proposed option at the institutional level is to create a separate board outside the typical local IRB to review QI projects to determine if IRB review is needed.^20,21 A survey of 34 academic medical centers found that only 50 percent of the centers had formal policies related to the review and approval of QI projects.²¹

Figure 2 provides a decision tree for distinguishing QI from HSR. The disclaimers for this tool are that each project must be considered individually, each institution may have a philosophy or even formal policies to address this exact issue, and local experts on IRB panels and QI leadership should be sought out for assistance in making this distinction. There have been several algorithms proposed by various organizations and institutions; the information provided here is not all-encompassing and is intended only to serve as a guide.^18,21,22

Conclusion

As surgeons continue to strive to improve the lives of their patients, and as health care reform requires us to demonstrate the value of the health care we deliver, QI efforts that address systemic issues will be an integral part of future inquiry and investigation. QI projects that address safety, effectiveness, efficiency, costs, and patient-centered outcomes are necessary as continuous QI has become an essential part of modern-day surgical practice.

Though there is often a distinction between HSR and QI, there is also significant overlap. This brief review highlights pertinent issues while providing a framework to help determine when a QI project should be considered HSR. This is an ongoing area of discussion for which the authors ultimately recommend that the surgeon’s moral compass be supported by timely and frequent discussions with institutional experts and QI personnel.

---

References

1. Kohn LT, Corrigan JM, Donaldson MS (eds). To Err Is Human: Building A Safer Health System. Committee on Quality of Health Care in America, Institute of Medicine. Washington, DC: National Academy Press; 2000.
2. Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001.
3. U.S. Department of Health and Human Services. National Quality Strategy. 2011 Report to Congress: National Strategy for Quality Improvement in Health Care. March 2011. Available at: http://www.ahrq.gov/workingforquality/nqs/nqs2011annlrpt.htm. Accessed November 16, 2013.
4. The Joint Commission. Specifications Manual for Joint Commission National Quality Core Measures (2010B). Surgical Care Improvement Project. Set measures. Available at: https://manual.jointcommission.org/releases/archive/TJC2010B1/SurgicalCareImprovementProject.html. Accessed May 19, 2014.
5. American College of Surgeons National Surgical Quality Improvement Program. About ACS NSQIP. Available at: http://site.acsnsqip.org/about. Accessed May 15, 2014.
6. Sellers MM, Hanson K, Schuller M, et al. Development and participant assessment of a practical quality improvement educational initiative for surgical residents. J Am Coll Surg. 2013;216(6):1207-1213.
7. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system—rationale and benefits. N Engl J Med. 2012;366(11):1051-1056.
8. ACGME Clinical Learning Environment Review (CLER) Program. Available at: http://www.acgme.org/acgmeweb/tabid/436/ProgramandInstitutionalAccreditation/NextAccreditationSystem/ClinicalLearningEnvironmentReviewProgram.aspx. Accessed May 15, 2014.
9. Sellers MM, Reinke CE, Kreider S, et al. American College of Surgeons NSQIP^®: Quality in-training initiative pilot study. J Am Coll Surg. 2013;217(5):827-832.
10. Sakran JV, Hoffman RL, Ko C, Kelz RR. The ACS NSQIP^® quality in-training initiative: Educating residents to ensure the future of optimal surgical care. Bull Am Coll Surg. 2013;98(11):30-35.
11. Nicolay CR, Purkayastha S, Greenhalgh A, et al. Systematic review of the application of quality improvement methodologies from the manufacturing industry to surgical healthcare. Br J Surg. 2012;99(3):324-335.
12. Davidoff F, Batalden P, Stevens D, Ogrinc G, Mooney SE; SQUIRE development group collaborators. Publication guidelines for quality improvement in health care: Evolution of the SQUIRE project. BMJ; 2009. Available at: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2769030/. Accessed May 19, 2014.
13.  U.S. Department of Health & Human Services. Code of Federal Regulations: Title 45: Public Welfare, Part 46: Protection of Human Subjects. Last revised January 15, 2009. Available at: http://www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.html. Accessed May 15, 2014.
14.  Shuster E. Fifty years later: The significance of the Nuremberg Code. N Engl J Med. 1997;337:1436-1440.
15. U.S. Department of Health & Human Services. Office of the Secretary. Ethical principles and guidelines for the protection of human subjects of research. The National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. The Belmont Report. April 18, 1979. Available at: http://www.hhs.gov/ohrp/humansubjects/guidance/belmont.html. Accessed November 13, 2013.
16. World Medical Association. World Medical Association Declaration of Helsinki: Ethical principles for medical research involving human subjects. JAMA. 2013;310(20):2191-2194.
17. Miller FG, Emanuel EJ. Quality-improvement research and informed consent. N Engl J Med. 2008;358(8):765-767.
18. Casarett D, Karlawish JH, Sugarman J. Determining when quality improvement initiatives should be considered research: Proposed criteria and potential implications. JAMA. 2000;283(17):2275-2280.
19. Lindenauer PK, Benjamin EM, Naglieri-Prescod D, Fitzgerald J, Pekow P. The role of the institutional review board in quality improvement: A survey of quality officers, institutional review board chairs, and journal editors. Am J Med. 2002;113(7):575-579.
20. Nerenz DR, Stoltz PK, Jordan J. Quality improvement and the need for IRB review. Qual Manag Health Care. 2003;12(3):159-170.
21. Johnson N, Vermeulen L, Smith KM. A survey of academic medical centers to distinguish between quality improvement and research activities. Qual Manag Health Care. 2006;15(4):215-220.
22.  Platteborze LS, Young-McCaughan S, King-Letzkus I, McClinton A, Halliday A, Jefferson TC. Performance Improvement/Research Advisory Panel: A model for determining whether a project is a performance or quality improvement activity or research. Mil Med. 2010;175(4):289-291.