ACS Clinical Scholars in Residence Program has lasting impact on surgeons’ careers

The American College of Surgeons (ACS) Clinical Scholars in Residence Program prepares early career clinicians to use data-driven research to address issues in health care quality, health policy, and patient safety. The two-year fellowship program, initiated in 2005, provides ACS Clinical Scholars with exposure to the ACS National Surgical Quality Improvement Program (ACS NSQIP^®), the National Cancer Database (NCDB), the National Trauma Data Bank^® (NTDB^®), the Trauma Quality Improvement Program, and other data registries with the dual goal of conducting research to enhance the College’s quality improvement (QI) initiatives and preparing residents for a career in academic surgery.¹

ACS Clinical Scholars also earn a master’s degree in health services and outcomes research or health care quality and patient safety at Northwestern University’s Medical School, Chicago, IL, during their two years at the ACS headquarters in Chicago. The goal of the master’s program is to educate clinicians in health care services and outcomes specifically within institutional and health care delivery systems, as well as in the external environment that shapes health policy.

In addition, ACS Clinical Scholars are assigned mentors representing a diverse background to guide participants in their research.

The program is open to Resident Members of the College who have completed two to three years of surgical training. In total, 16 scholars have completed the program; at present, five residents are ACS Clinical Scholars.²

In this article, four previous ACS Clinical Scholars describe how the fellowship affected their career path and why this program continues to result in improved patient outcomes and safer patient care.

Dr. Bilimoria: Defining the role of the ACS Clinical Scholar

As the inaugural ACS Clinical Scholar in Residence, Karl Bilimoria, MD, MS, FACS, performed numerous studies focused on improving care for surgical and oncology patients using data from the NCDB. He also worked extensively on ACS NSQIP to assess hospital surgical quality data, developing multiple initiatives for that program.

Before entering the ACS Clinical Scholars program, Dr. Bilimoria attended medical school at Indiana University, Indianapolis, and completed his general surgery residency at Northwestern University. In 2011, he entered a surgical oncology fellowship at MD Anderson Cancer Center, Houston, TX.

“The goal was to use the NCDB for a wide variety of research in order to identify what are good research uses of the database and then to try to improve the quality of the data and the NCDB through what we learned through the research process,” Dr. Bilimoria said. “We also needed more quality measures, so a lot of the work centered on quality measure development and testing. I also did some of the initial work in building out some aspects of the very early NSQIP program, such as developing the ACS NSQIP Surgical Risk Calculator.”

At the time, residents had few opportunities to engage in this kind of work, and it was the College’s first foray into this type of research, Dr. Bilimoria said. He first proposed the idea for the program to David P. Winchester, MD, FACS, Medical Director, ACS Cancer Programs, and then ACS Executive Director Thomas R. Russell, MD, FACS, who offered their enthusiastic endorsement.

Dr. Bilimoria said the program’s rollout went relatively smoothly. “It was not a matter of overcoming specific challenges, but of really trying to define what the role of the scholar could be,” he said. “It really was like being a kid in a candy store—you could set it up in any number of ways. The idea of being able to have access to some of the best data for research was very appealing, as was the opportunity to have a hand in crafting and developing the College’s Quality Programs.”

A key component of the Clinical Scholars program is the opportunity for young clinicians to earn a Master of Science in Health Services and Outcome Research. “We wanted to make sure that if the Fellows were doing this kind of work that they received formal training as well, and Northwestern has been a huge partner in reaching this goal,” Dr. Bilimoria said. “You need this formal training and the underpinnings of research methodology to be able to really have a diverse set of tools for research, specifically skills related to study design, biostatistics, and epidemiology.”

Today, Dr. Bilimoria is a surgical oncologist focusing on melanoma and sarcoma at Northwestern Memorial Hospital, and he is the founding director of the Surgical Outcomes and Quality Improvement Center (SOQIC), within the Feinberg School of Medicine. The SOQIC has earned more than $30,000,000 in research funding and has published more than 300 articles. Dr. Bilimoria attributes this success to his time as an ACS Clinical Scholar.

“The Clinical Scholars program has served as the foundation for everything I do now in my research, quality improvement, and health policy endeavors,” Dr. Bilimoria said. “One of the biggest things I do right now is oversee the Illinois Surgical Quality Improvement Collaborative, which is composed of 57 hospitals throughout Illinois and is one of the biggest and most robust collaboratives in the country.”³

An extensive knowledge of the ACS NSQIP program and data also contributed to Dr. Bilimoria’s role as principal investigator for the Flexibility in Duty Hour Requirements for Surgical Trainees (FIRST) Trial published in 2016.⁴ The FIRST Trial randomized 117 U.S. general surgery residency programs and 151 affiliate hospitals to different duty-hour policies and showed that flexibility in resident work hours does not affect patient safety or overall resident well-being.

“Having a deep understanding of how ACS NSQIP works really helped lead the way to developing the FIRST Trial,” Dr. Bilimoria said. “The FIRST Trial is the first of its kind, and labeled as “first” for a variety of reasons, including the fact that it was one of the first registry trials that was done with NSQIP—which makes it pretty unique, and I hope that it serves as a model for many other investigators to do registry trials going forward.”

In addition to working with ACS databases and receiving formal instruction in research methodology, the ACS Clinical Scholars program offers junior clinicians access to experienced mentors.

“The mentorship part is a critical component of what I consider my responsibilities right now,” explained Dr. Bilimoria. “Working with bright, curious residents is constantly refreshing. And giving them some basic tools for research, policy, politics, and professional development is enjoyable, but I also learn a ton from them. The Clinical Scholars are constantly questioning clinical issues and coming up with creative research ideas, and that back and forth is really important. I think I was like this as a scholar, too.”

For example, a research fellow suggested a research idea that Dr. Bilimoria thought was uninteresting. “But the research fellow was persistent in making a case and ran some preliminary data, and then brought it back. And when I looked at it again, I realized it would probably be a landmark study, and that the fellow’s persistence and view of the situation was correct. And I loved that the fellow didn’t give up on it despite getting an unfavorable initial review from me.”

Mark Cohen, PhD, Statistical Manager, Continuous Quality Improvement (CQI), ACS Division of Research and Optimal Patient Care (DROP-C), also said he has learned a lot as a mentor to Clinical Scholars. “Even in the statistics realm, scholars sometimes use approaches for their research that we’re not very familiar with,” Dr. Cohen said. “It’s a refreshing role reversal when we learn about new methods from them.”

Dr. Bilimoria said that participating in the ACS Clinical Scholars program is a singular experience, particularly for residents interested in pursuing a career in surgical health services, health policy, or quality improvement research.

Dr. Ingraham: Developing writing skills

Angela Ingraham, MD, MS, an ACS Scholar in Residence from July 2008 to June 2010, has focused her research on emergency general surgery. As a Clinical Scholar, Dr. Ingraham’s research examined patient- and hospital-level outcomes following emergency general surgery using data from ACS NSQIP.

Dr. Ingraham received her medical degree from Loyola University Chicago Stritch School of Medicine, IL. She was a general surgery resident at the University of Cincinnati, OH, and an acute care surgery fellow at the University of Pittsburgh School of Medicine, PA. At present, Dr. Ingraham is an assistant professor of surgery, University of Wisconsin (UW) Madison, and she is an investigator in the Wisconsin Surgical Outcome Research Program, UW.

Until recently, emergency general surgery patients really didn’t have a “home” in terms of quality, Dr. Ingraham said. “Through my research, as well as many of the collaborations and connections that I made through the Clinical Scholars program, I’ve been able to highlight this very vulnerable group of patients and the fact that we need to focus QI efforts in this area.”

Improving the quality of care begins with solid data analysis. Dr. Ingraham said one of the most valuable aspects of the program was learning how to communicate with statisticians. “As a scholar, I did my own data analysis, and I got to learn about, for example, SAS (previously known as the Statistical Analysis System), which is one of the most commonly used statistical programs and was developed by the SAS Institute. Today, as a faculty member, I don’t always do my own data analysis; but having previously run my own analyses, I have learned the ‘language’ and how to communicate with the statisticians so we can produce sound results.”

Through the scholars program, she learned how to create “spec sheets,” which define all of the data variables and the anticipated analyses. In the end, these spec sheets form the basis of a lab book for the project, providing a record of how the researchers made their decisions and why they excluded certain people from the analysis. “I think this documentation also helps the statisticians, because you want to make sure the code they’re writing reflects your ideas of how the analysis should be carried out,” Dr. Ingraham said.

“Clinical scholars usually have two years at the College during which they are expected to earn a master’s degree and publish extensively on clinical topics relevant to the College’s quality mission,” Dr. Cohen said. “That’s a lot to ask, particularly as scholars typically need to pick up training in research methods, statistics, and statistical coding before the hands-on data analysis can begin. On the other hand, the CQI statistics team knows the data better than anyone, is very knowledgeable about how to design quality-related research, and has a large library of data manipulation and analysis code already on the shelf. We can get scholars up and running in doing their research in a matter of weeks or a few months, rather than years.”

“Another opportunity provided by the Clinical Scholars program was that it allowed us to further hone our research questions based on our clinical experience. As surgeon-scientists, we have the ability to have our clinical work inform our research programs, and I’ve been able to do just that in my current work,” said Dr. Ingraham. “I noticed in my clinical realm that patients were being transferred from one acute care hospital to another for many reasons. I saw that there were opportunities for improvement in the transfer process, and this is what I have built my research program around—improving that transfer process between acute care hospitals,” she said.

As a Clinical Scholar, Dr. Ingraham completed a Master of Science and Clinical Investigation. “That program was key, at least partially, to my current success,” she said. “While physicians without advanced degrees can definitely pursue health services and other basic science research, earning that master’s gives you both credibility and extra knowledge.”

A notable component of the master’s program at Northwestern University is a course on professional writing. “In addition to the master’s program, the other advantage I had in terms of writing were my Clinical Scholar mentors, particularly Karl Bilimoria,” she said. According to Dr. Ingraham, they were in between presentations at a conference when Dr. Bilimoria mapped out for her and Dr. Mehal Raval (another Clinical Scholar profiled in this article), on the back of a napkin, how to approach a scientific manuscript. The napkin-turned-writing guidelines continue to hang on Dr. Ingraham’s office wall today as a reminder of how to format and organize a manuscript.

“The Clinical Scholars program gave me the opportunity to be fairly prolific during those two years at the College, because I had that dedicated time and I had the data available to me. I’m using that experience today as I continue to work to get my research program externally funded,” she said, referring to an Agency for Healthcare Research and Quality K08 award that was under review at press time.

As for advice to other physicians who are considering participating in the ACS Clinical Scholars program, Dr. Ingraham suggests planning ahead. “Talk to your home institution about it early. I think we’re fortunate now in academic surgery that health services research is becoming more accepted as a formal pursuit during the two to three years that people have when they take time off to go into the lab during their residency training. Departments often make budgets a year or two in advance, and so for them to be able to set aside money, you need to make them aware of your interest as soon as possible,” Dr. Ingraham said.

Dr. Paruch: Using data to implement change at the local level

Jennifer Paruch, MD, FACS, joined the ACS as a surgical oncology Clinical Scholar in Residence in July 2012, after completing three years of general surgery residency at the University of Chicago. As an ACS Clinical Scholar, she worked on several QI initiatives, including enhancing the ACS NSQIP Surgical Risk Calculator, developing cancer quality measures, and evaluating hospital performance in cancer surgery.

As a Clinical Scholar, Dr. Paruch participated in a study that examined variation in patients undergoing hepatectomy. “We looked at the complexity of liver resections and whether additional procedures at the time of resection impacted surgical complication rates and whether including that variable in modeling changed where hospitals fell in terms of their ACS NSQIP modeling—which it did not,” said Dr. Paruch. The study, which examined 11,826 patients who underwent hepatectomy at 261 hospitals, was published in the Annals of Surgical Oncology in June 2014, and was co-authored by several of her ACS Clinical Scholar mentors, including Dr. Bilimoria; Clifford Y. Ko, MD, MS, MSHS, FACS, Director, ACS NSQIP, and Director, ACS DROP-C; and Dr. Cohen.⁵

“My experiences as an ACS Clinical Scholar spurred my interest in how we take data and use it to implement change within individual institutions based on best practices and all of the available evidence,” she said. “My time at the College made me interested in how that next part happens, and that’s sort of what I am focusing my energy on now: How do you take great data from ACS NSQIP and the NCDB and actually implement QI at the local level?”

Dr. Paruch, a colorectal surgeon, practices at NorthShore University Health Systems, Evanston, IL, an affiliate of the University of Chicago. “We were one of the first hospitals to get Epic [an electronic health records data system] and because of that, we have access to a huge data warehouse. One of the projects I’m looking at is variation in opiate prescriptions within our institution. I’m sure once I actually get the data, I’ll use a lot of the statistics experience that I got from my time at the College to analyze that and think about different ways to look at it—and then how to harness that data to implement policies at our institution to minimize overprescription of opiates for outpatient surgery.”

As an ACS Clinical Scholar in Residence, Dr. Paruch earned a Master of Science in Clinical Investigation. “Speaking with other residents from our program who have tried to work on those projects without having had the opportunity to have any formal training, they struggle a lot more with how to design a study or how to pick the right statistical test to answer their specific question.”

She also values the mentorship she received as a Clinical Scholar, including guidance from Dr. Ko, who helped start the ACS Clinical Scholars program and currently oversees the initiative with several other core faculty members.

“My role as a mentor is to try to provide as comprehensive an experience as possible,” said Dr. Ko. “The health care environment is changing, surgery is changing, and academics is changing. For me, mentors have been invaluable in ‘personalizing’ my path, whether it was introducing me to key people, opening my eyes to different viewpoints, content supplementation, or pushing me to think more.”

“The scholars have the opportunity to be mentored directly by internationally renowned researcher surgeons and policy leaders like Dr. Ko—but perhaps just as importantly, they receive ‘indirect’ mentoring by being exposed to the leaders of our profession, such as ACS Executive Director David B. Hoyt, MD, FACS, on site at the College and at a variety of professional meetings and discussions,” added Bruce L. Hall, MD, PhD, MBA, FACS, Consulting Director, ACS NSQIP and ACS Clinical Scholar mentor. “These include interactions with policymakers, payors, nongovernmental organizations, the National Quality Forum, specialty societies, information technology experts, and outsider advocacy groups.”

“One thing that was really helpful, from a mentorship standpoint, is how to write a paper. Dr. Bilimoria used to tell us to write our abstract [first] and lay out our tables and figures even before we had our final data analysis,” Dr. Paruch said. “I think [this approach] gives you a good structural framework and helps you focus on the most important pieces that you want to highlight with that research early on. And it actually makes you more efficient in your data analysis.”

Dr. Raval: Mentoring is vital to the young surgeon investigator

As a Clinical Scholar in Residence from July 2008 to June 2010, Mehul V. Raval, MD, MS, FACS, FAAP, worked on several QI initiatives, including an assessment of multispecialty representation and case-mix adjustment in ACS NSQIP. He also assisted with the formation of the Illinois ACS NSQIP collaborative that Dr. Bilimoria mentioned previously, and developed online compliance reports for the ACS Bariatric Surgery Center Network (now the Metabolic and Bariatric Surgery Accreditation and Quality Improvement Program).

As an ACS Clinical Scholar, Dr. Raval worked on the initial testing of the ACS NSQIP Pediatrics module (now the ACS NSQIP Pediatric program). “During my time as a Clinical Scholar, we grew the ACS NSQIP Pediatric Program from an alpha phase that was in place at about four sites around the nation to a program that had about 30 centers,” said Dr. Raval, who received his medical degree from Wake Forest University School of Medicine, Winston-Salem, NC. Today, ACS NSQIP Pediatric has more than 100 participating hospitals.

Dr. Raval presented the initial ACS NSQIP Pediatric data as a Clinical Scholar at the American Pediatric Surgery Association meeting in 2010, for which he was awarded the M. Judah Folkman Best Podium Presentation Award. “This recognition really validated the work that we were doing and made me feel like I was actually making a big contribution to our field.”

The opportunity to engage with surgeon leaders both inside and outside the ACS is a fundamental component of the Clinical Scholars program. “As a resident, you may hear these leaders present at a national conference,” Dr. Raval said. “But you may not get to work closely with them on a specific project or on a recurring basis as you do as an ACS Clinical Scholar. On top of that, people outside the College are looking for opportunities to collaborate with the ACS, and the scholars become this medium through which the College leadership can communicate with the leaders in a specific surgical specialty, like pediatric surgery, and vice versa.”

Dr. Raval said the mentoring he received as a Clinical Scholar was a pivotal part of his development as a surgeon investigator. “One of the biggest strengths of the ACS Clinical Scholars program is that you get exposed to phenomenal mentors,” said Dr. Raval. “I can still hear Cliff Ko’s voice in my head asking me, ‘So what?’ after almost every project that I proposed. He did that to make me ask the questions that mattered the most, and not just ask the questions that were easy to answer with the database.”

“Asking the big questions, answering them with focus, and having a team approach are just a few lessons that I learned from my Clinical Scholar mentors. These lessons are ones that I continue to employ each day,” said Dr. Raval, who completed a Master of Science and Clinical Investigation as a Clinical Scholar.

Today, Dr. Raval is an associate professor at Ann & Robert H. Lurie Children’s Hospital and the SOQIC at Northwestern University’s Feinberg School of Medicine. He spends half of his time as a clinical surgeon taking care of pediatric patients and the other half as a health services researcher—with significant overlap between the two. His specific research interests include QI efforts for pediatric surgical patients, including econometrics, value-based care, and health policy. These research interests, he said, are directly related to his work developing the ACS NSQIP Pediatric platform as a Clinical Scholar. “My work today continues by making [this platform] more practical and applicable to how we make decisions regarding limited health care resources and how we envision the health care systems of the future.”

Conclusion

“The College must cultivate the brainpower, fresh ideas, and new perspectives that young surgeons have to offer,” Dr. Russell wrote in his “From my perspective” column in the March 2009 issue of the Bulletin.⁶ In that column, he emphasized the importance of “expanding the scope of fellowships available to young surgeons.”

“The initial discussions about this program between Dr. Russell and myself involved developing something that was akin to other research experiences—only better,” said Dr. Ko. “Before coming to the College, I was an associate director at the University of California Los Angeles/RAND site for the Robert Wood Johnson Foundation Clinical Scholars Program. That program was a national research training program with over 30 years of experience of developing researchers, but across many different specialties. From that program, we had a pretty good road map for creating a research experience at the ACS specifically for surgeons.”

Acknowledgment

The author gratefully acknowledges the assistance of Mehwesh Khalid, Executive Associate, and Rachel Sarcletti, Project Coordinator, CQI, DROP-C, Chicago, IL.

References

1. American College of Surgeons. ACS Clinical Scholars in Residence. Available at: facs.org/quality-programs/about/clinical-scholars-program. Accessed May 1, 2018.
2. Bilimoria K, Ko CY. ACS Clinical Scholars in Residence benefit from access to outcomes measures and mentors. Bull Am Coll Surg. 2018;103(1):74-75.
3. Northwestern Medicine. Feinberg School of Medicine. Institute for Public Health and Medicine. Illinois Surgical Quality Improvement Collaborative. November 8, 2013. Available at: www.feinberg.northwestern.edu/sites/ipham/research/awards/Bilimoria-ISQIC.html. Accessed May 1, 2018.
4. American College of Surgeons. FIRST Trial shows flexibility in resident work hours does not affect patient safety. Bull Am Coll Surg. 2016;101(3):53-54.
5. Paruch JL, Merkow RP, Bentrem DJ, et al. Impact of hepatectomy surgical complexity on outcomes and hospital quality rankings. Ann Surg Oncol. 2014;21(6):1773-1780.
6. Russell TR. From my perspective. Bull Am Coll Surg. 2009;94(3):4-6.