Unsupported Browser
The American College of Surgeons website is not compatible with Internet Explorer 11, IE 11. For the best experience please update your browser.
Close
Member ID
Become a Member
Open Search
Menu
ACS Home
My Dashboard
Donate to the foundation
About Us
Advocacy & Policy
Quality Programs
For Medical Professionals
For Patients
Become a member and receive career-enhancing benefits

Our top priority is providing value to members. Your Member Services team is here to ensure you maximize your ACS member benefits, participate in College activities, and engage with your ACS colleagues. It's all here.

Become a Member
ACS Home
My Dashboard
Donate to the foundation
Quality Programs
For Medical Professionals
For Patients
Advocacy & Policy
About Us
Become a member and receive career-enhancing benefits

Our top priority is providing value to members. Your Member Services team is here to ensure you maximize your ACS member benefits, participate in College activities, and engage with your ACS colleagues. It's all here.

Membership Benefits
Popular searches
Frequent searches
ACS
Bulletin

The Surgical Metrics Project: What was achieved, and where is it headed?

Clinical Congress 2019 provided the opportunity for surgeons to participate in The Surgical Metrics Project. A total of 255 attendees visited The Surgical Metrics Project booth to explore the use of wearable technologies as a means of measuring surgical decision making and surgical technique.

Carla Pugh, MD, PhD, FACS, Cassidi Goll, Anna Witt, Hossien Mohamadipanah, PhD, Brett Wise

July 21, 2020

Surg Metric Slideshow1
Participants at the Surgical Metrics Project booth at Clinical Congress 2019
Surg Metric Slideshow2
Participants at the Surgical Metrics Project booth at Clinical Congress 2019
Surg Metric Slideshow3
Participants at the Surgical Metrics Project booth at Clinical Congress 2019
Surg Metric Slideshow5
Participants at the Surgical Metrics Project booth at Clinical Congress 2019
Surg Metric Slideshow6
Participants at the Surgical Metrics Project booth at Clinical Congress 2019
Surg Metric Slideshow7
Participants at the Surgical Metrics Project booth at Clinical Congress 2019
Surg Metric Fig1
Figure 1. Survey respondent Demographics (n = 255)
Surg Metric Fig2
Figure 2. The Surgical Metrics Project in action
Surg Metric Fig2b
(left) Tweet by Andrew S. Wright, MD, FACS, Seattle, WA, describing his experience with The Surgical Metrics Project; (right) As The Surgical Metrics Project database grows, Stanford researchers anticipate that they will gain a clearer understanding of the variance in surgical decision making and technical approaches.
Surg Metric Fig3
Figure 3. Attitudes regarding data and assessment (n = 255)
Previous Next

The Surgical Metrics Project Research Advisory Board Members

  • Peter Angelos, MD, PhD, FACS
  • Megan Applewhite, MD, FACS
  • Jo Buyske, MD, FACS
  • E. Patchen Dellinger, MD, FACS
  • David Hoyt, MD, FACS
  • Carlos Pellegrini, MD, FACS, FRCSI(Hon), FRCS(Hon), FRCSEd(Hon)
  • Steven Stain, MD, FACS
  • Patricia Turner, MD, MBA, FACS

Committee on Surgical Skills Training for Practicing Surgeons

  • Barbara Bass, MD, FACS, FRCS(Hon), FRCSI(Hon), FCOSECSA(Hon)
  • Ajit Sachdeva, MD, FACS, FRCSC, FSACME

Clinical Congress 2019 provided the opportunity for surgeons to participate in The Surgical Metrics Project. A total of 255 attendees visited The Surgical Metrics Project booth to explore the use of wearable technologies as a means of measuring surgical decision making and surgical technique. Spearheading the effort is Carla M. Pugh, MD, PhD, FACS, coauthor of this article and professor of surgery, and her research team at the Technology Enabled Clinical Improvement (T.E.C.I.) Center, Stanford University, CA. The initial goal of The Surgical Metrics Project is to enable information exchange and facilitate data sharing. It is imperative that surgeons lead the discussion about how their data should be managed and applied.

The Surgical Metrics Project exhibit contained 10 surgical simulation stations where participants were equipped with wearable technologies and asked to “run the bowel” and perform an open repair of any small bowel enterotomies identified in a segment of porcine intestines.

Participation in The Surgical Metrics Project

Upon entering The Surgical Metrics Project exhibit, participants completed a one-page demographic survey. A summary of the survey results are shown in Figure 1.

After providing their demographic information, participants were fitted with a wearable electroencephalography (EEG) sensor to record and measure brain activity (see Figures 2a–2c). The sensors were attached to their foreheads, at which point participants were led to the cognitive testing area to undergo a baseline assessment of their cognitive skills.

Figure 1. Survey respondent demographics (n = 255)

Participants at the Surgical Metrics Project booth at Clinical Congress 2019
Participants at the Surgical Metrics Project booth at Clinical Congress 2019

Participants were then outfitted with magnetic motion-tracking sensors on their hands and an audio recorder on their lab coat. Once participants were fully instrumented with the wearable technologies (see Figure 2d), they stepped into one of the 10 simulated surgical environments equipped with two video cameras, a stopwatch, one surgical assistant, and all of the surgical tools necessary to operate on their “patient.”

Figure 2. The Surgical Metrics Project in action

Participants at the Surgical Metrics Project booth at Clinical Congress 2019
Participants at the Surgical Metrics Project booth at Clinical Congress 2019

The research team used a template and surgical tools to pre-injure the porcine tissue in the lab such that each surgeon was presented with the same case. Surgeons had varying completion times and approaches to the procedure. The team was able to capture all of these preferences and individual decisions using magnetic motion-tracking sensors, audio recorders, video cameras, and EEG sensors.

Upon completion of the simulated surgical procedure, researchers documented the surgeon’s procedure time and performed a leak test to determine the quality of the repair (see Figure 2e).

The standardized operative surgical task and wearable technologies allowed the team to collect baseline data to better understand how surgeons’ decision making, technical skill, and communication strategies contribute to procedural outcomes.

Additional Information

Additional information and videos about The Surgical Metrics Project are available at the following links:

Reactions to The Surgical Metrics Project

The Clinical Congress News, the daily paper published during the conference, queried participants for their initial reactions to The Surgical Metrics Project. “It’s really important to get baseline data in terms of where surgeons in practice are—in both their cognitive and technical abilities,” said Sabha Ganai, MD, PhD, FACS, Springfield, IL.

Arthur Berg, DO, Hackensack, NJ, said, “This is an interesting way to see and quantify variations of surgeon techniques in terms of different movements and different decisions. It’s really great that they’re actually using objective data—EEG monitoring—to see the different variations in technique, and it will be cool to see the differences between beginners and more highly trained, experienced surgeons.”

“This can help us to understand what the differences are between novices and experienced surgeons. Once we understand the differences, we’ll be able to get novice surgeons looking more like experienced surgeons in a shorter period of time. The value is getting lots of people to do it, so having it here at Clinical Congress is a great chance to get real data that will yield real results,” said David M. Notrica, MD, FACS, FAAP, Phoenix, AZ.

In addition to the anecdotes shared with the Clinical Congress News, several participants engaged in The Surgical Metrics Project conversation on Twitter. A tweet by Andrew S. Wright, MD, FACS, Seattle, WA, described his experience with The Surgical Metrics Project and encouraged others to participate (see photo).

Participants at the Surgical Metrics Project booth at Clinical Congress 2019
Participants at the Surgical Metrics Project booth at Clinical Congress 2019

Paving the way for a new frontier of data-driven metrics to advance quality, The Surgical Metrics Project aims to include all members of the surgical community in the conversation regarding data use and data sharing. As part of The Surgical Metrics Project data collection, Dr. Pugh and her research team had the opportunity to gauge participants’ attitudes regarding data and assessment. For the responses, see  Figure 3.

Figure 3. Attitudes regarding data and assessment (n = 255)

Participants at the Surgical Metrics Project booth at Clinical Congress 2019
Participants at the Surgical Metrics Project booth at Clinical Congress 2019

Looking forward

Ongoing data analysis is now taking place and with guidance from The Surgical Metrics Project Advisory Board (see sidebar), Dr. Pugh and her research team will determine how to distribute feedback to participants. The overall goal is to develop an anonymous database of video, motion, and audio data to facilitate surgical planning, training, and review.

“The Surgical Metrics Project is the foundation of assessing surgical technique,” according to American College of Surgeons Executive Director David B. Hoyt, MD, FACS. “Once validated it can define how we might measure skills acquisition, obtain mastery, and even demonstrate deterioration of skills over time.”

The College is proud to announce the continuation of its strategic partnership with the T.E.C.I. Center and plans to continue offering The Surgical Metrics Project at Clinical Congress.

Authors’ note

The T.E.C.I. Center is a research group directed by Dr. Pugh that aims to transform human health and welfare through advances in data science and personalized, data-driven performance metrics for health care providers. More information about the T.E.C.I. Center is available on the Stanford Medicine website.