Olga M. Jonasson, MD, Lecture: Never let a good crisis go to waste: Continuous professional development and COVID-19

Editor’s note: The following is an edited version of the Olga M. Jonasson, MD, Lecture that Dr. Greenberg delivered at the virtual Clinical Congress 2020. The presentation has been modified to conform with Bulletin style and space limitations.

I would like to express my deepest gratitude to the American College of Surgeons (ACS) Women in Surgery Committee for the tremendous honor of delivering the Olga M. Jonasson, MD, Lecture at Clinical Congress 2020. I never had the privilege of meeting Dr. Jonasson, but I’d like to think that she would be proud of the difficult conversations about gender equity taking place in the surgical community today.

Challenges to introducing disruptive technology in health care

I often speak about those topics, but my focus here is on a different issue—one related to the new and unique times in which we are living—in particular, how we might leverage opportunities presented during the coronavirus 2019 (COVID-19) pandemic to think differently about continuing professional development (CPD).

There are many examples of technologies that have disrupted other industries. Think about how rideshare apps like Uber have affected the taxi industry, how short-term housing rentals like Airbnb have changed the hospitality industry, how Amazon.com has disrupted shopping malls and brick-and-mortar stores, and how smartphones have disrupted every facet of life and work.¹

However, we find few examples of this kind of abrupt technology disruption in health care delivery. Why is that? Several aspects of health care make it difficult to introduce disruptive technology.^1-3

First, health care is incredibly wide-ranging and complex. There are fragmented systems of care with distributed decision-making by a diverse set of stakeholders. All of this makes it much more complicated than an individual deciding to open their Uber app rather than hailing a taxi. Technology within health care is poorly coordinated across these different systems of care. It is a very regulated industry and rightfully so.

I want to introduce this quote from Rahm Emanuel, White House Chief of Staff during the Obama Administration and former Mayor of Chicago, IL, at the beginning of the talk to provide context for the lecture’s title. The quote is believed to have originated with Machiavelli, and then was attributed to a number of other people throughout time including Winston Churchill, but I think Mr. Emanuel said it best: “You never want a serious crisis to go to waste. It provides the opportunity to do things that were inconceivable before.” ⁴

COVID-19: The great disruptor

Consider what has been happening with telehealth over the last year. If we think back to 2019 before the pandemic hit, we had the evidence to support telehealth as an effective way to deliver health care for a long time. We also had the capacity to use it. Yet, penetrance was poor. In fact, a 2020 survey of hospital chief executive officers (CEOs) suggested that only 38 percent of hospital CEOs had a digital component to their strategic plan.⁵ Lack of interest in telehealth is partly attributable to our payment structure, which offers little incentive to provide telehealth because in-person visits have been reimbursed at a much higher rate. Furthermore, complex data protection privacy regulations related to the Health Insurance Portability and Accountability Act (HIPAA) and the Health Information Technology for Economic and Clinical Health Act (HITECH) prevented us from introducing telehealth in a meaningful way.⁶

Enter COVID-19. Congress and the Centers for Medicare & Medicaid Services said they would reimburse telehealth services for all Medicare beneficiaries with payment parity, and other insurance carriers have followed suit. The U.S. Department of Health and Human Services (HHS) Office of Civil Rights agreed not to penalize providers for HIPAA violations if practices and hospitals use telehealth in good faith.⁷ As a result, suddenly hospitals were able to operationalize telehealth capacity at an unprecedented rate. We now perform many of our visits by video or telephone, making telehealth an incredibly disruptive innovation. COVID-19 probably has catapulted us forward 10 years compared with what would have happened had we not experienced this pandemic.

My goal is to convince you that COVID-19 presents a unique opportunity for a similar technology disruption when it comes to CPD. Opportunities to engage in CPD are particularly relevant because of significant new institutional limits on travel and spending for professional development. Many of us saw these limits starting even before COVID-19, but the pandemic presented us with the additional impediments of travel bans and limits on large gatherings, which make it difficult for us to meet in person. Yet, we still need opportunities for Continuing Medical Education (CME) and CPD. So, the question I want to ask today is, how do we leverage technology to meet these needs and perhaps be a little disruptive in the process?

First, I want to acknowledge how tremendously effective we have been at moving our CME programs online in a very short time frame. I was doubtful about how we would be able to interact when we moved our professional meetings to a virtual format, but recently I had the opportunity to participate in a women’s leadership in surgery conference and found it incredibly easy to engage. What struck me in talking to the course directors was how many more people were able to attend in the new format. So, a major advantage of moving our professional meetings to a virtual setting is increased penetrance.

But it comes at a cost, and I think a Twitter exchange between two of our surgical chairs illustrates that point nicely (see Figure 1). They begin by talking about the fact that our surgical meetings were canceled, and it has been great because everything has been moved on to Zoom so travel is unnecessary. Unfortunately, organizers quickly realized that without travel, surgeons stay busy during working hours. As a result, many organizations have been scheduling meetings after hours and on weekends. Three out of the four weekends in October 2020, I was scheduled to give a talk or participate in a conference. Given the ongoing pandemic and challenges that we are going to face in 2021, we need to think about how we protect our personal time and not just add CPD to our other responsibilities.

FIGURE 1. SURGICAL CHAIRS DISCUSSING THE UPSIDE AND DOWNSIDE TO SURGICAL MEETINGS DURING COVID-19

Moving from innovation to disruption in education

Moving programs online is innovative. As a discipline, we have accomplished some great things, but I want us to consider how we move from innovation to disruption. One of the best examples of what disruption means can be found in the National Basketball Association. A graph that appeared in the New York Times shows three-point shots that were taken and three-point baskets that were made in a season.⁸ As the graph illustrates, Stephen Curry had the two best seasons of all time until 2016, when he shattered everything that anybody had thought was possible, nearly doubling the number of three-point shots that he made that year. He did so by questioning traditional training methods and being willing to try new things.

Some of the innovative training approaches he used included the regular use of sensory deprivation chambers. He used goggles that had strobe lights and inhibited his ability to see, and then something called overloading, where he performed multiple tasks at once. As a result, when he was in a game and all he had to do was shoot, it was that much easier for him to do it. He also reimagined the ways that three-point shooting could occur. He started taking shots from places nobody else ever did, thereby expanding the size of the area that the other teams needed to defend by more than 25 percent, and that redefined the game and opened up more opportunities to score. If he could not take a three-point shot, there was more open court for him to find an open teammate to pass to.

Disruption in ongoing board certification

Next, I want to talk about Maintenance of Certification (MOC). In 2005 the American Board of Surgery (ABS)introduced the MOC program. MOC had four parts. A surgeon had to demonstrate maintenance of good professional standing, a commitment to lifelong learning and self-assessment, cognitive expertise, and ongoing evaluation of performance in practice.

The first requirement was relatively easy to fulfill. It meant having privileges and peers who were willing to write a letter of support if asked. The expectations for demonstrating cognitive expertise also were relatively straightforward. We relied on a traditional, multiple choice exam, but lifelong learning and self-assessment and evaluation of performance in practice were a bit more challenging.

That same year, Ajit Sachdeva, MD, FACS, FRCSC, FSACME, Director, ACS Division of Education, wrote an article for the Archives of Surgery (now JAMA Surgery) in which he described a new paradigm in CME for surgeons.⁹ He compared traditional CME with the new idea of CPD. He said CME was based on episodic interventions designed to address the educational needs of groups of learners; therefore, it was generally teacher-centered and teacher-driven. It often was delivered didactically and in formal settings, such as lecture halls or conference rooms. I would posit that most of our current activities adhere to this traditional approach.

CPD, on the other hand, as Dr. Sachdeva described it, is rooted in the concept of lifelong learning based on ongoing self-assessment of the individual learner’s educational needs. It is learner-centered and learner-driven. It is comprehensive in scope and can go beyond the clinical domain to assess practice management, leadership, and practice administration. It uses a range of learning formats and media. It is conducted in a variety of venues, including locations outside of the traditional educational setting. MOC and CPD both promoted the idea that we need to examine our practice. We need to find ways that we want to improve, make intentional changes to what we do, and then figure out what happens when we make those changes.

Proponents of MOC say it is voluntary, it improves knowledge, it ensures lifelong learning, it helps surgeons to keep up with advances in care and develop better practices, and it is peer-developed and externally validated. Opponents say it is involuntary because you must do it to maintain board certification. They say the mandates are burdensome, expensive, irrelevant, and not evidence-based. Other specialties expressed strong objections initially, but general surgeons, by and large, were relatively amenable to MOC.^10,11

Given growing controversy over MOC, in 2018, the ABS issued an update and eliminated the term MOC, changing it to Continuous Certification (CC). They changed the every 10-year test to every two years, which can be taken online and is open book and interactive. The ABS also decreased the requirements for CME hours and eliminated the requirements for self-assessment but did maintain the requirement for evaluation of performance in practice.

A year later, the American Board of Medical Specialties (ABMS) looked at the future of CC across all disciplines and issued a vision statement for moving forward.¹² The CC Vision Statement recommends that each board offer an alternative to the written exam. Probably most importantly, it promotes the concept that CC is unique from initial certification and should be formative to support practice advancement. It is not about pass/fail; rather, it is about how you continue to improve. To do so, learning activities need to be grounded in adult education and training principles. They need to be individualized and goal-directed, occur frequently, and offer opportunities for timely feedback. Activities should be repeated to ensure reinforcement and should be based on a gap analysis that aids in focus and helps to individualize learning opportunities to each surgeon’s unique goals.

The other important point this vision statement makes is that procedural skills must be addressed for relevant specialties. The authors acknowledge that video-based technologies and assessment capacity are just being developed, but as these tools are introduced, the disciplines must think about how to incorporate them in a meaningful way.

Let’s take a step back and think about where we were before the vision statement was introduced. MOC and CPD both were focused on education and quality and the advancement of both. However, we really thought about them as distinct domains. We thought about lifelong learning and self-assessment and defined those components as reading journal articles, going to grand rounds or other didactic courses, attending professional meetings, and taking national courses. With respect to evaluation of performance in practice, our opportunities are quite limited. Most surgeons meet these criteria because their hospital participates in a quality assessment program, yet assessment at this level is insufficient for meaningful evaluation of performance in practice.

My interpretation of the early ABS focus on lifelong learning, self-assessment, and evaluation of performance in practice was to encourage surgeons to begin to examine the intersection of quality and education (see Figure 2). Our educational activities should be targeted to improve the quality of the care that we provide. I would argue that we have made many innovations in the years since 2005 to support meaningful evaluation of performance and meaningful practice change. Yet, like telehealth prior to the pandemic, there have been small pockets of adoption, but penetrance into broad practice has been limited because of some challenges.

FIGURE 2. ABS CONTINUOUS CERTIFICATION AND CPD EXIST AT THE INTERSECTION OF QUALITY AND EDUCATION

These challenges include time and funding, but also the fact that the bar for meeting our professional requirements is quite low at present. It is hard when we have limited time, limited money, and limited energy to seek out more innovative ways to evaluate our performance in practice when there are easier and more immediate ways to meet our requirements. But what would it take to meet those requirements and engage in CPD in a meaningful way?

First, you need a quantitative assessment of your performance. You must know your starting point, identify areas for improvement, and set goals. Where could you be better, and how much better would you like to be? How would you define success? You then must take time to reflect on your practice and make intentional adjustments with someone or something holding you accountable.

Collaboratives provide opportunities to improve performance

The growing number of surgical quality collaboratives have supported CPD in this kind of innovative and meaningful way. Several collaboratives are based on the ACS National Surgical Quality Improvement Program platform. Other collaboratives use their own data platforms such as the Abdominal Core Health Quality Collaborative, a national quality collaborative, or existing data from partners such as hospital associations, including the Surgical Collaborative of Wisconsin.

Surgical quality collaboratives support CPD by providing benchmarked performance reports, which provide a quantitative assessment of your clinical performance. Collaboratives have regular meetings that allow individuals to review their performance and identify areas for improvement, primarily by comparing themselves with their peers to see what they could do differently. When it comes to reflection on practice, programs such as site visits and practice coaching, and different tools, such as implementation tool boxes, can help support intentional adjustments.

Several collaboratives recently have been able to take this paradigm to the next level, measuring and improving performance on an even more granular level. The surgical quality collaboratives in Michigan, which have been in existence the longest and probably have developed the best culture around continuous quality improvement, offer some of the best examples.

After a decade of tracking and sharing risk-adjusted outcomes data, the Michigan Bariatric Surgery Collaborative (MBSC) comprised a group of surgeons who were committed to quality improvement and trusted each other. The leaders of the organization asked MBSC surgeons to send a video of themselves doing a gastric bypass.¹³ They then sent the recording to other members of the collaborative and had them rate each other on a scale of one to five, with one being anchored where you should be when you finish your chief residency or fellowship and five representing the work of the best bariatric surgeon out there. They found wide variation in performance based on peer assessment. Perhaps the more important part of the study was the ability to capitalize on their extensive collection of risk-adjusted outcomes. They were able to show a linear relationship between surgical skill and risk-adjusted complication rates.

The person with the lowest score had a complication rate nearing 20 percent, and the person with the highest score had about a 4 percent complication rate, and this held up whether you looked at reoperation, readmission, emergency department visits, or any other complication, including mortality. This landmark study has been repeated, and a growing body of literature links peer review and grading of technical skill with individual surgeons’ risk-adjusted patient outcomes.^14,15

The MBSC then partnered with the Academy for Surgical Coaching to develop a video-based coaching program. We took the same surgeons who had participated in that study and enrolled 26 of them in a video-based coaching program between October 2015 and February 2018. At the same time that we were doing this study within the MBSC, two other collaboratives—the Michigan Surgical Quality Collaborative and the Illinois Surgical Quality Improvement Collaborative (ISQIC)—similarly started to look at video review as a way to supplement and help surgeons evaluate and examine their individual performance in the operating room (OR).

With that background, I want to explain why this now is a pivotal moment to introduce a new, more individually meaningful approach to CPD. I make this statement for the following reasons:

We have a professional responsibility, given the emerging data on the relationship between surgical skill and patient outcomes, to make certain we are looking at surgical skill and doing everything possible to optimize performance.

The ABMS vision for the future of CPD demands disruptive action. The boards are challenging us to monitor and evaluate performance in a different way.

Professional societies need to increase their value and offerings given financial constraints and travel limitations. The same level of support for travel to meetings that we have had in the past may not be reinstated following COVID-19. We must think about how our professional societies are going to continue to engage us in meaningful CPD activities. COVID-19 has forced us to go virtual.

Audio-video capture in the OR

Let’s look at what video-based capabilities exist to support these types of activities, including the following: the capacity for video-based capture and sharing, video-based assessments, and video-based improvement. Many platforms allow for video-based capture and sharing, and several different video-based assessments are being developed. Also available are approaches to video-based improvement, including my favorite: video-based coaching.

Video-based capture and sharing

Probably the best-known video-based capture and sharing platform is Surgical Safety Technologies OR Black Box, which came out of Toronto, ON, and was developed by Teodor Grantcharov, a Canadian surgeon. It was honored as one of Time’s Best Inventions in 2019.¹⁶ The OR Black Box takes in a variety of different data points, including video and audio of the OR, the scopic cameras and wearable technology, patient physiology, environmental factors, devices, and so on. It performs different types of analysis, depending on what you want to view. It can assess for hazards and risks, safety threats and resilience, surgical team performance, surgical technology, and efficiency.

Another approach emerged from the ISQIC’s video-based review. Members of the Illinois collaborative wanted to have a robust video-based platform for sharing surgical procedures. They developed VistaVideos, which is being expanded as a free video-sharing platform that allows surgeons in practice to give and receive feedback on their surgical skills to obtain CME credits.

Many other solutions have been developed, some with roots in academics and others in industry. For example, C-SATS (Crowd-Sourced Assessment of Technical Skills) started at the University of Washington, Seattle, and was acquired by Johnson & Johnson. TheatOR, Caresyntax, Proximie, TouchSurgery, SigmaSurgical, OrpheusMedical, and Surgus are just some of the platforms that are on the market, and many of them were created by surgeons who recognized a major gap in our ability to collect and share video data.

Video-based assessments

Validated performance assessment tools have been available for some time, including the Objective Structured Assessment of Technical Skills (OSATS), which originally was developed in Canada, and several others. These assessment tools have been used in a variety of settings and are being transitioned now into video-based analysis and assessments. More recently, surgical societies have started taking responsibility for assessment and are beginning to develop robust procedure-specific assessments.

One of the first programs was initiated within the American Society of Colon and Rectal Surgeons to develop an assessment tool for right colectomy.¹⁷ Individuals uploaded unedited intraoperative video of themselves performing a right colectomy. Experience level was assessed by expert reviewers based on eight criteria, and an overall assessment determined the level of the person being assessed. The reviewers validated and demonstrated that they could reliably stratify surgeons by skill level using this tool.

The most robust work currently exists within the Society of American Gastrointestinal and Endoscopic Surgeons (SAGES), which is building a toolbox of these video-based assessments, starting with laparoscopic fundoplication. They used a rigorous development and validation process, starting with input from 42 subject matter experts whom they interviewed.¹⁸ Panelists sought to obtain consensus on procedural steps, required skills, and common errors. Their findings then were converted into task inventory questions, which were distributed to 188 subject matter experts, who then gave feedback on the criticality and difficulty of each procedural step, the impact of common errors, and technical skills required. In the end, SAGES’ assessment encompasses 5 procedural steps, 11 sub-steps, 9 required skills, and 42 common errors. It is a complex, detailed analysis or assessment that can ensure surgeons are ready to perform this complex operation.

At present, we are dependent on individuals to review these videos and score them. That process can take a lot of time, especially when we rely on surgeons. Great work emerged during the development of C-SATS, which showed that it is possible to distribute this assessment to laypeople and use crowdsourcing to get accurate assessments.¹⁹ Given the novelty of video-based assessments, many surgeons are not quite comfortable with the idea of laypeople assessing our skills. As a result, we primarily rely on surgeons to do it. While this is expensive and logistically challenging, many surgeons learn by doing these assessments, so participating in the reviews also can be a valuable opportunity.

A variety of surgical groups are doing some exciting work to automatize this process. Carla Pugh, MD, PhD, FACS, at Stanford University, CA, and her team are leaders in this field and are developing an approach that uses artificial intelligence (AI) to analyze video and score assessments. The Surgical Metrics Project uses AI to distinguish the different phases of an operation, to identify different intraoperative events, to look at safety issues during a laparoscopic cholecystectomy, and to determine the disease severity because we know each operation is different based on the patient’s anatomy, the severity of the disease, and the difficulty of the procedure.²⁰

I want to acknowledge how forward-thinking the ABS is under the leadership of John Mellinger, MD, FACS, and Craig Kent, MD, FACS, as the current and immediate past-chairs of the board, respectively, and Jo Buyske, MD, FACS, as the executive director.  They recently convened a video-based assessment task force, chaired by Aurora Pryor, MD, FACS, at Stony Brook Medicine, NY, with the goal of examining opportunities for videobased assessment and how they can be integrated into CC. It is an exciting time, and I look forward to seeing what emerges.

Video-based coaching

I think it would be a disservice to the discipline to start measuring performance and telling surgeons how well or poorly they perform without at the same time offering opportunities to support each other’s improvement. You can approach improvement from a variety of perspectives, but here I want to focus on video-based coaching. It represents a productive and appealing approach to support professional development.

Surgical coaching is intended to unlock a person’s potential to evaluate his or her own performance. It is about helping them to learn rather than teach. It is about providing objective and constructive feedback to help someone recognize what works and what can be improved and inspire them to maximize their potential. It involves the establishment of goals, observation of a task, self-evaluation, as well as coach feedback that allows one to improve task performance and support change.

Our team conceptualizes surgical coaching in two different buckets. Most of our work is in peer coaching, where peers at a similar level of knowledge engage in an equal, noncompetitive relationship to improve performance. Expert coaching, on the other hand, usually serves as a supplement to other traditional educational paradigms. For example, video-based coaching for residents and fellows was shown to be more resident-centered and goal-directed while addressing more complex concepts than intraoperative teaching.²¹ Expert coaching also can support established surgeons as they learn a new technology, technique, or procedure.

Exciting developments in surgical coaching

The mission of the Academy for Surgical Coaching is to empower surgeons to improve clinical performance, well-being, and patient care. We do that by hosting live and, more recently, virtual coach training sessions. We provide coaching to individual surgeons and consultation to other organizations that are looking to develop coaching programs. We work to advocate for and educate people within the discipline and more broadly about the concept of surgical coaching. And we collaborate with health technology companies to support the development of the technological infrastructure that is needed to support video-based surgical coaching.

In general, whether it’s with the academy or with another organization, surgeons in these programs engage with a coach over three to 12 months to achieve their individualized performance improvement goals. Surgeons are recruited and trained, and, very importantly, we spend time trying to ensure that both the participants and the coaches understand what coaching is—that a coach doesn’t teach, but rather facilitates self-reflection.

Each surgeon is matched with a coach. At the first coaching session, they review the surgeon’s practice for the first time, which ideally would involve a baseline evaluation of surgical skill and establishment of goals. Each session starts with a review of the goals and ends with an action plan to bring about meaningful change in practice. Participants have ongoing coaching sessions with periodic check-ins, and video review and analysis where they assess progress toward achieving goals and then redefine those goals and make new action plans. Finally, depending on what their interests are, in the assessment of what they were targeting, sometimes they will do a final assessment to see if a goal is attained and to reflect on the coaching process.

SAGES has developed an innovative and potentially disruptive approach to expert coaching to support CPD during COVID-19. Many of you may be familiar with SAGES’ ADOPT programs, which allow the society to support development of a new skill in its members.²² They develop a core faculty, and all those faculty members are trained using the Laparoscopic Colectomy Train-the-Trainer (Lapco TT) Method, which was developed in the U.K. It teaches them to provide optimal constructive feedback during hands-on training.

Individual surgeons who want to learn a new skill in a hands-on course that emphasizes learner-centered teaching are enrolled. The learners develop their own goals for a course focused on a new technique or skill that can be implemented in the learner’s practice. Key to this program, like many programs that use expert coaching, is the idea of a longitudinal relationship in which each learner is paired with a mentor who serves as a resource and advisor for a year following the course, so the coach is available to support the learner in adopting the new skill into practice. Mentor activities include video review of procedures and coaching. SAGES also builds a community of practice through periodic webinars, mentor-mentee interactions, video review, and troubleshooting sessions over the course of the year.

In 2017 through 2019, SAGES offered an ADOPT program for hernia repair, which was based on classroom-based learning and hands-on simulations. In 2020, of course, it was not possible to participate in these types of live activities, so the directors of the hernia program partnered with Proximie, which is a tele-mentoring software company, and Kind Heart, which is a real tissue surgical simulation company that developed a portable porcine model. SAGES shipped a trainer box that came with the porcine model with the specimen, as well as a webcam and gooseneck webcam holder, to each participant with specific instructions on how to set up a station at home or at work. Then, using a Proximie interface, learners shared their videos with their coaches to get feedback.

This course has incredible potential to revolutionize how we approach this type of longitudinal coaching program to help with adopting a new technology or new procedure into practice.

Conclusion

The ABMS vision statement, which was published before COVID-19, has challenged us to be disruptive in our approach to CPD. Evidence linking surgeon performance to patient outcomes has magnified the need for evaluation of performance and practice over the last five to 10 years. COVID-19 has forced us to go virtual, offering opportunities to, as Mr. Emanuel said, “do things that were not possible before,” and we have many tools that can support video-based assessment and improvement.

New tools are being developed every day, and it is up to us to think about how we can use them in a meaningful way. I want to challenge each of you to take advantage of the opportunity created by the COVID-19 crisis as a disruptive moment in CPD. Explore what is currently available at your institution and encourage your institution to invest in video capture technology. Be sure your institution has policies that allow for video capture and sharing for education and quality improvement. Find ways to assess your performance, set goals, reflect on practice, and make intentional adjustments to your practice. This process can be as easy as asking one of your partners to scrub with you and setting some goals and seeing how you can get better and then doing the same for them.

If you still have funds available for CME, seek out opportunities that allow you to assess your technical and nontechnical performance and participate in activities that are individualized, goal-directed, and aligned with adult learning principles. Finally, encourage surgical societies and other professional organizations to apply innovative video-based technologies to support CPD.

Acknowledgments

This talk represents the ideas and work of many individuals with whom I collaborate. In particular, I want to acknowledge three individuals: Sudha Pavuluri Quamme, co-founder of the Academy for Surgical Coaching, who also leads the Wisconsin Surgical Coaching Program, our research team at the University of Wisconsin-Madison; and Adrienne Faerber and Jason Pradarelli, Executive Director and Medical Director of the Academy for Surgical Coaching, respectively.

References

1. Feygin D. The slow grind: Why the healthcare industry won’t change overnight. May 3, 2019. Available at: https://www.linkedin.com/pulse/slow-grind-why-healthcare-industry-wont-change-feygin-ph-d-mba. Accessed January 11, 2021.
2. Ghafur S. Why are health care organizations slow to adopt patient-facing digital technologies? March 6, 2019. Available at: www.healthaffairs.org/do/10.1377/hblog20190301.476734/full/. Accessed January 11, 2021.
3. Herzlinger RE. Why innovation in health care is so hard. May 2006. Available at: https://hbr.org/2006/05/why-innovation-in-health-care-is-so-hard. Accessed January 11, 2021.
4. “Rahm Emanuel on the Opportunities of Crisis.” YouTube, uploaded by Wall Street Journal. November 19, 2008. Available at: www.youtube.com/watch?v=_mzcbXi1Tkk. Accessed January 13, 2021.
5. PricewaterhouseCoopers. Top health industry issues of 2021: Will a shocked system emerge stronger? December 2019. Available at: www.pwc.com/us/en/industries/health-industries/top-health-industry-issues.html. Accessed January 11, 2021.
6. Keesara S, Jonas A, Schulman K. Covid-19 and health care’s digital revolution. June 4, 2020. Available at: www.nejm.org/doi/full/10.1056/NEJMp2005835. Accessed January 11, 2021.
7. U.S. Department of Health & Human Services. Telehealth: Delivering care safely during COVID-19. Available at: www.hhs.gov/coronavirus/telehealth/index.html?language=en. Accessed January 11, 2021.
8. Aisch G, Quealy K. Stephen Curry’s 3-point record in context: Off the charts. April 16, 2016. Available at: www.nytimes.com/interactive/2016/04/16/upshot/stephen-curry-golden-state-warriors-3-pointers.html. Accessed January 11, 2021.
9. Sachdeva AK. The new paradigm of continuing education. Arch Surg. 2005;140(3):264-269. Available at: https://jamanetwork.com/journals/jamasurgery/fullarticle/508457. Accessed January 11, 2021.
10. Teirstein PS. Boarded to death—why maintenance of certification is bad for doctors and patients. N Engl J Med. 2015;372(2):106-108. Available at: www.nejm.org/doi/full/10.1056/nejmp1407422. Accessed January 11, 2021.
11. Baron RJ. Professional self-regulation in a changing world: Old problems need new approaches. JAMA. 2015;313(18):1807-1808.
12. The Vision Initiative. Continuing Board Certification: Vision for the Future Commission. February 12, 2019. Available at: https://visioninitiative.org/commission/final-report/. Accessed January 11, 2021.
13. Birkmeyer JD, Finks JF, O’Reilly A, et al. Surgical skill and complication rates after bariatric surgery. N Engl J Med. 2013;369(15):1434-1442.
14. Stulberg JJ, Huang R, Kreutzer L, et al. Association between surgeon technical skills and patient outcomes. JAMA Surg. 2020;155(10):960-968.
15. Hogg ME, Zenati M, Novak S, et al. Grading of surgeon technical performance predicts postoperative pancreatic fistula for pancreaticoduodenectomy independent of patient-related variables. Ann Surg. 2016;264(3):482-491.
16. Kluger J. Learning from medical mistakes: Surgical Safety Technologies Operating Room Black Box. Time. Available at: https://time.com/collection/bestinventions-2019/5733095/surgical-safety-technologies-operating-room-black-box/. Accessed January 13, 2021.
17. Champagne BJ, Steele SR, Hendren SK. The American Society of Colon and Rectal Surgeons assessment tool for performance of laparoscopic colectomy. Dis Colon Rectum. 2017;60(7):738-744.
18. Ritter EM, Gardner AK, Dunkin BJ, Schultz L, Pryor AD, Feldman L. Video-based assessment for laparoscopic fundoplication: Initial development of a robust tool for operative performance assessment. Surg Endosc. 2020;34(7):3176-3183.
19. Chen C, White L, Kowalewski T, et al. Crowd-sourced assessment of technical skills: A novel method to evaluate surgical performance. J Surg Res. 2014;187(1):65-71.
20. Korndorffer J, Hawn M, Spain DA, at al. Situating artificial intelligence in surgery: A focus on disease severity. Ann Surg. 2020;272(3):523-528.
21. Hu YY, Mazer LM, Yule SJ. Complementing operating room teaching with video-based coaching. JAMA Surg. 2017;152(4):318-325.
22. Society of American Gastrointestinal and Endoscopic Surgeons. SAGES Adopt program. Available at: www.sages.org/sages-adopt/. Accessed January 13, 2021.