Daniel Kendrick, MD, MAEd, Michigan Medicine Department of Surgery Center for Surgical Training and Research Research Fellow

With the growing concern regarding the readiness of graduates in general surgery to enter into independent practice, it is imperative that we accurately assess the progression of resident competence throughout the training process. At the same time, we must do so in a way that minimizes measurement burden on both faculty and trainees.

Surprisingly, it is currently unknown how surgical training programs determine competence and how much individual programs have in common. There are some measures of resident performance that are consistent across all programs including the ABSITE exam, ACGME case logging, and Milestone ratings, but programs use many other assessments that have been developed locally.  This presents a problem in both understanding the results of these tools as well as validating and improving them.

Ideally, all training programs would have a standardized set of assessment tools that, together, accurately predict the eventual clinical performance of a trainee and allow for targeted intervention prior to graduation. In order to realize this goal, we must do several things.  First, we must define what performance domains are important to assess during general surgery residents, (i.e. what defines the performance of a competent practicing general surgeon). Next, we must understand what different training programs are doing in order to design a toolkit that is optimal for their diverse needs/resources. Last, we must validate these tools’ ability to subsequently predict early career clinical performance of graduating residents. 

Much effort has gone into determining important performance domains in general surgery through the development of the ACGME General Surgery Milestones.  Through their application, the Milestones guide training programs to develop methods to measure and report trainee competence in each of sixteen important areas of performance.  It follows that many programs, working in parallel, have arrived at distinct local evaluation processes, and the next step is to understand what these different methods are.  Before an effective standardized toolkit can be built, we must assess what tools/strategies are currently being used by training programs to measure resident performance and how they fit within each of these domains.

In order to address this, we plan to conduct a survey-based assessment inventory of the evaluation process at all sites participating in the Variability In Trainee Autonomy and Learning in Surgery (VITALS) trial. We will synthesize these into a comprehensive picture of how training programs in general surgery are currently measuring trainee performance. This will identify gaps in our current assessment process with the eventual goal to build and validate a standardized assessment toolkit to be shared across all training programs. 


Please email Dr. Kendrick for more information at This email address is being protected from spambots. You need JavaScript enabled to view it.

The PLSC is excited to announce a series of Town Halls happening during Surgical Education Week 2019 and the following week.  We would love to have you join us at one of the follow three events where we will discuss:

  1. Where the collaborative has been 
  2. Where we are heading, including the upcoming release of automatic case-logging in SIMPL
  3. Opportunities for leadership within our growing organization 

The Town Halls will be: 

  1. In-person during Surgical Education Week at the Fairmont Hotel starting at 5pm local time.
  2. Webinar on Tuesday, April 30th at 4pm Eastern at https://bluejeans.com/467850272
  3. Webinar on Thursday, May 2nd at 1pm Eastern at https://bluejeans.com/467850272

To RSVP, please go to the following link: https://forms.gle/5v846aYL4gdb8QCB6 

Group Photo small copy

On January 22nd, 2018, the members of the Procedural Learning and Safety Collaborative co-hosted a symposium with the American Board of Surgery, the Accreditation Council on Graduate Medical Education, the Association of Program Directors in Surgery, and the Resident Review Committee for Surgery titled: “Performance, Assessment, and Patient Safety:  Defining the Roadmap for Quality Improvement in Surgical Education.” Participants included representatives the hosting organizations, general surgery residency program directors, faculty, residents, surgery department chairs and psychometricians working in the field of surgical education.

The goal of the symposium was to develop and prioritize surgical education research topics that could be pursued by the Procedural Learning and Safety Collaborative members.

Symposium attendees spent the first half of the day discussing and ranking possible research ideas, including ideas originally proposed by Stefanidis et al in 2015.  The six top-ranked research questions were developed into complete research proposals by small break-out groups during the second half of the day.  Each draft proposal was then pitched in a “shark tank” format to the entire group.

The top six research questions were, in order:

  1. What faculty development tools can be developed to enhance appropriate resident supervision?
  2. How do surgical residents impact the safety, quality, efficiency, and cost of surgical services within their hospitals?
  3. What is a “competent” surgeon?
  4. Which are the best methods to assess resident performance and competence (intraoperative and clinical, procedural and cognitive)?
  5. What are the most effective methods to improve faculty teaching ability and promote interest in teaching?
  6. What are the performance criteria a resident has to meet to be considered competent and before independent practice is allowed?


As the next step, the Procedural Learning and Safety Collaborative will convene working groups to further explore each of these important questions—and turn our time in Chicago into high impact improvements in the quality of surgical education.

Fryer BrianMalangoniPottsShari Dan HeatherSchenartsMatthewsPano

williams 168x200pxIt is with great sadness that I must report the recent and unexpected passing of Reed Williams, PhD on June 20th, 2018.

Reed, a board member of PLSC, was one of the most respected and prolific educational researchers in North America. His contributions to performance evaluation in medical student and house-staff training have had a great impact on current practice, including everything we have done with SIMPL. He was also a kind, honest, humble, and thoughtful man who served as a friend, colleague, and mentor to many, including many of us. He is survived by his wife Sue and two sons. 

Reed spent many years working with surgical educators, first at Southern Illinois University and later at Indiana University. In both places he had an especially close collaboration with Dr. Gary Dunnington. More recently, he became an active member of the Procedural Learning and Safety Collaborative where he was central to the consortium's work. His contributions to this and many other organizations and projects stand as a testament to his intellectual generosity, keen insight, and rigorous approach. 

Above all, however, Reed had an unwavering commitment to the truth. This characteristic is well illustrated by the first interaction I had with Reed, long before we became collaborators. He was skeptical that our approach to using a single-item assessment (Zwisch) was sufficient, and matter-of-factly said so. But he wasn’t satisfied to simply disagree. Instead, he went back to re-analyze data he had previously collected in the process of validating a multi-item scale (OPRS) to see if his opinion held up to closer scrutiny. A month later he shared his results: a single item scale was, in fact, sufficient. I will always admire his willingness, even when it contradicted his prior experience, to believe in the scientific process. As a researcher, that is one of the highest compliments that I can pay this extraordinary man.

More generally, Reed wanted to use science to make the world a better place. He achieved his goal, even if there was so much more that he wanted to do. Thank you, Reed, for all that you gave. We will miss you terribly.

Brian George, on behalf of everyone at the PLSC

We will be organizing a memorial for Reed at SEW in April, which we will formally announce as the date approaches. We are also going to fund an award in his name, most likely through the Association for Surgical Education. If you would like to help organize his memorial or contribute to an award, please email us at This email address is being protected from spambots. You need JavaScript enabled to view it. for more information.

On 4/21/2017 Dr Brian George, representing PLSC, presented some of the major findings of the SIMPL study at the 137th annual meeting of the American Surgical Association. His presentation was met with significant interest from members of this organization that represents the academic leaders of surgery in the United States.

Annals of Surgery Visual Abstract

Published 9/22/2017

George Autonomy VA V2


In January 2015, the American Board of Surgery (ABS) agreed to fund a multi-institutional research study to test the SIMPL application as a real-time assessment tool for intraoperative performance of General Surgery residents. PLSC was able to recruit a total of 14 programs across the United States to participate, and the data collection phase of the trial ended in December 2016.

On Thursday, 4/20/2017 Dr Brian George, representing PLSC, shared the main findings of the trial with the ABS Executive committee. The results are thought provoking and suggest that even though our current way of educating general surgeons leads to overall acceptable results, it seems time to rethink some of the fundamentals how this education is currently structured. Members of the ABS were impressed by the large amount of data that were collected and by the sophistication of the data analysis.

This year's surgical education week was held in San Diego, CA from 4/18 - 4/22/2017. The SIMPL app was mentioned in multiple presentations, including the presidential address by Dr Dent. With the increasing interest in proficiency based education and entrustable professional activities real-time assessment of performances will become even more important. Software solutions like SIMPL could play a major role in making this process easy and convenient.

Surgeons in teaching hospitals are challenged with balancing their responsibility to patients for ensuring optimal operative outcomes, and their responsibility to prepare residents for independent practice.  Overseeing quality of care and supervising residents is further complicated by shortened resident duty hours, emphasis on operating room cost efficiency, as well as surgeon clinical productivity. These forces reduce time critical for teaching and honing residents’ operative skills and judgment.1  As noted in a  monograph published by the Josiah Macy Foundation, the development of essential skills  is delayed if learners are not afforded sufficient independence or authority for patient care.  The monograph reiterates the concerns of program directors and faculty that residents do not receive sufficient opportunity to act independently in the current teaching environment and are less prepared for practice.  These concerns were most strenuously expressed with procedure-based specialties.2  

We are excited to announce the formation of Procedural Learning and Safety Collaborative (PLSC)!  This is the first non-profit exclusively dedicated to improving the evaluation and training of residents in the procedural specialities.

Initially composed of surgical educators from Northwestern University, Indiana University, and Massachusetts General Hospital, we welcome new research collaborators and/or those who want to become more involved in the organization itself.  Our primary focus right now is to finish the development and then the launch of our smart-phone based evaluation system called SIMPL:  System for Improving and Measuring Procedural Learning.  We anticipate beginning beta testing at one of the founding institutions in the spring of 2015, with national launch to immediately follow.  At the same time we are actively fundraising and building out or organizational capacity.  On the research front we have several projects in the design phase, including a multi-institutional trial that we are planning to do in collaboration with the American Board of Surgery.