The Accreditation Council for Graduate Medical Education (ACGME) and American Board of Medical Specialties (ABMS) have a common goal: training highly skilled and knowledgeable specialists who can provide expert care to their patients. Along the way, ACGME accredits programs that assess individuals during residency and the ABMS Member Boards assess individuals for specialty certification as they make the transition from training into practice. It’s no wonder then that ACGME and some ABMS Member Boards have collaborated to develop and test rigorous, valid, and reliable assessments appropriate for each specialty as competency-based medical education (CBME) gains traction in the United States.
One of the most common assessment approaches to CBME is entrustable professional activities (EPAs), which are observable, routine activities that physicians should be able to perform safely and effectively to meet their patients’ needs. While a handful of medical and surgical specialties are early adopters of EPAs, others are just beginning to explore the potential impact that EPAs and other summative assessment approaches could have on undergraduate and graduate medical education as well as the transition to practice.
Similarly, some Member Boards are moving forward with using EPAs for certification decision-making in the next several years, while others are in the exploratory phase. Member Boards are committed to working with myriad stakeholders from the broader, specialty-specific educational community and specialty societies to make these determinations. Following are examples of how some of the boards are incorporating, piloting, or exploring assessment approaches as part of a CBME model.
Earlier this year, the American Board of Surgery (ABS) announced its transition to EPAs for general surgery resident evaluation, beginning in July 2023. The resident class completing training in 2028 will be the first to be required to provide attestation of entrustment decisions across 19 EPAs as part of their application for the ABS certifying process. ABS convened representatives from key stakeholder groups to develop five initial EPAs for general surgery that were piloted in 28 residency programs between 2018 and 2020. ABS is in the process of developing 14 additional EPAs to complete a full representative complement for general surgery. EPAs are also in the early stages of development for the other ABS specialties; vascular surgery, pediatric surgery, complex general surgical oncology, and surgical critical care. The vascular surgery EPAs are expected to launch in 2024, with the others to follow in the coming years. Additionally, ABS is working to provide a secure, interactive platform and data repository to facilitate the implementation of EPAs at the more than 350 U.S. general surgical residencies certified by ACGME.
The American Board of Pediatrics (ABP) plans to begin integrating EPA data into certification decision-making by 2028. Working closely with the Association of Pediatric Program Directors, other stakeholders, and the pediatrics community at large, in 2014, ABP completed development of an overall EPA framework that includes 17 EPAs for general pediatrics[PDF], 7 EPAs that are common to all pediatric subspecialties, and between three and six unique EPAs for each pediatrics subspecialty. Since that time, ABP has undertaken several investigations evaluating EPAs. In 2016, ABP pilot tested the general pediatrics EPAs at more than 20 residency training programs. Similarly, in 2018, ABP began a longitudinal study that is evaluating the pediatric subspecialty EPAs in more than one-third of pediatric subspecialty fellowship programs across all subspecialties. In 2021, a follow-up investigation in general pediatrics evaluating EPAs was initiated and it includes more than 40 pediatrics and medicine-pediatrics residency programs. While research is ongoing, ABP continues to collaborate with the broader pediatrics community to determine the optimal approach to data collection and how best to implement the EPAs in a meaningful way.
Both ABP and ABS are aligning their EPAs with the ACGME/ABMS core competencies and ACGME milestones as part of a unified approach to CBME and assessment. Several studies have validated both ABS’ and ABP’s EPAs as reliable tools for assessing levels of entrustment in residents.
Last year, the American Board of Orthopaedic Surgery (ABOS) launched the ABOS Knowledge, Skills, and Behavior (ABOS KSB) Program to support the shift to CBME in orthopaedic residency training. To date, nearly 50 residency programs have adopted, or are in the process of adopting, the program. The ABOS KSB Program was developed in collaboration with the American Academy of Orthopaedic Surgeons (AAOS), ACGME, American Orthopaedic Association (AOA), AOA Council of Orthopaedic Residency Directors, and orthopaedic residency programs across the country. The program assesses the following: resident knowledge through AAOS’ Orthopaedic In-Training Examination® taken each year; surgical skills via observation and real-time feedback from faculty using a surgical skills assessment tool; and professional behavior at the end of every rotation by each faculty member plus a 360o evaluation consisting of multiple professional behavior assessments completed once a year. ABOS validated the surgical skills and professional behavior assessment tools during a four-year pilot project. The board provides the ABOS KSB program and training to program directors and coordinators for free.
The American Board of Pathology (ABPath) began investigating CBME at its 2018 Summer Retreat. It is a member of the EPA Working Group, a collaboration between the Association of Pathology Chairs and Graduate Medical Education Committee of the College of American Pathologists. The EPA Working Group is completing a one-year pilot program evaluating the feasibility of using four EPAs in pathology training at seven residency programs. It is planning another trial to run at least two years to test the validity of the EPAs. Additionally, in 2020, ABPath’s Board of Trustees approved the support of a Promotion in Place pilot, which is testing the use of CBME across the specialties at Massachusetts General Hospital. ABPath is investigating additional options to further competency-based assessment in training and certification.
Currently, CBME is being pilot tested at two plastic surgery residency programs, with the first residents potentially graduating in 2024. In 2017, the American Board of Plastic Surgery (ABPS) hosted a Strategic Planning Retreat where its Board of Directors and various stakeholders agreed to support a pilot program for the CBME model in plastic surgery used by the Royal College of Physicians and Surgeons of Canada. Four U.S. residency programs created a consortium and developed a curriculum with assessments that was approved by ACGME’s Advancing Innovation in Residency Education Committee. Although it is still considered a pilot program, ABPS has incorporated the CBME model into its current acceptable training pathways. The board is tracking the competency-based residents within the program and will track their performance on future ABPS certification examinations and Continuous Certification Self-Assessments. Two of the original four residency programs withdrew from the pilot, which is in its fourth year, as they were not equipped to provide multiple training pathways. Those two programs transitioned their residents back into the standard Integrated plastic surgery training pathway.
In 2015, 20 EPAs for family medicine were developed through a collective effort involving seven national family medicine organizations as a part of an initiative entitled Family Medicine for America’s Health. Currently, the only organization formally using these EPAs is the Association of Family Medicine Residency Directors, which was the major contributor to EPA development along with the Society of Teachers of Family Medicine. The EPAs are not formally used as part of program director attestation for board eligibility for the American Board of Family Medicine (ABFM), which was involved in the initiative. However, the EPAs have been used, along with the milestones, as part the recent Residency Redesign effort that led to new ACGME training requirements for family medicine residencies that will be released later this year. ABFM uses a list of clinical activities, which function like EPAs, informing and guiding the certification exam blueprint. Currently, ABFM is in the process of developing a new exam blueprint, in which it will refer to the EPAs. Once the broader specialty develops assessments for each of the EPAs, ABFM will use them for its new exam blueprint, as well.
In 2021, the American Board of Emergency Medicine (ABEM) began exploring the use of EPAs as part of its Becoming Certified Initiative. If the Becoming Certified Task Force recommends the adoption of EPAs, then ABEM will begin collaborating with stakeholders to develop EPAs for emergency medicine.
As residency programs begin to incorporate EPAs and other CBME assessment approaches, ABMS Member Boards are working side-by-side with them to ensure the individuals graduating from residency are ready and eligible for the becoming certified process. The path is just beginning and will take years to complete as the specialties develop, pilot test, and implement these new systems. No matter what approach they determine is the best suited, one thing is certain: residency programs, through their Clinical Competency Committees, will continue to be responsible for helping to decide whether a resident is ready to start the becoming certified process. These assessments serve as potential tools that have been designed to assist these committees in making this all-important decision.
