As part of their commitment to promoting diversity, equity, and inclusion (DEI), the American Board of Medical Specialties (ABMS) Member Boards are striving to ensure that their certifying examinations are free of bias, their examiners recognize and mitigate potential biases, and their continuing certification programs support health equity. The following are a few examples of how different boards are meeting these goals.
Collecting robust diplomate data
Underlying their DEI efforts is the need to collect robust demographic data about board certified physicians and medical specialists (also known as diplomates). To date, most of the 24 Member Boards collect data regarding race and ethnicity and nearly all of them collect gender data. A growing number of boards use their diplomate portal to collect this information.
In addition to race, ethnicity, and gender, the American Board of Emergency Medicine (ABEM) collects clinical practice setting information on a voluntary basis on its diplomate portal. ABEM also reviews data from the state in which a diplomate practices and the number of years diplomates are certified. The demographic data are used to improve the specialty’s understanding of the diversity of practicing emergency medicine (EM) physicians, advance research about the practice experience of EM physicians, identify gaps in EM physician diversity representation, and detect potential impacts to board exams.
The American Board of Family Medicine (ABFM), which has been collecting race and ethnicity data for a decade, recently added multiple categories and began allowing multiple responses across all race and ethnicity options as well as disaggregated Asian ethnicities to gain a better understanding of racial and ethnic groups in family medicine. Additionally, the gender category now includes non-binary and choose not to disclose options. In 2023, ABFM integrated demographic data collection into its physician portfolio. The American Board of Internal Medicine (ABIM) has also improved the gender, race, and ethnicity functionality on its physician portal to enable diplomates to accurately self-identify.
Removing bias on exams
Member Boards are exploring how best to use the demographic data they are collecting to conduct DEI analyses of their certification exams. For starters, they are identifying and removing inadvertent biases on oral and written exams, incorporating health equity content, and conducting equity research on exam performance.
The American Board of Pediatrics (ABP) conducts an annual differential item functioning (DIF) analysis* of all questions on the initial General Pediatrics Certifying Exam to identify any subgroup differences and review questions flagged by the Bias and Sensitivity Review (BSR) Panel. In 2021, no items on the exam were flagged for DIF by gender and 2.8 percent of the items were flagged for DIF by race and ethnicity. Of the latter items, the BSR considered 14 percent of them to contain biased language that may have undermined what the item was intending to measure and recommended removing them. By repeating the DIF/BSR process annually, ABP hopes to increase the board’s understanding of how language nuances and other characteristics impact item performance to improve guidelines for developing future items. ABP also conducts analyses of differential exam pass rates by physician characteristics, including race and ethnicity. Additionally, the board investigates and mitigates possible bias in the use of entrustable professional activities to assess residents and fellows through two research projects funded by the ABP Foundation.
Many other Member Boards use DIF analysis or BSR-type panels, as well. In 2022, ABFM published results of a DIF analysis of exams administered between 2013 and 2020 that revealed a small percentage of biased items that the DIF Panel recommended to remove or rewrite (approximately 0.1 percent). The same year, ABFM added questions to the certification survey to assess disparities in other at-risk populations, including educational background (e.g., first-generation college students), rural origin, and state/country of origin. In 2024, the board included new race and ethnicity categories (Asian subgroups and Middle Eastern or North African) and socioeconomic status-related questions in its certifying exam and Family Medicine Certification Longitudinal Assessment.
For exams with sufficiently large subgroups, the American Board of Surgery’s (ABS) Psychometrics Department uses DIF analysis and a bias panel review to analyze performance across demographic subgroups to evaluate performance differences on the exams overall. Regarding exam content, ABS is working to eliminate potential bias in exam writing and use more inclusive language. Additionally, as exam blueprints are revised, DEI-related topics are being incorporated into the test specifications for future exams.
Like ABS, some boards are homing in on the language used in exams. ABEM created a two-page DEI Language Guide to help item writers navigate the landscape of equity-focused language during item development. This reference document uses best practices in DEI nomenclature and item writing from the American Medical Association’s Center for Equity and the National Board of Medical Examiners. The importance of language in item development was emphasized during a DEI Workshop conducted at ABEM’s 2024 Item Writers Workshop held this July. The American Board of Physical Medicine and Rehabilitation (ABPMR) added a section on inclusive and mindful language in its style guide. ABPMR is enhancing its in-house item-banking software and plans to include metadata that will help ensure its exam accurately represents diversity of the patient population. In addition to using DIF analysis, the American Board of Pathology (ABPath) recently implemented a process to eliminate race-based language in certification exam questions, while retaining identifiers of genetic determinants of disease.
After auditing scenarios in its Objective Structured Clinical Exam, the American Board of Anesthesiology (ABA) incorporated culture, social determinants, and lived experiences to the patient/setting details of the scenarios. Addressing visual content, the American Board of Dermatology (ABD) called on its item writers to submit photographs depicting skin of color and other diverse patient groups for its exams. By 2022, ABD met its goal to reflect United States Census data and depict skin of color in at least 20 percent of images in all ABD exams and new items. ABD is also planning to track overall performance of diversity items on exams to determine knowledge gaps and additional study opportunities.
ABIM efforts include performing fairness reviews, including gender, race, ethnicity, and medical school type across ABIM assessments. ABIM piloted this work in four disciplines this year (internal medicine, gastroenterology, nephrology, and cardiovascular disease) using statistical analysis and content review to identify any biased items. A diverse panel of diplomates with appropriate content expertise reviewed flagged questions. Current work is focused on repeating the analysis in the same four disciplines for a second cycle, and ultimately operationalizing it so all specialties can be reviewed annually.
Providing implicit bias training
Most Member Boards provide implicit bias training for item writers and examiners. Many have built this training into item writing sessions or examiner orientation. Some boards are considering having diverse subject matter experts review exam material prior to publication.
In addition to using DIF analysis for its exams, the American Board of Orthopaedic Surgery (ABOS)
created an unconscious bias training video that oral examiners are required to view annually prior to administering exams. ABOS also reviewed its oral exam processes to determine if there is any implicit bias within the methods used by the examiners. The American Board of Plastic Surgery (ABPS) coordinates annual bias sensitivity training for examiners prior to the oral exams and every three years for its item writers and subject matter experts. Additionally, ABPS provides related materials each year with item-writing assignments.
Board examiners for the American Board of Ophthalmology (ABO) are required to participate in a two-part implicit bias training program prior to each exam they administer. ABO item writers are trained in best practices for using inclusive language including, but not limited to, gender identity, disability status, and correct use of race and ethnicity. The American Board of Radiology’s DEI Committee is evaluating a pilot program comparing pre-recorded versus live DEI training for oral examiners.
Including health equity content
Many Member Boards are incorporating DEI content in their continuing certification programs. While some boards routinely identify disparities in health care processes and outcomes commonly arising in their respective specialties, others review national datasets of health care disparities and/or peer-reviewed medical literature or other publications to gather this information. A few boards partner with their specialty societies to identify key conditions and aspects of care within the specialty. They then use the information to develop items for assessment.
More than half of the boards occasionally include questions related to health care disparities in their continuing certification assessments. Many are working to develop more questions in their item banks.
In 2021, ABA revised its MOCA Content Outline to include sections on health disparities and DEI. The following year, the board added 56 assessment questions based on DIF analysis. In 2023, ABA launched DEI, quality improvement (QI), and health disparity questions into MOCA Minute and exams. The board is working to identify and close DEI knowledge gaps identified by its assessments.
In 2023, ABD established six CertLink Content Advisory Groups, one of which focuses on Skin of Color and Gender Identity Disparities. The group has developed 16 article-based questions focused on both topics that will be added to CertLink in 2025. ABD also provides diplomates with access to all articles on which CertLink questions are based. Beginning in 2025, articles on skin of color and other diversity topics will be given a special icon to make them easier to find in the CertLink library.
ABPS’ DEI Standing Committee reviews items from the written exam, oral examination, and the continuous certification self-assessments to identify and correct items that may contain unintentional bias. ABPS has incorporated DEI content into the exam blueprint and is developing DEI-related questions for the continuous certification item bank.
ABPMR routinely rolls out new longitudinal assessments that include article-based items, including those that address issues of health equity and improving health and health care.
ABIM has developed health equity content for ABIM assessments in four pilot disciplines (internal medicine, gastroenterology, nephrology and cardiovascular disease). ABIM has also completed a study to compare assessments by program directors’ scores of Black and Latino trainees to other trainees.
Developing QI activities to reduce health disparities
Nearly half of the Member Boards either currently promote or are working toward promoting QI activities to reduce health care disparities. Almost all accept these activities. A growing number of boards currently require lifelong learning/continuing professional development activities related to reducing health care disparities. Some boards plan to develop a template for one or more social determinants of health and encourage specialty societies or educational partners to develop programs.
ABFM has modified 13 performance improvement (PI) activities to incorporate health equity. Family physicians are provided the option to stratify measures they use for their PI efforts by race, ethnicity, socioeconomic status, sexual orientation/gender identity, disability, rural practice, or underserved group. A Self-Directed Clinical PI activity also allows family physicians to tailor their approach to reducing health disparities in a way that is most relevant to their specific practice or community needs. Developed in partnership with the American Academy of Family Physicians (AAFP), ABFM’s self-assessment activity entitled Health Equity: Leading the Change contains five modules aimed at helping physicians improve their knowledge of health equity and social determinants of health, supporting vulnerable populations, championing economics and policy to reduce health disparities, and understanding the impact of climate change on population health. The activity is currently unavailable as it is being reviewed for updates through the AAFP member education platform.
ABEM has expanded its Improvement in Medical Practice (IMP) requirement to include a section on Advancing Health Equity. It contains seven PI activities that can be leveraged to help address health inequities, such as increasing access to linguistically and culturally appropriate care and securing pregnancy-related care for Black and/or American Indian/Alaskan Native women.
The American Board of Obstetrics and Gynecology (ABOG) requires diplomates to complete at least one article in the category now referred to as Health Equity and Patient Safety. The board continuously releases DEI-related articles across all categories and has started tagging them based on relevance to racial disparities or health inequities, enabling diplomates to easily sort and find them. ABOG is working to develop a more robust assortment of IMP activities in this category.
ABO diplomates have the option to earn IMP activity credit for completing Stanford University’s free, online Unconscious Bias in Medicine continuing medical education course. Peer-reviewed articles about health equity remain a popular category on ABO’s Quarterly Questions menu.
ABP plans to release a template for a QI activity that supports pediatricians in stratifying their clinical data on race, ethnicity, and spoken language to identify and address possible pediatric health care inequities. The board continues to offer continuing certification credit for DEIB (the B is for Belonging) training and includes articles that discuss health inequities in continuing certification activities.
ABMS Member Boards continue to strengthen their efforts to promote DEI in their exams and incorporate health equity in their continuing certification programs to better serve board certified physicians and the diverse population of patients they serve.
*DIF analysis is a statistical method that identifies potentially biased items in assessments.
© 2024 American Board of Medical Specialties
Read the latest issue of ABMS Insights and sign up to receive future updates about board certification.
-
Read More:
- ABMS Insights |
- Member Boards |
