Certification And Maintenance

Otolaryngol Clin N Am 40 (2007) 1347–1357

Certification and Maintenance of Certification in Otolaryngology–Head and Neck Surgery Robert H. Miller, MD, MBAa,b a

American Board of Otolaryngology, 5615 Kirby Drive, Suite 600, Houston, TX 77005-2444, USA b Department of Otolaryngology–Head and Neck Surgery, Baylor College of Medicine, Houston, TX, USA

The mission The mission of the American Board of Otolaryngology (ABOto) is to assure that, at the time of certification and recertification, diplomates certified by the ABOto have met the ABOto’s professional standards of training and knowledge in otolaryngology–head and neck surgery.

History Shortly after the turn of the twentieth century, the American Academy of Ophthalmology and Otolaryngology established two committees to explore the concept of certification in these two specialties. The result of the deliberations was the development of the American Board of Ophthalmology in 1916 and the American Board of Otolaryngology (ABOto) in 1924. Initially, 465 otolaryngologists were invited to receive certification, and 354 were certified [1]. Through 2007, 16,989 otolaryngologists have been certified by the ABOto. In addition to certifying individuals, the ABOto set the standards for and accredited otolaryngology residency programs until 1953, at which time the Accreditation Council for Graduate Medical Education assumed this function. The certification process has evolved over time. Initially, the certification examination consisted of a written histopathology examination, a practical examination with real patients, and an oral knowledge examination, which was waived for ‘‘experienced’’ practitioners. The first written examination E-mail address: [email protected] 0030-6665/07/$ - see front matter Ó 2007 Elsevier Inc. All rights reserved. doi:10.1016/j.otc.2007.07.010

oto.theclinics.com

1348

MILLER

required essay-type answers but is currently in a multiple choice question format. The practical examination became the oral examination, which in its early format permitted a fair amount of individual examiner discretion in its administration. Currently, the oral examination consists of highly structured protocols that measures an examinee’s knowledge more effectively and consistently. The written examination is considered a qualifying examination because a candidate must pass it before having his or her score on the oral certifying examination considered. If a candidate passes the written examination but fails the oral examination, he or she has 3 years to pass the oral examination and become certified before having to retake the written examination. Both examinations are administered annually in Chicago. Examination development and process The examination development process is detailed, costly, and time consuming. The items (questions) for the two multiple choice question examinationsdthe written and otolaryngology training examinations (in-service examination)dare generated by the Task Force for New Material, which consists of 36 item writers. Item writers are selected from a pool of individuals nominated by all of the otolaryngology specialty societies and by the ABOto directors and senior examiners; they serve 3-year terms. Item writers participate in the annual item writers’ workshop, during which they learn the nuances of writing effective multiple choice questions. After the item is generated, it goes through a thorough and extensive editing and vetting process, which ensures that the medical information is accurate and refines the question wording and syntax. Finally, all new items are ‘‘field tested’’ on either the otolaryngology training examinations or written examinations. The statistical performance of each new item is carefully reviewed by the ABOto directors and a psychometrician. Items are evaluated as to whether they are too easy or too hard and how well an individual item’s performance correlates with how well the people who selected the correct answer did on the test as a whole. Items that do not meet the ABOto’s standards are either revised or discarded. The items that are used for scoring on the otolaryngology training examinations and written examinations are selected from a pool of successfully field-tested items based on the examination blueprint, which is available on the ABOto Web site (www.aboto.org). The directors select the best items from the pool based on several statistical and other parameters. The protocols used in the oral and neurotology examinations are written by the oral examiners, which include the directors, senior examiners, and guest examiners. Each protocol is carefully reviewed and edited several times by the appropriate specialty-specific group before its use. The results of all of the exams are analyzed by a psychometrician, who identifies any items that have an unusual statistical performance. These

CERTIFICATION AND MOC

1349

items are reviewed by the examination committee, and if the question is not thought to be valid, it is discarded from the examination and not used in scoring. The written and otolaryngology training examination scores are adjusted for annual variations in the ‘‘hardness or easiness’’ of the examination so that scores are consistent over time. Similarly, the oral examination scores are adjusted for variations in examiner leniencies and variations in the test severity for the 2-day examination period and year-to-year variations. Before 1999, the passing score was set by failing a fixed percentage of examinees. In 1999, the ABOto switched to a criterion referenced standard, in which the directors participated in a psychometric standard setting exercise that determined a fixed pass/fail point. Theoretically, it is possible to have a 100% pass rate. This standard-setting exercise is repeated approximately every 5 years to ensure its validity. Health care quality milestones A few historical events have had a major influence on health care quality in the United States. Perhaps one of the most important was the 1910 Flexner report on medical school education [2]. At the time there were many proprietary schools, many of which did a poor job of educating medical students. Commissioned by the Carnegie Foundation, the report summarized US medical schools and their curricula and provided recommendations on how to improve the quality of physician education. The establishment of medical specialty boards in 1916 was also a milestone because it not only provided a mechanism for evaluating physicians but also ultimately provided a mechanism for residency training accreditation. The public values board certification as a measure of quality [3]. Although quality of care has been an important subject for many years, the Institute of Medicine reports on quality of care in the United States have advanced this issue rapidly in many sectors, including the government, health care providers, insurers, purchasers of insurance, and public interest groups [4,5]. As a consequence, many groups are exploring methods to improve all levels of the health care system. As an example, two organizations that have instituted quality improvement programs are the Joint Commission on Accreditation of Healthcare Organizations for hospitals and the National Committee for Quality Assurance for health plans. Similarly, the two groups that have the main responsibility for overseeing the quality of individual physicians are the state medical licensing boards and their umbrella organization, the Federation of State Medical Boards and the American Board of Medical Specialties (ABMS), which is the umbrella organization for the 24 medical specialty boards. The ABMS and the Federation of State Medical Boards are active in the invigorated national health care quality improvement movement.

1350

MILLER

Board certification continuum Many people view the board certification process as simply passing the board examinations at the end of residency. The ABOto, however, views the certification process as a continuum that begins when residents are selected for training. The selection process is an important screen to identify the brightest, most capable medical students for otolaryngology–head and neck surgery training. Within the first few months of training, the new residents must register with the ABOto, which obtains and verifies information about the residents’ previous training. The ABOto views the program director as a critical component of the certification process because the program director, along with the faculty, has the most exposure to the residents’ knowledge, skills, and behavior over the 5-year residency period. Based on the Accreditation Council for Graduate Medical Education’s six competencies (Box 1), the board examinations have strengths in the measurement of medical knowledge and patient care (with the exception of surgical skills), but the program director is in a much better position to evaluate the other four competencies (Table 1). At the end of training, the chairperson and program director must attest that the candidate is of ‘‘high moral character and worthy of examination by the ABOto’’ and is acceptable for the examination process. Before 2002, all ABOto diplomates received timeless certificates, which means that the individual was certified forever unless the diplomate committed a crime or some adverse action were taken by the state licensing boards. Unless the diplomate committed an act that would be deemed egregious enough to result in decertification, the diplomate would continue to be certified although he or she had not been evaluated by the ABOto since the date of initial certification. Unless there was some local review by, for example, the diplomate’s hospital, a diplomate’s practice was not evaluated for remaining current on the latest medical developments or for quality of care. As the interest and pressure to improve health care quality increased, it became clear that physicians needed more scrutiny to ensure they were practicing high-quality medicine. As a consequence, the ABMS and its member boards determined that a new approach was needed to ensure that certified

Box 1. The Six Competencies 1. 2. 3. 4. 5. 6.

Medical knowledge Patient care Interpersonal and communication skills Professionalism Systems-based practice Practice-based learning and improvement

1351

CERTIFICATION AND MOC

Table 1 Ability to evaluate competencies in residents Competency

ABOto examinations

Program director

Medical knowledge Patient care Interpersonal and communication skills Professionalism System-based care Practice-based learning and improvement

þþþ þþþa þþ þ þ þ

þþþ þþþ þþþ þþþ þþþ þþ

a

Excludes surgical technique.

physicians were maintaining the ability to practice high-quality medicine. No longer could it be acceptable to provide lifetime certification without periodic assessment. After careful study, the ABMS developed the Maintenance of Certification program (MOC) to address the needs for health care quality improvement. All otolaryngology diplomates (both primary and subspecialty) certified in 2002 and thereafter receive 10-year time-limited certificates. To maintain certification and have the certificate renewed, all diplomates with time-limited certificates are required to participate in MOC.

Maintenance of certification MOC is a quality improvement program designed to improve physicians’ practice of medicine. Although the various boards are taking somewhat different approaches to its implementation, all MOC programs consist of four parts: 1. 2. 3. 4.

Professional standing Continuing education and self-assessment Cognitive expertise Performance in practice

Professional standing The ABOto requires that all MOC participants possess a valid ABOto certificate. The participant also must have unrestricted licenses to practice medicine in all states in which he or she practices. The diplomate also must have privileges at a hospital or ambulatory surgery center. If the participant does not have privileges, he or she must attest that the privileges were not lost because of an adverse action by the facility. In the past, it was difficult for the ABOto to routinely obtain information about adverse actions taken against its diplomates. Occasionally, the American Medical Association or some other source would notify the board if one of its diplomates had been involved in unprofessional activity. In

1352

MILLER

2004, however, the Federation of State Medical Licensing Boards established the Disciplinary Alert Notification Service (DANS), which has facilitated transfer of important information to each of the ABMS specialty boards. Through DANS, the ABOto receives regular reports on any adverse action taken by hospitals, state licensing boards, and others against ABOto diplomates. It is important for the ABOto to obtain this information, which covers a wide variety of actionsdfrom restrictions on privileges at hospitals to loss of licensure. Most of these offenses are minor in nature and do not reflect professional misconduct or evidence of poor practice. Although severe adverse actions against board-certified otolaryngologists reported through the disciplinary alert notification service are uncommon, the ABOto has taken action against these diplomates, including the most severe penalty: decertification. The ABOto’s Credentials/Ethics Committee reviews the details of the sentinel event and obtains letters of explanation from the diplomate. Only after careful analysis does the Credentials/Ethics Committee make a recommendation to the full ABOto board of directors, which is the final arbiter for these actions. It is important for the ABOto to enforce these standards because it maintains the importance and value of certification by the ABOto. Continuing education and self-assessment Critical to practicing high-quality otolaryngology–head and neck surgery is staying up-to-date on the latest developments in the specialty. One way of achieving this goal is to participate in continuing medical education (CME) programs. To encourage this activity, the ABOto requires that all MOC participants earn category 1 CME credits as a component of Part II of MOC. The current requirement is that each participant earn as many CME hours as are required to maintain his or her state medical license. These requirements vary from state to state, and a few states do not have CME requirements as part of licensure. For individuals in states that do not have a CME requirement, the ABOto requires a minimum of 15 hours of category 1 CME credits. Sixty percent of CME credits for all MOC participants must be related to the specialty of otolaryngology–head and neck surgery. Diplomates who are subcertified in neurotology are required to have 60% of their CME credits in either neurotology or otology. Self-assessment of one’s performance as an otolaryngologist–head and neck surgeon is a critical component of a quality improvement program. There are various approaches to self-assessment, and the ABOto is exploring several options. The first approach is through a patient simulation, in which the participant manages a patient with a given medical condition. These modules will be Internet-based and will simulate real patient encounters in which the participant must manage the patient. Various high-quality visuals, including imaging studies, histopathology, and patient photographs, enhance the patient management protocol, in which the participant can

CERTIFICATION AND MOC

1353

choose multiple options regarding evaluation and management. Rather than a linear progression through the program, decisions made by the participant result in different paths, some of which are not viable. After completion of the module, the participant is given feedback on which of the decisions were wise and which were not. References are provided to encourage the participant to study in areas of weaknesses, although the participant may choose products from other sources to supplement the module. The module is not pass/fail but is intended to assist the participant in identifying areas of strengths and weaknesses. The participant may be asked to repeat the module after he or she has had an opportunity to review the subject. In addition to specialty-specific modules, the ABOto plans to incorporate in MOC more generic modules from other sources that cover such topics as patient safety, communication, and other subjects. Another potential option is the use of surgical simulations as they are developed. It is anticipated that the MOC participant will complete modules several times during the 10-year MOC cycle as more modules become available. These modules will be developed in partnership with the specialty societies, academic departments of otolaryngology–head and neck surgery, and perhaps individuals who have a particular interest and expertise in this type of material. Other options for self-assessment, including participation in interactive sessions at meetings, are also being explored. Cognitive expertise At the end of the 10-year MOC cycle, each participant will be required to take and pass a multiple choice question examination. The computer-based examination will be administered in testing centers throughout the country so that most participants will be able to complete Part III in their own community. Otolaryngology–head and neck surgery is a diverse specialty, and many otolaryngologists tend to focus on subspecialty areas, although there is no formal recognition of many of these subspecialties. For example, it would be difficult for someone who practices head and neck surgery to remain current on all aspects of otology. An otologist might find it difficult to answer questions regarding facial plastic surgery. To address this situation, the ABOto has determined that the Part III examination consist of two modules, both of which must be passed to renew a diplomate’s certificate. The first test component is termed the fundamentals module, and it consists of questions on material that all otolaryngologists should know. Some of the topics that could be included in the core module are ethics, fluid and electrolytes, antibiotics, anesthesia, and patient safety, among others. The second test component is a specialty-specific module in various areas within otolaryngology. The participant selects a test module based on the focus of his or her practice. The areas that are planned to be available are listed in Box 2. The neurotology and sleep medicine modules will be available to individuals

1354

MILLER

Box 2. Part III subspecialty areas General otolaryngology Head and neck surgery Otology Allergy Pediatrics/bronchoesophagology Laryngology Rhinology Facial plastic surgery Neurotologya Sleep medicinea a

For individuals subcertified in these areas

who have been subcertified in these areas. Holders of these subspecialty certificates will take the examination in their respective area, which will renew not only their subspecialty certificate but also their primary certificate. The Part III examination will be available to MOC participants 3 years before the expiration date of the certificate so that the individual has three opportunities to pass the examination before the certificate expires. The first examination is scheduled to be available in 2010 for diplomates whose certificates expire in 2012. Upon passing the examination and completing all four MOC components, the successful participant receives a document that indicates that his or her primary certificate has been renewed for another 10-year period, during which the MOC cycle is repeated. Other than the neurotology and sleep medicine modules, passing the Part III examination does not imply any particular expertise in the subspecialty area. Rather, the primary certificate is renewed. Performance in practice Perhaps the most important component of MOC and the most difficult to implement is Part IV: performance in practice. The intent of this component is to measure a participant’s quality of practice. Part IV includes the outcomes of the knowledge, ability, and skills as they apply to the management of patients. The plan is to develop measures of these outcomes that are accurate, reliable, and implemented with a minimum of intrusion. All of the ABMS boards are working diligently to develop these measures. Although a challenging project, the measures that will be developed for MOC may be used in other programs that are being implemented by other organizations to measure the quality of health care. For example, Part IV quality measures could be used in pay-for-performance programs and vice versa.

CERTIFICATION AND MOC

1355

Other components of Part IV include Consumer Assessment of Healthcare Providers and Systems (CAHPS), which many health systems and large health care organizations already use. The program is a means for patients to provide feedback about their health care. The CAHPS program consists of a standardized, validated questionnaire given to patients at the time of a visit. Patients complete the questionnaire at their convenience using a touch-tone phone or the Internet. Groups that have used this system have found it helpful in improving patient satisfaction, which includes communication and quality of care. As part of Part IV of MOC, CAHPS will be available to smaller practices that otherwise may not be able to take advantage of the program. Who is required to participate in Maintenance of Certification? In 2002, the ABOto began issuing only 10-year time-limited certificates with the requirement that the individual participate in MOC. Currently, 1482 certified otolaryngologists participate in MOC, including all primary certificate holders certified in 2002 and thereafter and all neurotology and sleep medicine certificate holders. The Board of Directors of the American Board of Otolaryngology has voluntarily agreed to participate in MOC in support of the program. The Federation of State Medical Boards is in the process of developing a maintenance of licensure program, which parallels MOC in many ways [6]. Although early in development, the Federation of State Medical Boards envisions that all physicians (regardless of whether they are certified by a specialty board) will need to participate in maintenance of licensure. Board-certified physicians who participate in their respective board’s MOC program will meet the maintenance of licensure requirement, however. It is likely that all licensed physicians may be required to participate in some sort of maintenance of licensure/certification program in the future. Finally, any otolaryngologist–head and neck surgeon with a timeless certificate can voluntarily participate in MOC. Internists who have participated in the American Board of Internal Medicine MOC program have found it valuable [7–9]. Voluntary participation does not jeopardize a timeless certificate holder’s certificate, and although MOC in otolaryngology–head and neck surgery is still in its infancy, the ABOto will be offering more components in the not-too-distant future. We hope that many in our specialty find these various components to be useful as a value-added program and voluntarily participate, as do the ABOto directors.

Summary Based on its mission, the fiduciary responsibility of the ABOto is to the public. When examining an issue, the ABOto directors view the debate from several perspectives, but in the end, what is best for the publicdour

1356

MILLER

patientsdguides the decision-making process. The issue of quality of health care always has been important, as indicated by the milestones of quality improvement that have occurred over the years. The issue is rightfully prominent on the radar screen nationally. All the groups in health care are stepping up to the plate, including physician groups such as the American Board of Otolaryngology. When I get on a plane, I feel comfortable that the pilot is well trained and has undergone a vigorous ‘‘certification’’ process. Each pilot is tested in a simulator at least annually and is observed by a Federal Aviation Administration investigator while actually making a flight. The industry also promotes a culture in which ‘‘errors’’ can be reported on a nonpunitive basis, which further adds to safety. Because the airline industry, in conjunction with the Federal Aviation Administration, has made commercial aviation one of the safest means of travel, aviation is sometimes mentioned as a potential model for health care. Although an attractive consideration, it is not clear that all aspects of their safety system are possible or practical in health care. The important point, however, is that people in the airline industry have collaborated to develop a system that has been successful. There is no reason why we physicians cannot do the same. Some physicians become defensive when the question of quality of care is raised. Most otolaryngologist–head and neck surgeons practice high-quality medicine on most patients most of the time; however, we all know that there are a few practitioners to whom we would not refer a patient or a family member because of questions of practice. These few ‘‘bad apples’’ exist, and everyone agrees that they should be identified, given an opportunity to improve their practices, and removed from practice if unsuccessful. Fortunately, few practitioners fit into this category. Most, if not all, of us could do a better job of staying current within the field, being more consistent in managing patients, or improving patient safety by reducing errors such as incorrect prescriptions or wrong site surgery. It is in this arena that quality improvement programs, such as MOC, can improve the practices of our fellow otolaryngologists. This is the raison d’eˆtre of MOCd not to penalize practitioners but to raise the standard of care by ensuring that participants are aware of and knowledgeable about current information in the field, are attentive to patient safety, and strive to improve patient communication, among other attributes. It is hoped that MOC can provide a mechanism or process to achieve this laudable goal. The ABOto recognizes that MOC requires additional expenditure of time and money. The ABOto is sensitive to these issues and is trying to implement a program that is cost effective, minimally intrusive, and meaningful. MOC is a program in evolution. Considering that it is less than 5 years old, MOC already has evolved into a value-added program. Physicians in some specialties have seen a reduction in malpractice premiums by participating in MOC. It seems natural that the same benefits would accrue in our specialty as the otolaryngology MOC program matures. Some health care insurance

CERTIFICATION AND MOC

1357

companies are interested in using MOC as one measure of performance for pay-for-performance programs, which makes participation in MOC and pay-for-performance more efficient. When we were in medical school and residency, our performance was reviewed regularly in the form of tests, rotation evaluations, and other mechanisms. Ultimately, we passed the ABOto examination and became certified but without any subsequent formal evaluation to ensure that we were staying current on the latest medical knowledge. Most certified otolaryngologists provide excellent care. The national movement to improve the quality of health care mandates a program to ensure that we remain up-to-date, however, which is the reason for MOC. If we physicians do not address the health care quality issue, someone else in the form of the government or another nonphysician group will. The thrust of the MOC program is a continuous quality improvement program. It is not intended to be punitive but rather serve as a stimulus for all of us to stay current in our practices. Our patients deserve no less.

References [1] Cantrell R, Goldstein G. The American Board of Otolaryngology 1924–1999. Houston (TX): The American Board of Otolaryngology; 1999. [2] Flexner A. Medical education in the United States and Canada. New York: Carnegie Foundation for the Advancement of Teaching; 1910. [3] Brennan T, Horwitz R, Duffy D, et al. The role of physician specialty board certification status in the quality movement. JAMA 2004;292(9):1038–43. [4] Kohn L, Corrigan J, Donaldson M, editors. To err is human: building a safer health system. Washington, DC: Institute of Medicine; 2000. [5] Committee on Quality of Health Care in America, Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington, DC: Institute of Medicine; 2001. [6] Steinbrook R. Renewing board certification. N Engl J Med 2005;353(19):1994–7. [7] Brennan T. Recertification for internists: one ‘‘grandfather’s’’ experience. N Engl J Med 2005; 353(19):1989–92. [8] Baron R. Personal metrics for practice: how’m I doing? N Engl J Med 2005;353(19):1992–3. [9] Batmangelich S, Adamowski S. Maintenance of certification in the United States: a progress report. J Contin Educ Health Prof 2004;24(3):134–8.