The Core Competencies: Why, What, And How?

The Core Competencies: Why, What, and How? CORD-EM Best Practices in Residency Training: Reaching for Excellence February 2003 Pamela L. Dyne, MD

Arthur Sanders, MD

Associate Professor of Medicine D. Geffen School of Medicine at UCLA Residency Director UCLA/Olive View-UCLA Emergency Medicine

Professor of Emergency Medicine University of Arizona COM Chair, RRC-EM

OUTCOMES SHIFT - WHY?  Accountability - Our system of medical education

relies heavily on considerable public funding. We therefore need to be accountable to the public in terms of both meeting public needs and preparing well-qualified new physicians in the most costeffective way possible.  Process vs. Outcome - Measuring program quality by examining structure and process is not a direct or complete measure of the quality of the educational outcomes of a program. ACGME

OUTCOMES SHIFT - WHY?  Need



for Better Measures of Quality -

Availability of educational outcomes-based data is necessary to inform policymakers and others who have become increasingly focused on issues related to funding for medical education, and, most recently, on patient safety. It is incumbent upon us as medical educators to demonstrate the effectiveness of our educational programs and to be held accountable for our work. ACGME

QUALITY IMPROVEMENT  Structure

- institution, number of faculty,

patient volume and acuity, number of procedures, curriculum schedule

 Process

- resident shifts and

responsibilities, block rotations, conference attendance, feedback and evaluations, teaching methods, etc.  Outcome - board certification, successful completion of program, etc.

ACGME COMPETENCIES “Minimal Threshold Model” for GME accreditation  Minimal

processes for education curriculum, conferences, patient populations, procedures, faculty, etc.  Program has the potential to educate competent physicians

ACGME OUTCOMES Educational Outcomes - “Evidence showing the degree to which programs purposes and objectives are or are not being attained, including achievement of appropriate skills and competencies by students.”

ACGME Outcomes Project

ACGME COMPETENCIES  “In

the competency-based model …programs will be asked to show how residents have achieved competency-based educational objectives and in turn, how programs use information drawn from evaluation of those objectives to improve the educational experience of the residents. Stated another way, the minimal threshold model identifies whether a program has the potential to educate residents; the competency-based model examines whether the program is actually educating them.” ACGME Outcome Project

ACGME COMPETENCIES Competency Based Model  Educational

objectives will need to be competency based  How programs evaluate competencies based on the educational objectives  How programs use evaluation information to improve the educational experience.

IDENTIFYING COMPETENCIES 





Identifying the competencies was stimulated by increased attention to how adequately physicians are prepared to practice medicine in the changing health care delivery system. The ACGME derived its general competencies through a careful study of existing research on general competencies for physicians. It also gathered input on the proposed competencies from various constituencies and stakeholders of GME. From this process, the Outcomes Project Advisory Committee identified six general competencies that were subsequently endorsed by the ACGME in February 1999. ACGME

Program Requirements-draft RequirementsPrograms must define the specific knowledge, skills, behaviors, and attitudes required and provide educational experiences as needed in order for their residents to demonstrate the following:

The ACGME General Competencies:  Patient

care  Medical knowledge  Practice-based learning and improvement  Interpersonal and communication skills  Professionalism  Systems-based practice (What are the competencies for EM?)

ACGME Assessment Toolbox      

360° evaluation Chart stimulated recall Checklist eval Global rating of live or recorded performance OSCE Procedure, operative or case logs

      

Patient surveys Portfolios Record review Simulations and models Standardized oral exam Standardized patients Written exam (MCQ)

EMERGENCY MEDICINE COMPETENCIES Who

defines the specific EM competencies? The Model of the Clinical Practice of Emergency Medicine RRC-EM Task Force

…and CORD-EM is at the table “The ACGME Core Competencies: Getting Ahead of the Curve” CORD-EM, March 2002 Academic Emergency Medicine, November 2002, Vol. 9, No.11

Patient Care: Goals  ACGME:

“Residents must be able to provide patient care that is effective, appropriate, and compassionate for the treatment of health problems and promotion of health.”  CORD: “EM residents must be able to provide patient care that is timely, effective, appropriate, and compassionate for the management of health problems and promotion of health.” King, Schiavone, Counselman, Panecek, Patient Care Competency in Emergency Medicine Graduate Medical Education: Results of a Consensus Group on Patient Care. AEM 2002;9:1227-1235

EM Patient Care: Objectives a.

b. c.

Gather accurate, essential information in a timely manner from all sources, including medical interviews, physical examinations, outof-hospital care personnel, medical records, and diagnostic/therapeutic procedures. Integrate diagnostic information and generate an appropriate differential diagnosis. Implement an effective patient management plan including therapy, appropriate consultation, disposition, and pt. education

EM Patient Care: Objectives Competently perform the diagnostic and therapeutic procedures and emergency stabilization considered essential to the practice of EM. b. Demonstrate the ability to appropriately prioritize and stabilize multiple patients and perform other responsibilities simultaneously. a.

EM Patient Care: Assessment Checklist

Evaluation of Live Performance (Direct Observation):  “Snapshot” approach using on-shift

attending and repeated isolated mini-evals  Comprehensive approach involving a non-onshift faculty member for several hours at a time

+Advantages: real clinical environment, time efficient for residents and faculty (potentially) -Concerns: Hawthorne effect, observer training bias, disturbance of physician-patient relationship

EM Patient Care: Assessment •

Simulations and Models with D.O. procedures and stabilization

 Secondary

methods:

ALL toolbox items relevant; patient surveys, record review, 360° eval, and procedure logs limited applicability

Medical Knowledge: Goals  ACGME:

“Residents must demonstrate knowledge about established and evolving biomedical, clinical, and cognate (eg. epidemiological and socialbehavioral) sciences and the application of this knowledge to patient care.”

EM Medical Knowledge: Goals  CORD-EM:

“Residents are expected to formulate an appropriate DDx with special attention to life-threatening conditions, demonstrate the ability to utilize available medical resources effectively and concurrent with patient care, and apply this knowledge to clinical problem solving and clinical decision-making.”

Wagner, MJ, Thomas, HA, Application of the Medical Knowledge General Competency to Emergency Medicine, AEM 2002;9:1236-1241

EM Medical Knowledge: Objectives a. b. c. d. e. f. g.

Identify life threatening conditions Identify the most likely diagnosis Synthesize acquired patient data Identify how and when to access current medical information Properly sequence critical actions in patient care Generate a DDx for an undifferentiated patient Complete disposition of patients using available resources

EM Medical Knowledge: Assessment 

Checklist Evaluation of Live Performance (Direct Observation):  Progressive questioning by on-shift attending

Beyond the usual applied questions Content area specific approach 

 Comprehensive approach involving a non-on-

shift faculty member for several hours at a time Structured clinical assessment

EM Medical Knowledge: Assessment  Objective

standardized examination (OSE)

 National In-Service exam  Locally written tests 

Topic specific modular curriculum with exams

 Computer-based learning modules with exams

+Advantages: objective, criterion referenced, prep for “the real thing,” easy to track and provide data to RRC

−Concerns: labor and time intensive, (external locus of control for learning may not promote development of career learning habits)

EM Medical Knowledge: Assessment  Simulations

and models

 Procedures and low-frequency, critical content

areas  OSCE, SP, computer models  Needs objective evaluation tool development

in style vs. observation Inconsistencies knowledge?  Chart-stimulated recall needs scoring protocol  Global rating form less precise, halo vs. millstone effect  360° eval TNTC confounders  portfolios ? for remediation

 Classroom

Practice-Based Learning and Improvement: Goals  ACGME:

“Residents must be able to investigate and evaluate their patient care practices, appraise and assimilate scientific evidence, and improve their patient care practices.”

EM Practice-Based Learning and Improvement: Objectives A.





Analyze and assess your practice experience and perform practice-based improvement. Locate, appraise, and utilize scientific evidence related to your patient’s health problems and the larger population from which they are drawn. Apply knowledge of study design and statistical methods to critically appraise medical literature.

Hayden, SR, Dufel, S, Shih, R, Definitions and Competencies for Practice-based learning and improvement, AEM 2002;9:1242-1248

EM Practice-Based Learning and Improvement: Objectives Utilize information technology to enhance your education and improve patient care.  Facilitate the learning of students, colleagues, and other health care professionals in emergency medicine principles and practice. 

EM Practice-Based Learning and Improvement: Assessment  Portfolio  CSR  360°

QA projects, individual learning plans, journal club write-ups, etc…self-reflection of learning and how their EM practice might change as a result

Focus on decision-making, test interpretation, rationale for diagnostic and therapeutic interventions; “educational prescription”

global eval

frequency, efficiency, and utilization of evidence in clinical decision-making; lectures, bedside teaching

 Computer

simulation Web-based modules requiring searching, analyzing medical info resources; monitoring software to automatically record computer sessions

Interpersonal and Communication Skills: Goals  ACGME:

“Residents must be able to demonstrate interpersonal and communication skills that result in effective information exchange and teaming with patients, their families, and professional associates.”

EM Interpersonal and Communication Skills: Objectives a. Demonstrate the ability to respectfully, effectively, and efficiently develop a therapeutic relationship with patients and their families b. Demonstrate respect for diversity and cultural, ethnic, spiritual, emotional, and age-specific differences in patients and other members of the health care team. c. Demonstrate effective listening skills and be able to elicit and provide information using verbal, nonverbal, written, and technological skills. Hobgood, Riviello, Jouriles, Hamilton, Assessment of Communication and Interpersonal Skills Competencies. AEM 2002;9:1257-1269

EM Interpersonal and Communication Skills:Objectives a. Demonstrate ability to develop flexible communication strategies and be able to adjust them based on the clinical situation b. Demonstrate effective participation in and leadership of the health care team c. Demonstrate ability to elicit patient’s motivation for seeking health care d. Demonstrate ability to negotiate as well as resolve conflicts e. Demonstrate effective written communication skills with other providers and to effectively summarize for the patient upon discharge

EM Interpersonal and Communication Skills:Objectives a. Demonstrate ability to effectively use the feedback provided by others b. Demonstrate ability to handle situations unique to EM: Intoxicated patients AMS Delivering bad news Difficulties with consultants DNR/end-of-life decisions Patients with communications barriers

High-risk refusal of care patients Communication with outof-hospital personnel and non-medical personnel Acutely psychotic patients Disaster medicine

EM Interpersonal and Communication Skills:Assessment

 Direct

Observation (D.O.)

+on-shift or not-on-shift attending; direct feedback faculty time, Hawthorne effect, disrupts -expensive doc/pt relationship, lack of objective measures

 Standardized

Patients (SP)

practice low-frequency/high stakes events (death telling)  simulations

and models, OSCE, CSR, standardized oral examinations especially for conflict resolution and consultations

Professionalism: Goals  ACGME:

“Residents must demonstrate a commitment to carrying out professional responsibilities, adherence to ethical principles, and sensitivity to a diverse patient population.”

EM Professionalism: Model Behaviors Arrives on time and prepared to work Appropriate (inoffensive) dress and cleanliness Willingly sees patients throughout the entire shift Appropriate sign-outs, both giving and receiving Observable patient advocacy in disposition Appropriate use of symptomatic care Completes medical records honestly and punctually  Treats patients/families/staff/paraprofessional personnel with respect       

Larkin, Binder, Houry, Adams, Defining and evaluating professionalism: A core competency for graduate emergency medical education. AEM 2002;9:1249-1256

EM Professionalism: Model Behaviors        

Protects staff/family/patient’s interests/confidentiality Demonstrates sensitivity to patient’s pain, emotional state, and gender/ethnicity issues Actively seeks feedback and immediately self-corrects Shakes hands with the patient and introduces himself or herself to the patient and family Effectively coordinates team Unconditional positive regard for the patient, family, staff, and consultants Accepts responsibility/accountability Recognizes the influence of marketing and advertising

EM Professionalism: Model Behaviors 

   

Open/responsive to input/feedback of other team members, patients, families, and peers Uses humor/language appropriately Discusses death honestly, sensitively, patiently, and compassionately Participates in peer-review process Fairness in recruitment of residents, faculty, and staff

EM Professionalism: Assessment  Knowledge

and awareness of professional norms and behavior

written testing detached from clinical setting

 Moral

reasoning and professional capacity

simulations: OSCE, computerized, oral exams, SPs ??? Gender bias in approach: justice vs. care

 Professional

behavior

D.O.; 360° global eval; SPs

Systems-Based Practice: Goals 



ACGME: “Residents must demonstrate an awareness of and responsiveness to the larger context and system of health care and the ability to effectively call on system resources to provide care that is of optimal value.” CORD-EM: “ EM Residents must demonstrate an awareness of health care systems and the ability to effectively mobilize system resources to provide optimal care.”

Dyne, PL, Strauss, RW, Rinnert, S, Systems-based practice: The sixth core competency. AEM 2002;9:1270-1277

EM Systems-Based Practice: Objectives  Understand,

access, appropriately utilize, and evaluate the effectiveness of the resources, providers, and systems necessary to provide optimal emergency care.  Understand the different medical practice models and delivery systems and how to best utilize them to care for the individual patient.

EM Systems-Based Practice: Objectives  Practice

cost-effective health care and resource allocation that does not compromise quality care.  Advocate for and facilitate patients’ advancement through the health care system.

EM Systems-Based Practice: Assessment  Portfolios  Requires resident self-reflection content selection; establishes a pattern for continued life-long learning  Inclusion items for SBP: admin/QA

project, relevant scholarly project, patient care example, etc.  Evaluation of success: standardized vs. based on the educational process faculty development:“teach the teacher”

EM Systems-Based Practice: Assessment  Direct

Observation (D.O.)

especially for multitasking and team-management

 Global

rating

assessment instrument needs development

 360°

evals

takes advantage of peer pressure, very labor intensive

 Standardized

oral exams

modify existing format to include SBP content

What should CORD do?  Develop

validated and reliable assessment instruments  Validated checklist of live performance  New simulators and/or computer-based

interactive programs  Validated portfolio assessment  Attention to low-frequency but critical skills assessment  Focus

on faculty development

 “Teach from the middle”

WHAT SHOULD PROGRAMS DO?  Begin

the think in terms of competencies  Evaluate ACGME toolbox for assessment  Educational faculty retreats  Develop

measurable learning objectives  Assess tools for measuring objectives  DO

NOT DO NOTHING

OUTCOME QUESTIONS  Do

the residents achieve the learning objectives set by the program?

 What

evidence can the program provide that it does so?

 How

does the program demonstrate continuous improvement in its educational processes?

Transforming the Accreditation Process  The

shift from emphasis on structureand-process components to emphasis on outcomes will be a gradual transition. The need for programs to provide evidence of structures and processes will not disappear but will gradually become less critical to the overall accreditation process.