Re: Medical Professionalism And Assessment

Medical professionalism and assessment systems 1

Essay for the module of ‘Learning and Teaching for Adults’ Master of Arts in Clinical Education, Institute of Education, University of London Module tutor: Dr. Tony Nasta; Essay supervisor: Dr. Norman Lucus Title Current usage and limitation in assessment for medical professionalism in the UK Akira Naito, Aug. 2008 (4893 words)

As much as or more than through the constraints of curriculum and syllabus, the acquisition of legitimate culture and the legitimate relation to culture is regulated by the customary law which is constituted in the jurisprudence of examinations and which owes its main characteristics to the situation in which it is formulated. Pierre Bourdieu and Jean-Claude Passeron 1990; p.142 ‘Society’ is increasingly viewed and treated as a ‘network’ rather than a ‘structure’: it is perceived and treated as a matrix of random connections and disconnections and of an essentially infinite volume of possible permutations. Zygmunt Bauman 2005; p.3

Medical professionalism and assessment systems 2

Abstract Assessment systems for medical professionals, in all aspects of knowledge, skills and attitudes, have constantly been developing and revising their structure. The assessment system in the workplace was recently introduced in the foundation years and individual speciality training phases. Time period for the workplace assessment may soon extend to include one’s entire professional lifetime. The forthcoming revalidation scheme is one example. Assessment systems for both formative and summative purposes have increasingly involved more in the attitudinal aspect, namely medical professionalism.

The General-Medical-Council (GMC) published the Tomorrow’s-Doctors [1993, and a revision in 2003] and the Good-Medical-Practice [1995 and 2006] as guidelines for pre-registration curricula and clinical practice, respectively. According to those documents, practical governing organisations (e.g. the Royal Colleges and medical schools) have refined their programmes. To define the attitudinal domain that should be the focus in the workplace, the Royal-College-of-Physicians [2005] conducted a survey regarding medical professionalism, and reported their agreed definition in the Doctors-in-society with results of their survey.

The present essay examines the contemporary concept of medical professionalism and its assessment system in the UK to understand current values and expectations on medicine and to investigate potentials and limitations of its assessments. (194 words)

Medical professionalism and assessment systems 3

Background From its beginning in 1948, when the National-Health-Service (NHS) was established, the healthcare system in the UK (including medical education and its curriculum) frequently revises its structure. The introduction of the Modernising-Medical-Careers programme [2005] was a recent example of this revision. The initial practical implementation process of this programme was heavily criticised, leading to labelling it ‘disastrous’ because it left thousands of junior doctors without careers. Subsequent adjustments (see the recommendations made in the-Independent-Inquiry-into-Modernising-Medical-Careers [Tooke et al., 2008] for details) have been made in the programme. However, the basic underlying principles and goals of the re-structuring remain the same: that is, the creation of a transparent unified and integrated curriculum that lasts throughout a clinician’s entire career pathway from his/her undergraduate education [General-Medical-Council, 2003], postgraduate training(s) [Postgraduate-Medical-Education-and-Training-Board, 2007; TheFoundation-Programme, 2007], through a lifelong programme of continuous professional development (including ongoing revalidation, re-licensing and re-certificating [Chambers et al., 2008; Royal-College-of-General-Practitioners, 2008]). This trend of unification reflects a proposal to integrate two governing educational bodies (i.e. the General-Medical-Council: GMC and the Post-graduate-Medical-Education-and-Training-Board: PMETB) starting in 2010.

These revisions in the UK aim to improve medical education by integrating the educational systems within the entire healthcare system in order to espouse public inquiries (the-NHSNext-Stage-Review [Darzi, 2007], for example) which reflect contemporary expectations, of

Medical professionalism and assessment systems 4

the public and also of doctors, on medicine as a healthcare profession. The aim of the revisions also mirrors contestations against continually emerging socio-cultural and political challenges associated with, and generated by, the ongoing advancements particularly in medicine and technological science. These changes have resulted in altering the concepts of knowledge and profession.

Concepts of knowledge and profession at the present era Over the recent decades, socio-cultural and technological advancements have contributed to the development of explicit detailed specialised knowledge and skills that have lead to a number of specialities as individual professional bodies. At the same time, the advancements have also enabled non-professional individuals to have an access to myriads of explicit information, to which only professionals would traditionally have had access. As a result, professionals act more as an ‘interpreter’ (or even a mediator) than a ‘legislator [Bauman, 1987].’ Bauman explained this shift of social role by describing the work of the profession as changing from serving their professional knowledge (by stating ‘what was the case,’ and offering definitive judgement and prescriptions to the case) to offering their professional insights into problems/conditions and professional support so that individuals can judge and manage themselves with minimum help.

Medical professionalism and assessment systems 5

Together with this recent transformation, a professional individual may have become categorised himself/herself as a non-professional in the other (but closely-related) areas of specialities. Therefore, a capability and expertise of collaborative teamwork across multiand inter-disciplinary professional members, which could also include ‘non-professionals,’ becomes increasingly important to effectively practise any vocational work as a modern professional. Thus, the relationship between (specialised) professionals, fellow (other) professionals and non-professionals has become increasingly more democratic, and the relationship has formed a hybrid partnership-oriented association whereby each team member needs to take both his/her specific but shared responsibility [Fainzang, 2005]. In this regard, the team for medical practice (in which healthcare team members share the goal to attain the best patient-care and safety) may now include patients. This role of patient means not only the Sir William Osler’s notion of the “patient is the best teacher” but it also signifies the partnership as an active player in medical practice.

A team for an enterprise with this partnership-oriented relationships (especially of multidisciplinary members) can be viewed as ‘communities of practice [Wenger, 1998]’ where every member shares goals and ‘culture [Bruner, 1996]’ by taking part in social interactions with various other members. In this view, the process of negotiation and re-conciliations of problems, which can sometimes comprise contradictions within team members, may be a key activity leading to a new type of professional expertise labelled as the ‘knot-working [Engeström, 2004]’:

Medical professionalism and assessment systems 6

… a new generation of expertise … not based on ‘supreme and stable individual knowledge and ability’ but on ‘the capacity to cross boundaries and to negotiate and improvise “knots” of collaborations in meeting constantly changing challenges ...’ [p.143]

This ‘knot-working’ expertise may be supplemental to traditional expertise domains, which focus mainly on specialised knowledge and skills. In this regard, Bernstein [1999] made a distinction with labelling two types of discourses in (horizontal and vertical) knowledge describing the new horizontal type as a form of knowledge that “… has a group of wellknown features: it is likely to be oral, local, context dependent and specific, tacit, multilayered, and contradictory across but not within contexts.” This can be differentiated from a traditional vertical type of knowledge that “… takes the form of a coherent, explicit, and systematically principled structure, hierarchically organised, as in the sciences, or … a series of specialised languages with specialised modes of interrogation and specialised criteria for the production and circulation of texts as in the social sciences and humanities [Bernstein, 1999; p.4].”

The current shifts in the relationships between professionals and clients/customers and in the expanded view of expertise to include a horizontal discourse, together with an increased number of specialities and an improved accessibility to specialised information, all contribute to the changes in the concept of the profession, specifically professionalism. The definition of professionalism, therefore, differs as a function of one’s perspective in view of

Medical professionalism and assessment systems 7

each socio-cultural standpoint. Medical professionalism is not an exception, of course, but a significant exemplar.

Medical professionalism in the UK The General-Medical-Council (GMC), the regulating body which all clinicians must register in order to practise medicine in the UK, published the Tomorrow’s-Doctor [1993, and a revision in 2003] to provide a guideline of the principles of professionalism for pre-register medical students and apprentices. More recently, the GMC published the Good-Medical-Practice [1995, and a revision in 2006] to provide specific examples of medical professionalism in individual situations. This Good-Medical-Practice [General-Medical-Council, 2006] guideline is now utilised to help determine whether or not doctors act in a professional manner, namely performance or health through fitness to practise procedures. At this point, assessment systems (to be used throughout one’s clinical career from undergraduate education to revalidation) that allow individual doctors to continue their registrations to the GMC are now required to be consistent with the principles and values stated in the document.

The Royal-College-of-Physicians [2005] also recently conducted a research programme to develop a definition of medical professionalism in the Doctors-in-society document. The Working-Party of the Doctors-in-society, which consisted of representative members

Medical professionalism and assessment systems 8

nominated from both professional clinicians and non-professional citizens, agreed on the statements of the Doctors-in-society, and provided a description of the need for a clear definition of professionalism: The practice of medicine is distinguished by the need for judgement in the face of uncertainty. Doctors take responsibility for these judgements and their consequences. A doctor’s up-to-date knowledge and skill provide the explicit scientific and often tacit experiential basis for such judgements. But because so much of medicine’s unpredictability calls for wisdom as well as technical ability, doctors are vulnerable to the charge that their decisions are neither transparent nor accountable. In an age where deference is dead and league tables are the norm, doctors must be clearer about what they do, and how and why they do it. [p. xi]

The present essay investigates the contemporary concept of medical professionalism by examining the Doctor-in-society [Royal-College-of-Physicians, 2005], Good-Medical-Practice [General-Medical-Council, 2006] and the assessment instruments currently implemented in the UK. The two primary questions that are addressed in this essay are: (1) What is the professionalism in medicine at the present time? and (2) What are the actual procedures currently in place to assess and develop professional standards in relation to the defined professionalism? In order to address the first question, the next section begins with a summary of the Doctors-in-society document, in order to provide an overview of the current discourses

Medical professionalism and assessment systems 9

regarding medical professionalism. Next, this discourse is compared with the Good-MedicalPractice guideline to develop a working definition and description of medical professionalism in the UK. To address the second question, the essay then discusses contemporary concepts of the assessment process, particularly with regards to competence-based assessment [Wolf, 1995] and in the context of the lifelong learning in post-compulsory education and training [Boud and Falchikov, 2007; Ecclestone, 2005; Knight and Yorke, 2003]. Finally, the current assessment systems used in the UK [Postgraduate-Medical-Education-and-Training-Board, 2007; The-Foundation-Programme, 2007] are analysed. As will be seen, this analysis indicates that the modern notion of professionalism emphasises the aspect of mutually interacting and socially constructed integrated network. This is in contrast to a generation ago where authoritative definitions of professionalism might have been taken for granted.

Terminology of medical professionalism in the UK Because the definition of medical professionalism varies as a function of context, any discussion of this concept should begin with a working definition to specify what it means in the current context. The Doctors-in-society document described the specific terms, that would be used for their definition of the medical professionalism. Table 1 shows a list of all of the words specified in the document that are labelled as the abandoned, restricted and retained terms.

Medical professionalism and assessment systems 10

Table 1: Terminology for description of professionalism in the “Doctors in society”

Terms

Abandoned words

Restricted words

Retained words to use

mastery, autonomy, privilege, self-regulation, power, superiority, control, independency and personal-authority

competence, art, morality, altruism, vocation, socialcontract and appropriateaccountability

excellence, judgement, moral-contract, practice, profession, knowledge, skills, science, society, service, commitment and integrity

The words of ‘mastery,’ ‘autonomy,’ ‘privilege’ and ‘self-regulation’ were abandoned due to their ambiguous and potentially misleading connotations. The unfavourable connotations of these are ‘control,’ ‘personal authority,’ ‘power,’ ‘superiority’ and ‘independency from the patient and preponderance of medical evidence’ for example. These decisions, together with the notions of ‘shared decision-making’ and ‘shared practice,’ is consistent with a shift in focus from a paternalistic and therapeutic style in hierarchical relationships to a supporting and empowering style in mutually respecting partnerships between healthcare-providers and clients.

Also the traditional use of the terms ‘competence,’ ‘art’ and ‘social contract and morality’ was identified as potentially confusing and restricting. The term of ‘competence’ in modern educational discourses often connotes a ‘minimum competency’ particularly for the purpose of passing a mark of tick-box (pass-or-fail) criteria in a summative assessment [Wolf, 1995]. The ‘art’ and ‘social contract and morality’ may indicate to letting ambiguity in decisionmakings survive in doctor’s mind without an explicit explanation. These connotations may

Medical professionalism and assessment systems 11

signify a separation, between doctor(s) and patient(s) and even a separation between doctors (that is, between one who is assessed and the other who assesses), rather than teamwork with the shared aim and goal, namely for the public good. Thus, ‘excellence,’ ‘judgement’ and ‘moral contract,’ respectively, were recommended to use as alternative terms useful for reflecting current medical professionalism, since these terms better place an emphasis more on mutual agreement [Royal-College-of-Physicians, 2005].

In contrast, the notions of ‘knowledge,’ ‘skills,’ ‘science,’ ‘practice,’ ‘profession,’ ‘society,’ ‘service,’ ‘commitment’ and ‘integrity’ were retained as descriptors for professionalism because they have less ambiguity and a neutral connotation with regards to equality and dignity. The terms ‘vocation,’ ‘appropriate accountability’ and ‘altruism’ were also retained but with some qualifications. The qualifications again emphasise the equal and co-operative associations between doctor(s) and patient(s) as opposed to any connotations implying a separated/dichotomised hierarchical relationship such as one in which doctors have a ‘Godgiven status’ or a ‘managerial accountability’ and that a doctor should ‘sacrifice’ oneself entirely for the profession of medicine, respectively [Table 1].

In summary, the terms applied in the Doctors-in-society document were selected for the purposes of (1) decreasing ambiguity with particular connotations of hierarchical (paternalistic) relationship, and (2) increasing an acknowledgement of equality (agreements) and human dignity which is thought to enhance effective teamwork and a mutual respecting partnership.

Medical professionalism and assessment systems 12

The working definition of the medical professionalism The final definition of medical professionalism presented in the Doctors-in-society document is: “… a set of values, behaviours and relationships that underpin the trust the public has in doctor.” The specific descriptions of medical professionalism provided include: In their day-to-day practice, doctors are committed to: integrity, compassion, altruism, continuous improvement, excellence, working in partnership with members of the wider healthcare team. These values, which underpin the science and practice of medicine, form the basis for a moral contract between the medical profession and society. Each party has a duty to work to strengthen the system of healthcare on which our collective human dignity depends.

These statements have marked a shift towards mutual respecting partnerships in a day-today practice between doctors, and other healthcare professionals, and also include patients and their families as members of an integrated team. The specific implementation of the concept of professionalism is made for two levels of operation [Royal-College-of-Physicians, 2005]:

(1) Institutional level as a profession: a. Partnership and mutual respect in patient-doctor interaction; and

Medical professionalism and assessment systems 13

b. Value and purpose of the profession in the context of a healthcare system. (2) Individual level as a professional: a. Respect and care for well-being and human dignity in patient; and b. Compassion and continuous improvement in doctor.

In the institutional level of operation, the Doctors-in-society document identified six themes where the concept has important implications: (1) leadership, (2) teams, (3) education, (4) appraisal, (5) careers and (6) research [Royal-College-of-Physicians, 2005]. At the individual level, the Good-Medical-Practice guideline lists seven areas of applications: (1) Good professional practice, (2) Maintaining good medical practice, (3) Relationships with patients, (4) Working with colleagues, (5) Teaching and training, (6) Probity, and (7) Health. A list of specific duties associated with each principle serves as a practical guideline for implementation, with a rationale that “… patients must be able to trust doctors with their lives and health.” These principles and the associated rationale are consistent with the major principle of the definition of the medical professionalism presented in the Doctors-in-society document, which itself was the result of a nationwide survey and subsequent discussions. For the purpose of this essay, the above definition and description of professionalism provided in the Doctors-in-society document is used.

Medical professionalism and assessment systems 14

Assessment systems for the medical professionalism as a practical implementation The assessment systems and instruments used in the workplace (both in individual and institutional levels of operation) may have a significant influence on ultimate performance. Consistent with this idea, Boud [1988] argued that “... assessment methods and requirements probably have a greater influence on how and what students learn than any other single factor [quoted by Pickford and Brown, 2006].” One of the explanations for this influence comes from behavioural theory. Specifically, this theory argues that the assessment process, including both preparative and implementing courses of action, can provide external (and also introjected/internalised at times) motivation (whether with or without intrinsic incentives) for change in behaviour. That is, results of the assessment process can provide rewards (which increase specific behaviours) or punishers (which decrease specific behaviours). This is also consistent with some network theories such as the theories of ‘reciprocity [Kiyonari et al., 2000; Siegrist, 2005]’ and ‘indirect reciprocity [Nowak and Sigmund, 2005].’ Hence, designing effective workplace based assessment systems should be one of the key strategies to ensure effective and practical implementation of medical professionalism.

For the implementation of medical professionalism in the UK, the primary goal has been described as one that will serve “… to underpin the ‘trust’ the public has in doctors [RoyalCollege-of-Physicians, 2005].” Although quantifying the degree of quality is recognised as a difficult challenge [Epstein, 2007], the trust in individual level of operation can be said to comprise three different practical domains of capabilities, namely knowledge, skills and

Medical professionalism and assessment systems 15

attitudes. The attitudinal facet further consists of (1) suitable manners and etiquettes as explicit behaviours [Kahn, 2008] and (2) proper (value) judgement in each situation invisibly nurtured and embedded in a culture of their communities of practice.

Professional standards for assessment of professionalism, in the institutional level of operation, are described [Postgraduate-Medical-Education-and-Training-Board, 2008] as a series of statements under the five headings:

(1) Planning:

(i)

Curriculum purpose and development; (ii) Assessment system must be fit for the purpose;

(2) Contents:

(iii)

Content of the curriculum; (iv) Content of the assessment will be referenced to all of the areas of the Good-Medical-Practice;

(3) Delivery:

(v)

Managing curriculum implementation; (vi) Model of learning; (vii) Learning experiences; (viii) Assessment system methods;

(4) Outcomes:

Medical professionalism and assessment systems 16

(ix)

Supervision of the trainee; (x) Role of the Assessor; (x-i) Assessment feedback; (x-ii) Standards for classification of performance and/or competence; (x-iii) Documentation will be standardised and accessible nationally;

(5) Review:

(x-iv) Curriculum review and updating; (x-v) Resources; (x-vi) Lay and patient involvement; (x-vii) Equality and diversity.

Three of these five categories (designing contents, delivery of assessment system, and utilising outcomes of assessment data) are particularly associated with a practical implementation of the medical professionalism in a workplace-based assessment. Hence, the following sections utilise these categories in order to investigate the potentials and limitations of assessment for professionalism.

Designing contents for effective assessment systems in the workplace Many aspects of assessment (including validity, reliability, feasibility, cost-efficiency, acceptability, authenticity, manageability, consistency, transparency,

fairness

and

educational-impact) have been suggested as important to take into account in order to maximise the efficacy of assessment instruments [Epstein, 2007; Pickford and Brown, 2006; Swanwick and Chana, 2005]. One assessment instrument may be strong in one aspect but weak in others, and these individual aspects can sometimes conflict with each other,

Medical professionalism and assessment systems 17

requiring appropriate balance and adequate levels of triangulation [Postgraduate-MedicalEducation-and-Training-Board, 2007]. One attempt to integrate those aspects is the creation of the ‘utility index,’ which adopts what can be viewed as the six most important aspects from the above list, and is calculated by the equation of: ‘utility index = educational-impact x validity x reliability x cost-efficiency x acceptability x feasibility [Postgraduate-MedicalEducation-and-Training-Board, 2007; p.7-12. for further detailed descriptions].’ The usefulness of this index, however, has yet to be determined.

Theoretically, the more tests and clearer the list of objectives in the measures, the better the capabilities can be assessed. However, there needs also to be a balance ‘… between aggregation and specificity [Wolf, 1995; p.73].’ Despite the clear benefit in terms of transparency for both assessors and assessees, well-articulated outcomes can become ‘(minimum) competencies [Wolf, 1995]’ and may have a possibility to promote the ‘surface learning’ rather than the ‘deep learning [Brown and Knight, 1994]’ in professional development due to a large amount of workloads for the mere purpose of recording in their assessment [Ecclestone, 2005]: … the pressure of external targets can make teachers and students adopt a low-risk approach, thereby minimising engagement in order to ‘get through’ the requirements. In contrast to their aims of empowering learners, competence-based and outcomebased models appear often to create a tedious paper chase as part of accumulating evidence of achievement [p.59].

Medical professionalism and assessment systems 18

Using multi-methods and longitudinal assessment systems has been suggested as a way to overcome the limitations and conflicts. Such methods can also avoid excessive burden to assessees at one point in time [Epstein, 2007; Swanwick and Chana, 2005]. For example, the use of a portfolio as a longitudinal learning record with multiple assessment data has been introduced in medical education worldwide, although the efficacy of such a portfolio as assessment for both formative (developmental) and summative (judgemental) purposes has also yet to be studied and determined.

Formative and summative roles for delivery of assessment systems Since 1970’s, a dominant trend in designing assessment systems, particularly for postgraduate qualifications and workplace-based assessment, has shifted its focus [Ecclestone, 2005] from systems that create norm-referencing examination data (for summative purpose) to systems that develop ‘criterion-referencing’ or ‘competence-based’ data (for formative purpose) [Carless et al., 2006; Ecclestone, 2005; Kirkwood, 2007]. The norm-referencing ranking systems basically attempt to compare learners in order to select the better achievers who have relatively higher scores, while the criterion-referencing systems aim to recognise areas of relative weakness in order to foster trainee’s ability and to promote further development. This latter approach is based on the notion of ‘scaffolding’ based and developed on the concept of ‘zone of proximal development [Vygotsky and Cole, 1978]’:

Medical professionalism and assessment systems 19

It is the distance between the actual development level, determined by independent problem solving, and the level of potential development, as determined through problem solving under adult guidance or in collaboration with more able peers. … The zone of proximal development defines those functions that have not yet matured but are in the process of maturation, functions that will mature tomorrow but are currently in an embryonic state [p.86]. The key component of the formative assessment, therefore, is to provide effective feedback (or ‘feed-forward [Pickford and Brown, 2006: p.13]’) that can help the learner to have scaffolds so that his/her independent, self-directed [Knowles, 1975], and ‘reflective [Schön, 1987]’ learning can be further promoted and enhanced.

In addition, the notions of ‘outcome-based’ and/or ‘performance-based’ assessment systems have also been introduced to the concept of ‘competence-based’ assessment particularly in workplace-based assessment systems because of their different connotations [PostgraduateMedical-Education-and-Training-Board, 2008]:

… [Assessment systems] may comprise different methods, and be implemented either as national examinations, or as assessments in the workplace. The balance between these two approaches principally relates to the relationship between competence and performance. Competence (can do) is necessary but not sufficient for performance (does do), and as trainees’ experience increases so performance-based assessment in the workplace becomes more important [p.3].

Medical professionalism and assessment systems 20

Workplace-based assessment of medical professionalism, therefore, should apply this performance-based assessment system, and the following series of instruments have been introduced in the UK [Postgraduate-Medical-Education-and-Training-Board, 2007; TheFoundation-Programme, 2007] to achieve this goal: 

Direct observations using the ‘Direct-Observation-of-Procedural-Skills (DOPS)’ for practical skill performances, ‘mini-clinical-evaluation-exercise (mini-CEX)’ for clinical problem-solving manners, and ‘Case-Based-Discussion (CBD)’ for decision-making and clinical reasoning procedures in the workplace context assessed by supervisors;



‘Multi-Source-Feedback (MSF)’ or the ‘360-degree’ assessments by peers, fellow professionals (in other specialities), other members of team (co-medical professionals), and patients, using: o Mini-Peer-Assessment-Tool (mini-PAT), o Team-Assessment-of-Behaviours (TAB); and



‘Portfolio’ which includes self-reflective log notes on one’s practice and on the feedbacks of above assessment as a practical record of one’s own development.

For quality management and quality assurance purposes, two important considerations have been suggested for applying these workplace-based assessment in practice, namely the number of assessments regarding both (1) assessment time points and (2) assessors performances [Postgraduate-Medical-Education-and-Training-Board, 2007]:

Medical professionalism and assessment systems 21

Extensive sampling for borderline trainees may be needed to precisely identify the problems behind their difficulties so that a plan can be formed to find remedial solutions where possible. The main value of workplace based assessments is that they provide immediate feedback. The information acquired during a workplace based assessment can also provide evidence of progression of a trainee and therefore contribute evidence suitable for recording in their learning portfolio [p.21]. With regard to the aspect of assessors’ performance, the next section explores some specific limitations as well as potential solutions for these limitations.

Interpretation of outcomes in the workplace and assessors’ performance Various assessment systems have been developed on the basis of the fundamental assumption of “… transparent ‘benchmark’ of the performance criteria [Wolf, 1995; p.24].” This assumption often generates “… philosophical tension between what we know and can express, and what we know but cannot express in words [Ecclestone, 2005; p.29].” The area of this philosophical tension becomes the space where the ‘compensation’ would operate [Wolf, 1995]: ... [Assessors] compensate, make allowances, interpret, and explain away. The more experienced the assessor, the more they are operating in a familiar field, and the more they have internalised a model of competence (which may or may not be the same as

Medical professionalism and assessment systems 22

other people’s), the more ‘active’ their judgemental aggregation becomes. … People are often unaware of the degree to which they are operating in this way [p.71].

Those internalised and unaware models of competence may be heterogeneous and distributed within all individuals in the communities of practice as parts of the whole culture or as the ‘distributed cognition [Salomon, 1993]’: There is no doubt that culture is patterned, but there is also no doubt that it is far from uniform, because it is experienced in local, face-to-face interactions that are locally constrained and, hence, heterogeneous with respect to both “culture as a whole” and the parts of the entire cultural toolkit experienced by any given individual [p.15].

Any assessment procedures inevitably involve cultural adjustments to a certain degree: … Assessors do not simply ‘match’ candidates’ behaviour to assessment instructions in a mechanistic fashion. On the contrary: they operate in terms of an internalised, holistic set of concepts about what an assessment ‘ought’ to show, and about how, and how far, they can take account of the context of the performance, make allowances, refer to other evidence about the candidate in deciding what they ‘really meant,’ and so on. [Wolf, 1995; p.67] This statement emphasises the importance of an implicit/unarticulated culture in order to maintain a coherence/reliability within a community of practice by filling the gaps between the content of instructions and the context of the performance. Wolf continued to argue

Medical professionalism and assessment systems 23

that this maintenance of coherence (or an increase of reliabilities) can be achieved by keeping effective networks in the process of implementing the explicit rules in each real situation: The key requirements are exemplars and networks of assessors – plus a good deal of realism about what can be claimed and achieved. … Different users read the same exemplars differently: you cannot assume that the aspects which you consider ‘key’ will be the ones which others identify and generalise from. … The finding underlines how important and, potentially, how effective assessor networks are. They are, in fact, the key element in ensuring consistency of judgement [ibid.; p.76-77].

This type of network activity, labelled as the ‘knot-working [Engeström, 2008]’ activity, often emerges and evolves sporadically (in time and location) and informally within an organisation or system. Although there may be a limit in articulation, a series of considerations for measures has been suggested as a check-list to reduce the level of heterogeneity in each local knot-working activity [Baker et al., 1994; quoted in Swanwick and Chana, 2005]: 

Specification — of standards, criteria, and scoring guides;



Calibration — of assessors and moderators;



Moderation — of results, particularly those on the borderline;



Training — of assessors, with retraining where necessary; and

Medical professionalism and assessment systems 24



Verification and audit — through quality assurance measures and the collection of reliability data [p.465].

Specifically, the anchored rating scales (1: very poor, 2: poor, 3: fails to reach the required standards, 4: fair, 5: good, to 6: excellent, for example) and the borderline methods for grading in assessment systems (pass, borderline, and fail) were introduced to apply in the current workplace-based assessment in order to decrease a possibility of inevitable measurement-errors as well as to increase opportunities to provide assessees with a second chance [Postgraduate-Medical-Education-and-Training-Board, 2007].

Summary and conclusions This present essay has investigated and attempted to answer two questions: (1) What is the current view of medical professionalism in the UK? and (2) What are the procedures currently in place for assessing medical professionalism? To address the first question, the modern concept of medical professionalism in the UK, was discussed with respect to how key documents (the Doctors-in-society and Good-MedicalPractice guidelines), and a working definition of medical professionalism was given as “… a set of values, behaviours and relationships that underpin the trust the public has in doctors.”

Medical professionalism and assessment systems 25

Medical professionalism has increasingly emphasised a ‘mutual respecting partnership’ between doctors, other healthcare professionals, and patients and their families, with a focus on human dignity. In other words, medical professionalism has increasingly emphasised the concept of teamwork, particularly the ‘knot-working’ activity among multidisciplinary members that aims to attain public good, including the best patient care.

To address the second question, the essay reviewed current systems of assessment, particularly workplace-based assessment systems. Assessment systems typically assess three specific practical domains of capability at the individual level; namely knowledge, skills and attitudes. The third domain, attitude, was suggested to be influenced by the culture of institution. Therefore, a practical implementation of medical professionalism in the current workplace-based assessment system was examined in the individual level, but the some of the categories in the institutional level were also reviewed in relation to the individual level of operation, specifically contents, delivery and outcomes.

For designing ‘contents’ and ‘delivery’ of assessment instruments, educational-impact, validity, reliability, cost-efficiency, acceptability, and feasibility were highlighted to be important to take into account, and use of combination of multi-methods and longitudinal assessment systems, a learning portfolio for example, was introduced as a recommendation. The importance of feedbacks in the performance-based assessment system was also emphasised in the use of effective delivery of assessment system.

Medical professionalism and assessment systems 26

In interpreting the ‘outcomes’ of workplace based assessment, the influence of culture as ‘knot-working’ activity of assessors was signified as to minimise the effect of inevitable measurement-errors. The anchored-rating-scales and the-borderline-method have been introduced in many workplace-based assessment systems as potential methods to help to enhance ‘knot-working’ so as to reduce the error.

These recommendations and potential solutions must be studied further to determine their utility and efficacy. However, there appears to be a consistent message in current thinking. Specifically, these attempts should be conducted by not separate individual professionals or institutions, but by an integrated team of communities of practice including professionals, fellow professionals, other professionals and the public/patients as a ‘society [Bauman, 2005]’: ‘Society’ is increasingly viewed and treated as a ‘network’ rather than a ‘structure’: it is perceived and treated as a matrix of random connections and disconnections and of an essentially infinite volume of possible permutations [p.3].

Medical professionalism and assessment systems 27

References Baker, E., O'Neil, H., and Linn, R. (1994). Policy and Validity Prospects for Performance-Based Assessment. American Psychologist, 48(12), 1210-1218. Bauman, Z. (1987). Legislators and interpreters: on modernity, post-modernity, and intellectuals. Cambridge: Polity. Bauman, Z. (2005). Liquid life. Cambridge ; Malden, MA: Polity. Bernstein, B. (1999). Vertical and Horizontal Discourse: an essay. British Journal of Sociology of Education, 20, 157-173. Boud, D., and Falchikov, N. (Eds.). (2007). Rethinking Assessment in Higher Education - Learning for the longer term. Abingdon: Routledge. Bourdieu, P., and Passeron, J.-C. (1990). Reproduction in Education, Society and Culture. London: Sage Publications. Brown, S., and Knight, P. T. (1994). Assessing Leaners in Higher Education. London: Kogan Page Limited. Bruner, J. S. (1996). The culture of education. Cambridge, Mass.: Harvard University Press. Carless, D., Joughin, G., and Liu, N.-F. (2006). How assessment supports learning. Hong Kong: Hong Kong University Press. Chambers, R., Wakley, G., and Bright, P. (2008). Revalidation - prepare now and get it right. Abingdon: Radcliffe Publishing Ltd. Darzi, A. (2007). NHS Next Stage Review Final Report. available online at: http://www.dh.gov.uk/en/Publicationsandstatistics/Publications/PublicationsPolicyAndGuid ance/DH_085825 (accessed 27 Aug. 2008): Department of Health. Ecclestone, K. (2005). Understanding assessment and qualifications in post-compulsory education and training (2 ed.). Leicester: National Institute of Adult Continuing Education. Engeström, Y. (2004). The new generation of expertise: seven theses. In Rainbird, H., Fuller, A. and Munro, A. (Eds.), Workplace learning in context (pp. 145-165). London: Routeledge. Engeström, Y. (2008). From Teams to Knots. New York: Cambridge university press. Epstein, R. M. (2007). Assessment in Medical Education. N Engl J Med, 356(4), 387-396. Fainzang, S. (2005). When doctors and patients lie to each other. Lying and power within the doctorpatient relationship. In van Dongen, E. and Fainzang, S. (Eds.), Lying and Illness. Power and Performance (pp. 36-55). Amsterdam: Het Spinhuis. General-Medical-Council. (2003). Tomorrow's Doctors. available online at: http://www.gmcuk.org/education/undergraduate/GMC_tomorrows_doctors.pdf (accessed 27 Aug. 2008): General Medical Council. General-Medical-Council. (2006). Good Medical Practice. available online at: http://www.gmcuk.org/guidance/good_medical_practice/GMC_GMP.pdf (accessed 27 Aug. 2008): General Medical Council. Kahn, M. W. (2008). Etiquette-based medicine. N Engl J Med, 358(19), 1988-1989. Kirkwood, M. (2007). The contribution of sustainable assessment to teachers' continuous professional development. In Boud, D. and Falchikov, N. (Eds.), Rethinking Assessment in Higher Education (pp. 57-71). Abingdon: Routledge. Kiyonari, T., Tanida, S., and Yamagishi, T. (2000). Social exchange and reciprocity: confusion or a heuristic? Evolution and Human Behavior, 21(6), 411-427. Knight, P. T., and Yorke, M. (2003). Assessment, Learning and Employability. Maidenhead: Open University Press. Knowles, M. S. (1975). Self-directed learning: a guide for learners and teachers. Chicago: Association Press.

Medical professionalism and assessment systems 28 Nowak, M. A., and Sigmund, K. (2005). Evolution of indirect reciprocity. Nature, 437(7063), 12911298. Pickford, R., and Brown, S. (2006). Assessing skills and practice. Abingdon: Routledge. Postgraduate-Medical-Education-and-Training-Board. (2007). Developing and maintaining an assessment system- a PMETB guide to good practice. available online at: http://www.pmetb.org.uk/fileadmin/user/QA/Assessment/Assessment_good_practice_v020 7.pdf (accessed 27 Aug. 2008). Postgraduate-Medical-Education-and-Training-Board. (2008). Standards for curricula and assessment systems. available online at: http://www.pmetb.org.uk/fileadmin/user/Standards_Requirements/PMETB_Scas_July2008_ Final.pdf (accessed 27 Aug. 2008). Royal-College-of-General-Practitioners. (2008). Good Medical Practice for General Practitioners. available online at: http://www.rcgp.org.uk/ (accessed 27 Aug. 2008). Royal-College-of-Physicians. (2005). Doctors in society: medical professionalism in a changing world. available online at: http://www.rcplondon.ac.uk/pubs/books/docinsoc/ (accessed 27 Aug. 2008). Salomon, G. (Ed.). (1993). Distributed cognitions : psychological and educational considerations. Cambridge England ; New York, NY: Cambridge University Press. Schön, D. A. (1987). Educating the reflective practitioner : toward a new design for teaching and learning in the professions (1st ed.). San Francisco: Jossey-Bass. Siegrist, J. (2005). Social reciprocity and health: new scientific evidence and policy implications. Psychoneuroendocrinology, 30(10), 1033-1038. Swanwick, T., and Chana, N. (2005). Workplace assessment for licensing in general practice. Br J Gen Pract, 55(515), 461-467. The-foundation-programme. (2007). The foundation learning portfolio. available online at: http://www.foundationprogramme.nhs.uk/ (accessed 27 Aug. 2008). Tooke, J., Ashtiany, S., Carter, D., Cole, A., Michael, J., Rashid, A., et al. (2008). Aspiring to Excellence findings and final recommendations of the independent inquiry into modernising medical careers. available online at: http://www.mmcinquiry.org.uk/draft.htm (accessed 27 Aug. 2008). Vygotsky, L. S., and Cole, M. (1978). Mind in society : the development of higher psychological processes. Cambridge: Harvard University Press. Wenger, E. (1998). Communities of practice : learning, meaning, and identity. Cambridge, U.K. ; New York, N.Y.: Cambridge University Press. Wolf, A. (1995). Competence-based Assessment. Buckingham: Open University Press.

Medical professionalism and assessment systems 29

The actual recommendations in the Doctors-in-society made to the governing organisations include: (1) To strengthen clinical leadership and managerial skills as key competencies of professional practice; and to create a national group to define the requirements for a common forum, in order to speak out on behalf of medicine with a unified voice. (2) To review how doctors can best be supported in their contributions to multiprofessional teams; and to explore ways of strengthening common learning to enable better inter-professional education and training. (3) To review selection procedures (to include lay members in the panels, for example) to identify students with the potential to develop qualities of medical professionalism; and to ensure time for professional engagement with students, including raising managerial and organisational awareness. (4) To review professional content of appraisal, with a view to incorporating professional values as key components in evaluating a doctor’s performance and development. (5) To establish a mechanism to examine how best to improve the management of medical careers. The goal is to create career paths that meet the present and future needs of patients, reflecting demographic changes in both society and medicine. (6) To establish a forum as the funders to call for and consider research proposals into how medical professionalism might best be studied as part of an overall goal to improve health outcomes.