Reading 02 Stcw Learning And Assessment Canada

Evaluating the competencies of seafarers: Challenges in current practice Gholamreza Emad and Wolff-Michael Roth

ABSTRACT: Maritime education and certification and its related assessment of officers and engineers on-board ships are regulated by the international convention on Standard of Training, Certification and Watchkeeping of seafarers (STCW). According to the convention, the relevant administrations have to ascertain that merchant navy ship officers are highly competent for the job and evaluate them according to the Competency-Based Training and assessment system (CBT) before they issue any certificate. The research on marine accidents shows that this is not always the case. The present ethnographic case study highlights and theorizes some of the challenges and contradictions present in the current practice of maritime education and training system (MET) that adopts a recently mandated competency-based training model. In our study, we draw on observations, interviews, and document reviews to articulate some of the contradictions present in the system and that prevent the assessment and further development of mariners’ authentic competency. The assessment for certification is mainly carried out by a series of written and oral examinations, where students are evaluated without being assessed for their ability to perform the task competently. Consequently, the current assessment system is promoting a change in the objectives of the education and training practices: from learning skills and developing knowledge required to safely work onboard ships to how to merely pass the written exams. This research identifies and bridges the gap in the literature and research of competency-based training and assessment in maritime domain and provides practical solution for improving this system. KEYWORDS: Competency; maritime training; assessment; certification; International Maritime Organization.

Introduction: The international seafarer’s community created a certification system to insure that the ships all around the world are manned by competent mariners. As a result ship officers have to be issued certificates of competency (IMO, 1996) to be allowed to work onboard ships—meaning that they are assessed for their compe-

T. Pelton, G. Reis, & K. Moore (Eds.), CONNECTIONS ‘07 (pp. 71–75). © 2007 Gholamreza Emad & Wolff-Michael Roth

72

EMAD & ROTH

tencies. The assessment for and issuance of certificates is delegated to the maritime administration of each country. In Canada, these certificates are issued by the maritime branch of Transport Canada (TC). TC is a governmental organization that is responsible for regulating the requirements candidates have to fulfill during their training, such as working on-board ships for a minimum period of time, attending a series of safety and technical short-duration courses, and passing written and oral examinations—the so called competency certification assessment. In the present study, we analyze how, from the perspective of students and lecturers, the way that the assessment system is set does not allow a legitimate evaluation of the competency of the candidates. On the contrary, since students are not tested for their skills and ability to do the job accordingly, there is no indication that the people who have received the certificate of competency are actually able to perform the job as required—representing an important contradiction within the competency-based learning model adopted in the current maritime education and training (MET) system. Likewise, our analysis shows that the assessment system needs to be redesigned in such a way that it reverts the change of candidates’ objectives—from learning how to pass the test to learning what is considered necessary for a good start in the profession. Ultimately, results imply that written exams may have to be replaced by a competency-based assessment, in which candidates are evaluated at work as they perform their jobs realistically. Method This research is based on an ethnographic case study conducted in a Canadian maritime training institute. Our database was collected during a course presented for mariners applying for second-level watchkeeping certificate of competency. These mariners have acquired the first-level certificate earlier and worked for some time onboard ships in an officer function and then enrolled to upgrade their certificate for working in some higher position onboard ships. The participants (students) have extensive backgrounds as mariners and are from different marine industries including: commercial fishing, passenger ferries, towboat, and coast guard. We attended and videotaped the classes and did interviews with students and the course lecturer. The database includes field notes, videotaped sessions in the classroom and interviews, student’s manual, teaching syllabus, sample tests and certification procedures, rules and regulations. Current practice of maritime education and training The current maritime training system is not considered successful by many in the field as it cannot attain its primary objectives: i.e., to reduce the contribution of human element in accidents through competency training. Our study reveals that the mariners trained in this system are generally dissatisfied with the education that they receive and perceived that is not of much benefit to them. Thus, one mariner stated the issue in a rather typical way: “It is an education designed to screw you up, not education designed to help you in a working world.” Another participant, a certified mariner, expressed his experience of the system and considering himself as being only minimally qualified although he was success-

Evaluating the competencies of seafarers

73

ful in competency exams stated: “Now I am qualified (air quoting) but really I didn’t learn very much, learned a little bit.” These comments from experienced mariners attending a prerequisite course for a second level maritime certificate of competency are not uncommon. Thus our research question is: How has the current practice in maritime education, training and certification resulted in these apparent contradictions between the intentions of the curriculum and the experience of the students? Each candidate should be able to demonstrate the minimum requirements of standard competency prescribed in the convention. These competencies are to be achieved through a combination of education and training plus practical experience onboard ship (IMO, 1996). The maritime administrator assess the students for their competencies before issuing the certificate Assessment for certification Although the current Standard of Training, Certification of Watchkeeping Convention (STCW) stipulates maritime certification to be based on assessing competencies, our study reveals an emphasis on knowledge assessment by means of written and oral examinations. This leads to an inner contradiction whereby some aspects of the system become impediments to achieving the goal of the CBT as a whole i.e. competent performance. It changes the way that the mariners approach learning and shifts their objective from acquiring competencies to memorizing what is required to pass examinations. It was clear from the beginning of the course that the students’ previous experiences with this type of the certification examination were mediating their current approach to learning. Students were concerned about specific issues concerning the exams. For example, they doubted the validity of the exam questions, considering the questions to be outdated and not to have practical implications for their onboard work. Thus, one of the students mentioned: “a lot of the stuff on the exams was like, out dated information, completely irrelevant to what is in practice and even in theory today.” The instructor appeared to be in agreement: “Quite often the examinations are reflecting history, and have not been up-dated. For example, my student may go down and write an examination for a topic, examination was put together in 1976.” Even the conventional assessment system is not achieving its objective—i.e. to assess the knowledge an officer needs onboard ship, as opposed to skills and competencies. Because examination questions are drawn from a question bank, they may appear identically across different examinations: “Yes they were all the same. They haven’t changed in thirty years, forty years, you know, they had converted them from imperial to metric. They are that old” (Rick 13). Our findings are consistent with those of other studies that revealed the severely compromised nature of examinations that reused the same questions year after year (Stutman, 1997). Students’ perceptions about assessment significantly influence their approaches to learning and studying (Struyven et al., 2005). This affects the whole education and training system as the students aim to pass the exams knowing that all they have to do is get ready for the set of largely known

13 Pseudonyms are used throughout the paper to preserve participants’ anonymity

74

EMAD & ROTH

questions: “Ian or Peter or any other instructor have to teach you to pass the exams, right . . . they have to teach you the kind of trickery to get you trough the exams as well, which is the waste of his time, our time and the industry’s time” (Dave). Answering these questions became the primary objective of teaching and learning, leading the instructor to muse, “So what I am having to do is trying to figure out what TC (transport Canada) may want and what I am end up to do is wasting lot of students learning time teaching history instead of teaching today.” He also oriented toward teaching outdated knowledge, because he wants the students to be successful in the certification examinations, rather than focusing on teaching useful, present-day knowledge. Students’ demands resulted in final sessions of the course to be assigned for reviewing the sample questions from the available previous competency examinations question banks. When the students are under pressures for achieving a high score they have to give up or beat the system (Ebel & Frisbie, 1991), teaching the test questions and corresponding answers is one response (Muirhead, 1997). The administrator knew about this flaw in the system but attempted to eschew it by creating a new set of questions. This was a temporary solution as one of the students mentioned: “Now I understand they are coming up with questions but twenty years down the road everyone is going to have those questions.” As the students noticed the solution presented by the administration may not solve the problem but at best it may postpone it. Inappropriate assessment procedures encourage superficial learning and varying the examination questions may not be enough to evoke deep approaches to learning (Ramsden, 1997). It seems that it might not change the perception of students about the examination and as a result may not affects the way that they approach learning. Discussion and conclusion Our study reveals considerable contradictions inside a system designed to improve the education and training of mariners. The structure of the competency certification system exercised an influence on students’ learning process. The competency examinations are important mediators in that they provide a motive for the teaching practice as well as the students’ learning objectives. Assessment serves to define the objective of the whole learning activity. Our study shows that the way that the assessment system is set today does not require the candidate to be competent in the task. As a result, students in this education and training system are not assisted for performing in a way that makes them more competent in doing the tasks required onboard ships. Our suggestion for removing the contradiction and as a result improving the output of the system is to apply the authentic principal of the competency-based assessment system. It requires the candidates to be assessed on the job or job-like environment while they are performing the task. As a result the mariners do not just learn how to be successful in the examinations but they learn to be authentically competent and perform better on the job. Our study implies that the IMO and the examination administrators have to do more than just preparing guidelines regarding CBT but they have to arrange a proper transition process to this training concept. The certification system has to be modified as it has a direct effect on the way that the maritime institutions and work places provide opportunities and the students obtain skills and knowledge

Evaluating the competencies of seafarers

75

require to be a competent seafarer. As a result the mariners would learn more than how to be successful in the examinations: they would learn to be competent and perform better on the job. The implication of the competency based training in its right format will help the students to attain the required competency and maybe to reduce the number of accidents. References Bobb, J. (2000). Using assessments to measure performance. Marine Safety Council Proceedings, 58(4), p. 6-8. Ebel, R. L. and Frisbie, D. A. (1991). Essentials of educational measurement. Englewood Cliffs: Prentice Hall, IMO, (1996). International convention on Standards of Training, Certification and Watchkeeping for Seafarers (1978, as amended in 1995) (STCW Convention). London: International Maritime Organization. Muirhead, P. (1997). An introduction to norm-referenced and criterionreferenced assessment, marking and grading. In Holder L. (Ed.), Maritime education and training: a practical guide (pp. 173–179). London: Nautical Institute. Ramsden, P. (1997). The context of learning in academic departments. In: Marton, F. Hounsell, D. and Entwistle, N. (Eds), The experience of learning: Implication for teaching and studying in higher education (pp. 198–217). Edinburgh: Scottish Academic Press. Struyven, K; Dochy, F; Janssens, S. (2005). Students’ perceptions about evaluation and assessment in higher education: a review. Assessment & Evaluation in Higher Education, 30(4), p. 325-341. Stutman, P. A. (1997). The development and evaluation of examination systems based upon multiple choice criteria. In Holder L. (Ed.), Maritime education and training: a practical guide (pp. 207–220). London: Nautical Institute.