2019 Obe Slide.pdf

1

OBE@RETI

2 0 1 9

MAR 2019

OBE IMPLEMENTATION RANACO EDUCATION & TRAINING INSTITUTE 18-19 MARCH 2019

SITI HAWA HAMZAH

Phd, P.Eng, FIEM, Hon. MAFEO ASEAN Engr, MRM, PSWM Associate Director (Civil Engineering), Engineering Accreditation Department, BEM Malaysia [email protected]

SITI HAWA HAMZAH

PhD, P.Eng, FIEM, Hon MAFEO ASEAN Engr, MRM, PSWM Retired Professor (Civil & Structural Engineering), Faculty of Civil Engineering, UiTM, 40450 Shah Alam, MALAYSIA [email protected] Associate Director (Civil) AD, Associate Director (Structural) AD, Past Council Member IEM, Past Excomm IEM, EAD Lead Panel, P.Eng Principle Interviewer, BEM T&E Comm. Member

33 years teaching & academic experience 20 years experience managing engineering programmes Accreditation experience in more than 120 programmes Completed 21 funded research projects Authored 11 books in structural engineering, more than 165 technical papers, accreditation videos on You Tube External examiner to B. Eng Civil programmes at Monash University Malaysia, Universiti Malaysia Sarawak, INTI International University, Universiti Tun Hussein Onn Malaysia. EAC Advisor to Universiti Malaya OBE Advisor to Universiti Industri Selangor IEM Women Engineer Award Honorary Member to ASEAN Federation of Engineering Organisation Three times recipient of UiTM excellence service award More than 20 research accolades *IEM – The Institution of Engineers Malaysia, BEM – Board of Engineers Malaysia, AD – Engineering Accreditation Department, EAC – Engineering Accreditation Council, UiTM – Universiti Teknologi MARA,

4

WORKSHOP LEARNING OUTCOMES

At the end of the workshop, participants are able to; 1. Clearly explain the definition and purpose of OBE and the concept of constructive alignment in enhancing the effectiveness of HE system and the quality of graduates 2. Effectively involve in curriculum development process 3. Write the learning outcomes correctly to the intended students ability and level of taxonomy 4. Identify and adapt the appropriate learning and teaching methods that ensures the development and achievement of the intended learning outcomes 5. Plan and uses appropriate assessment methods that enable the learning outcome to be measured and analysed correctly 6. Contribute to CQI process in the effort of improving learning process and achievement of the learning outcomes of the students OBE@RETI

MAR 2019

OBE : AT A GLANCE MALAYSIAN QUALIFICATION FRAMEWORK ACT 679

ACCREDITATION NEEDS

6

OBE : WHAT & WHY

 Outcome Based Education (OBE) - process that involves the restructuring of curriculum, assessment and reporting practices in education to reflect the achievement of high order learning and mastery.  The system measured OUTCOMES, making students/learners demonstrate that they have learned the required knowledge and developed the required skill and attributes.  Main components - curriculum development process, learning outcomes - intended student’s ability and level of taxonomy, learning and teaching methods – alignment to learning outcomes, assessment and continual quality improvement (CQI).  OBE shifts the paradigm from TEACHER-CENTERED to the STUDENTCENTERED learning. OBE@RETI

MAR 2019

1 www.mqa.gov.my Engineering Accreditation Council, Board of Engineers Malaysia

2

MALAYSIA : WA – 2009, SA & DA - 2018

2018 – 20 (WA), 11(SA), 9(DA) Australia, Canada, China, Taiwan, Hong Kong, India, Ireland, Japan, Korea, Malaysia, New Zealand, Pakistan, Peru, Russia, Singapore, South Africa, Sri Lanka, Turkey, United Kingdom and United States

The Need for an Outcome Approach forOBE@RETI Continual Programme Improvement

7

MAR 2019

8

MALAYSIA : MOHE 2018

HE4.0: Knowledge, Industry, Humanity HE4.0 : 4IR – Automation, AI and Robotic TVET 4.0 FRAMEWORK

21ST CENTURY PEDAGOGY

• HEUTAGOGY – self determined learning • PARAGOGY – peer oriented learning • CYBERGOGY – virtual based learning OBE@RETI

Learning without lectures Evaluation without examination

Knowledge without borders

FLUID & ORGANIC CURRICULUM LATEST T&L TECHNOLOGIES REDESIGNING LEARNING SPACES

MAR 2019

9

ACT 679 - Malaysian Qualifications Agency Act 2007 (MQA) an act to establish the Malaysian Qualification Agency as the national body to implement the Malaysian Qualifications Framework, to accredit higher educational programmes and qualifications, to supervise and regulate the quality and standard of higher education providers, to establish and maintain the Malaysian Qualification Register and to provide for related matters. ACCREDITATION IS VOLUNTARY BUT is advised OBE@RETI

MAR 2019

10

WHY ACCREDIT ?

 Accreditation - achievement as a result of quality assessment by MQA. A commitment by MQA to all stakeholders in higher education i.e students, parents, employer that the programmes accredited by MQA is quality-assured.  Professional bodies such as the Board of Engineers Malaysia (BEM), Malaysian Medical Council (MMC), Malaysian Nursing Board will use the accreditation to recognise graduates for registration as professional.  Public Service Department (PSD) will use this accreditation status to recognise the qualification for employment in the public service.  Students in accredited programmes are eligible to apply for loan from funding agencies such as PTPTN. OBE@RETI

MAR 2019

11

MQF 2nd edition 2018

MQF - an instrument that develops and classifies qualifications based on a set of criteria that are approved nationally and benchmarked against international best practices, and which clarifies the earned academic levels, learning outcomes of study areas and credit system based on student academic load.

MQF LEVEL

MIN GRADUATING CREDIT

ACADEMIC SECTOR

8

No credit rating 80

PhD by Research Doctoral Degree by Mixed Mode &CW

7

No credit rating 40 30 20

Master’s by Research Master’s by Mixed Mode & CW Postgraduate Diploma Postgraduate Certificate

6

120 66* (inc 6cr gen studies) 36* (inc 6cr gen studies)

Bachelor’s Degree Graduate Diploma Graduate Certificate

5

40

Advanced Diploma

5

4

90

Diploma

4

Min 2 yrs old, Relevant work experience

3

60

Certificate

3

Min 19 yrs old, Relevant work experience

2

30

MQF 2nd edition considered ASEAN Qualifications Reference Framework (AQRF)

2

3R

1

15

wef 1/4/2019 NP

1

3R

OBE@RETI

Certificate

Certificate

TVET SECTOR

LIFELONG LEARNING / APEL (A) – all MUST passed APEL assessment except MQF 1&2 Min 35 yrs old, Bachelor’s Degree, 5 yrs work experience Min 30 yrs old, STPM/Diploma/Equivalent, Relevant work expereince

Min 21 yrs old, Relevant work experience

MAR 2019

12

GUIDELINES & QUALITY ASSURANCE DOCUMENTS  Code of Practice for Programme Accreditation (COPPA – 2nd edition 2017)  Code of Practice for Institutional Audit (COPIA – 2nd edition 2009)

 Code of Practice for Open and Distance Learning (COP-ODL)  Programme Standards (22)

 Standards (4)  Guidelines to Good Practices (11)  Malaysia Qualification Framework (MQF – 2nd edition 2018) + Malaysia Qualification Register (MQR) LAN (1996) – certification/accreditation of academic programmes in private IHL QAD-MOE (2002) – quality assurance system in public IHL OBE@RETI MQF (2007) – LAN and QAD dissolved

MAR 2019

OBE AWARENESS

OBE IMPLEMENTATION

13

2019 2015

2016

Outcomes 1999

2003

Fully prescriptive with qualitative assessment OBE@RETI

2006 (Rev. 2007)

2019

STANDARDS

2012

Direct monitoring of outcomes (cohort & individual)

2017

MAR 2019

14

MQA CODE OF PRACTICE : 9 ASPECTS /CRITERIA - STANDARDS FOR HEP (IHL)

Vision, mission, educational goals and learning outcomes

Curriculum Design & Delivery

Assessment of Students

Student Selection & Support Services

Academic Staff

Educational Resources

Programme Monitoring & Review

Leadership, Governance & Administration

Continual Quality Improvement

OBE@RETI

MAR 2019

8 MQF LO DOMAINS 15

5 CLUSTERS MQF edition 2, 2018 CLUSTER 1 Knowledge & Understanding

1.Knowledge

CLUSTER 2 Cognitive Skills

Practical Skills Social Skills and Responsibilities

CLUSTER 3 ** Functional & Work Skills

Ethics, Professionalism and Humanities

CLUSTER 4 Personal & Entrepreneurial Skills

Communication, Leadership and Team Skills

CLUSTER 5 Ethics & Professionalism

Scientific Methods, Critical Thinking and Problem Solving Skills Lifelong Learning and Information Management Skills OBE@RETI Entrepreneurship and Managerial Skills

CLUSTER 3 ** a. b. c. d.

Practical skills Interpersonal & Communication Skills Digital & Numeracy Skills Leadership, Autonomy & Responsibility

MQF 2nd edition considered ASEAN Qualifications Reference Framework (AQRF) MAR 2019

wef 1/4/2019 NP

LEVEL OF LEARNING OUTCOMES 16

• Educational Objectives/ Programme Objectives

Broad/Long Term (Few years after graduation – 3 to 5 years)

PO

• Programme Outcomes/ Programme Learning Outcomes

Upon graduation

CO

• Course Outcomes/ Course Learning outcomes/ Learning Objectives

Upon course completion

• Weekly/ Topic/ Lesson Outcomes

Upon weekly / topic completion

PEO

LO

OBE is an education system that emphasize on Learning Outcomes OBE@RETI

MAR 2019

example PO 17

DIPLOMA IN BUSINESS STUDIES

Upon completion of the programme, graduates should be able to

 PO1 - outline the relevant concepts and theories;  PO2 - apply practical and theoretical understanding to solve business problems;  PO3 - show entrepreneurial skills in addressing business issues;  PO4 - demonstrate teamwork, interpersonal, effective communication and social skills;  PO5 - display professional and ethical business conduct; and  PO6 - use the skills and principles of lifelong learning in their academic and career development. (FROM MQA STANDARD) OBE@RETI

MAR 2019

18

example PO BACHELOR DEGREE IN BUSINESS STUDIES Upon completion of the programme, graduates should be able to

 PO1 - evaluate theories and concepts in business studies;  PO2 - communicate creative and innovative ideas effectively;  PO3 - apply critical thinking skills for decision making;  PO4 - display innovative entrepreneurial skills;  PO5 - demonstrate leadership, teamwork, communication and social skills in accordance with ethical and legal practices; and  PO6 - apply the skills and principles of lifelong learning in their academic and career development.

(FROM MQA STANDARD) OBE@RETI

MAR 2019

19

example PO DIPLOMA IN HOSPITALITY & TOURISM Upon completion of the programme, graduates should be able to

 PO1 - demonstrate technical knowledge and associated hands-on skills in the area of specialisation;  PO2 - demonstrate creativity and entrepreneurship in operational issues;  PO3 - demonstrate supervisory ability, teamwork, interpersonal, and social skills;  PO4 - communicate effectively and solve operational problems;  PO5 - use information from multiple sources;  PO6 - demonstrate professionalism in accordance with ethical and legal practices; and  PO7 - nurture intellectual and professional growth through lifelong learning activities. OBE@RETI

(FROM MQA STANDARD)

MAR 2019

20

example PO BACHELOR DEGREE IN HOSPITALITY & TOURISM Upon completion of the programme, graduates should be able to

 PO1 - apply in-depth knowledge and skills in the area of specialisation;  PO2 - communicate ideas and information effectively;  PO3 - seek and analyse information for decision making;  PO4 - display leadership and entrepreneurship qualities ethically;  PO5 - demonstrate creativity and innovativeness in operational issues;  PO6 - demonstrate managerial attributes, teamwork, interpersonal, and social skills;  PO7 - apply professionalism in accordance with ethical and legal practices; and  PO8 - nurture intellectual and professional growth through lifelong learning activities. OBE@RETI

(FROM MQA STANDARD)

MAR 2019

example PO

21

PhD BY MIXED MODE & RESEARCH (INFO. SC.) Upon completion of the programme, graduates should be able to

 PO1 - demonstrate a systematic comprehension and in-depth understanding of the discipline;  PO2 - demonstrate a mastery of research methods related to the field of information science and other related fields;  PO3 - critically analyse, evaluate and synthesise new and complex ideas;  PO4 - show scholarly capabilities to generate, design, implement and adopt the integral part of the research process;  PO5 - contribute to original research that broadens the boundary of knowledge through an in-depth study, which has been presented and defended according to international standards, including writing in refereed publications;  PO6 - communicate with peers, scholarly communities and society at large through the preparation, publication and presentation of scholarly materials;  PO7 - promote technological, social and cultural progress in a knowledge based society in both the academic and professional contexts;  PO8 - demonstrate behaviour that is consistent with codes of professional ethics, legal requirements and responsibility;  PO9 - display leadership qualities through communicating and working effectively with peers and stakeholders;  PO10 - manage and advocate information for lifelong learning;  PO11 - generate solutions to problems using scientific and critical thinking skills; and MAR 2019 OBE@RETI  PO12 - apply values, ethics, morality and professionalism in the pursuit of their goals

(FROM MQA STANDARD)

IEA ACCORDS (EAC & ETAC)

example PO

22

WASHINGTON ACCORD (WA) ENGINEERS

SYDNEY ACCORD (SA) TECHNOLOGISTS

PO

DUBLIN ACCORD (DA) TECHNICIANS

GRADUATE ATTRIBUTES OBE@RETI

MAR 2019

Graduate Attributes

Characteristics

SK

SP/ TA

Apply knowledge of mathematics, natural science, engineering fundamentals and an engineering specialization to defined and applied engineering procedures, processes, systems or methodologies.

SK1, SK2, SK3, SK4

SP

Identify, formulate, research literature and analyse broadly-defined engineering problems reaching substantiated conclusions using analytical tools appropriate to the discipline or area of specialisation.

SK1, SK2, SK3, SK4

SP

Design solutions for broadly- defined engineering technology problems and contribute to the design of systems, components or processes to meet specified needs with appropriate consideration for public health and safety, cultural, societal, and environmental considerations

SK5

SP

Investigation

Conduct investigations of broadly-defined problems; locate, search and select relevant data from codes, data bases and literature, design and conduct experiments to provide valid conclusions.

SK8

SP

Modern Tool Usage

Select and apply appropriate techniques, resources, and modern engineering and IT tools, including prediction and modelling, to broadlydefined engineering problems, with an understanding of the limitations.

SK6

SP

23

Engineering Knowledge Problem Analysis

Design/ Development of Solutions

Demonstrate understanding of the societal, health, safety, legal and cultural

The EngineerOBE@RETI issues and the consequent responsibilities relevant to engineering and Society technology practice and solutions to broadly defined engineering problems.

SK7

SP

MAR 2019

Graduate Attributes

Characteristics

24

SK

SP/ TA

Environment and Sustainability

Understand and evaluate the sustainability and impact of engineering technology work in the solution of broadly defined engineering problems in societal and environmental contexts

SK7

SP

Ethics

Understand and commit to professional ethics and responsibilities and norms of engineering technology practice.

SK7

-

Function effectively as an individual, and as a member or leader in diverse teams.

-

-

Communicate effectively on broadly defined engineering activities with the engineering community and with society at large, by being able to comprehend and write effective reports and design documentation, make effective presentations, and give and receive clear instructions.

-

TA

Demonstrate knowledge and understanding of engineering management principles and apply these to one’s own work, as a member or leader in a team and to manage projects in multidisciplinary environments.

-

-

Individual and Team work Communication

Project Management and Finance OBE@RETI

Lifelong learning

Recognize the need for, and have the ability to engage in independent and life-long learning in specialist technologies.

-

MAR 2019

-

25

Graduate Attributes Characteristics Engineering Knowledge Problem Analysis

Design/ Development of Solutions

DK

DP/NA

Apply knowledge of mathematics, natural science, engineering fundamentals and an engineering specialization to wide practical procedures and practices.

DK1, DK2, DK3, DK4

DP

Identify and analyse well-defined engineering problems reaching substantiated conclusions using codified methods of analysis specific to their field of activity.

DK1, DK2, DK3, DK4

DP

DK5

DP

Design solutions for well-defined technical problems and assist with the design of systems, components or processes to meet specified needs with appropriate consideration for public health and safety, cultural, societal, and environmental considerations.

Investigation

Conduct investigations of well-defined problems; locate and search relevant codes and catalogues, conduct standard tests and measurements.

Modern Tool Usage

Apply appropriate techniques, resources, and modern engineering and IT tools to well defined engineering problems, with an awareness of the limitations.

Demonstrate knowledge of the societal, health, safety, legal and cultural The EngineerOBE@RETI issues and the consequent responsibilities relevant to engineering and Society technician practice and solutions to well defined engineering problems.

DP DK6 DK7

DP

DP

MAR 2019

26

Graduate Attributes Characteristics

DK

DP/NA

Environment and Sustainability

Understand and evaluate the sustainability and impact of engineering technician work in the solution of well defined engineering problems in societal and environmental contexts.

DK7

DP

Ethics

Understand and commit to professional ethics and responsibilities and norms of technician practice

DK7

-

Function effectively as an individual, and as a member in diverse technical teams.

-

-

Communicate effectively on well-defined engineering activities with the engineering community and with society at large, by being able to comprehend the work of others, document their own work, and give and receive clear instructions

-

NA

Demonstrate knowledge and understanding of engineering management principles and apply these to one’s own work, as a member or leader in a technical team and to manage projects in multidisciplinary environments

-

-

Individual and Team work Communication

Project Management and Finance OBE@RETI

Lifelong learning

Recognize the need for, and have the ability to engage in independent updating in the context of specialized technical knowledge

-

MAR 2019

-

12 PO Trays

27

Engineering Knowledge

Ethics

Individual and Team work

Problem Analysis

Environment and Sustainability

Communication

Design/ Development of Solutions

The Engineer and Society

Project Management and Finance

Investigation

Modern Tool Usage

Lifelong learning

OBE@RETI

MAR 2019

28

BEM : EAC MANUAL 2017 & ETAC STANDARDS 2019 Academic Curriculum (Criteria 1) Quality Management Systems (QMS) (Criteria 5)

Facilities (Criteria 4) OBE@RETI

Students (Criteria 2)

OBE Academic & Support Staff (Criteria 3)

MAR 2019

The Shift From Inputs to Outcomes

29

Traditional TEACHER-CENTERED

Outcome-Based Education STUDENT-CENTERED

Quality in terms of input: Quality in terms of • Intention & Efforts outputs : • Institutions & Services • Goals and Ends • Resources & • Products & Results Spending • Outcomes & Effects OBE@RETI

MAR 2019

Outcome-based education means starting with a clear picture of what is important for students to be able to do, then organising the curriculum, instruction, and assessment to make sure that this learning ultimately happens. (Spady, 1994)

statements to explicitly inform what student is expected to be able to know or do

Focus on OUTCOMES

Learning Outcomes activities which help the student to achieve these OBE@RETI outcomes

Teaching and Learning activities

to which extent the student meets these outcomes through the use of explicit assessment criteria

Assessment of the Learning Outcomes MAR 2019

Student enjoy physical education class

TC/SC

versus Student can run 50 meters in less than 1 minute

Student write a good report

TC/SC

versus Student analyse the social context of World War II in an essay.

Student understand the Principle of Archimedes

TC/SC

versus

TRANSPARENT & EVIDENT BASED

31

LO : TRADITIONAL vs OBE

example LO

Student apply the Principle of Archimedes in buoyancy of a ship OBE@RETI

MAR 2019

Kul Sharif Mosque (White Mosque) Kazan Kremlin, Russia, 16th century 32 (rebuilt 1996-2005)

OBE@RETI

B R E A K T I M E

Saint Basil’s Cathedral, Red Square Moscow (built 1555-1561) since 1991 as State Historical Museum

MAR 2019

OUTCOMES REVIEW AND REALIGN CURRICULUM WITH PROGRAMME OUTCOME

PEO STATEMENTS PO STATEMENTS PEO-PO MATRIX (LINK) C:P:A RATIOS

34

OBE@RETI

MAR 2019

LEVEL OF LEARNING OUTCOMES 35

• Educational Objectives/ Programme Objectives

Broad/Long Term (Few years after graduation – 3 to 5 years)

PO

• Programme Outcomes/ Programme Learning Outcomes

Upon graduation

CO

• Course Outcomes/ Course Learning outcomes/ Learning Objectives

Upon course completion

• Weekly/ Topic/ Lesson Outcomes

Upon weekly / topic completion

PEO

LO

OBE is an education system that emphasize on Learning Outcomes OBE@RETI

MAR 2019

36

OBE CURRICULA OUTCOMES

Have programme educational objectives, programme outcomes, course outcomes and performance indicators/ criteria or rubrics

PEO,PO, CO,LO

CONSTRUCTIVE ALIGNMENT Stated objectives and outcomes can be assessed and evaluated

Centered around the needs of the students and the stakeholders

STUDENT CENTRED OBE@RETI

C/P/A Programme outcomes address Knowledge, Skill & Attitude (C,P,A) to be attained by students

Course outcomes must satisfy the stated programme outcomes. There is no need for ANY (individual) course to address all programme outcomes.

Teaching/ Learning method may have to be integrated to include different delivery methods to complement the traditional LECTURING method.

ACTIVE LEARNING

CQI Learning outcomes are intentional and assessed using suitable performance indicators

MAR 2019

Learning outcomes focus on what the learner has achieved rather than the intentions of the teacher.

LEARNING OUTCOMES

37

Learning outcomes focus on what the learner can demonstrate at the end of a learning activity

 defines the type and depth of learning students are expected to achieve  provides an objective benchmark for formative, summative, and prior learning assessment  clearly communicates expectations to learners  clearly communicates graduates’ skills to the stakeholders  guides and organizes the instructor and the learner  guides the selection/design of appropriate assessments  allows educators to match teaching strategies to stated outcomes  allows us to assess the impact of instruction  acts as a template for course design OBE@RETI

MAR 2019

WRITING LEARNING OUTCOME  It is always expressed in terms of the learner.  It is precise and supports only one interpretation.  It describes an observable behavior

 It specifies conditions under which the behavior is performed  It specifies criteria for accomplishment  First, clarify what you are trying to assess.  Second, the outcome must be measurable.

 Third, the intended outcome must measure something useful and meaningful OBE@RETI

MAR 2019

COMPONENTS

CHARACTERISTICS

38

39

Specific – to one learning behavior Measurable - includes the criteria for success Actionable - can be demonstrated with evidence, or observed

Relevant – to future experiences or work place Timed – to be achieved within certain time constraints

LO HAS TO BE OBE@RETI

SMART MAR 2019

PROGRAMME EDUCATIONAL OBJECTIVES 40

MISSION / VISION STATEMENT

UNIVERSITY / FACULTY MISSION & VISION

PEO STATEMENTS

PEO MEASUREMENT OBE@RETI

1.What’s constitute behind the statement? 2.What are the indicators to measure the achievement of the Mission / Vision?

PEO STATEMENTS 1.How does PEO relates to Mission / Vision? 2.Who are the stake holders? 3.Where do you foresee your students 5 years after graduation? 4.What are their roles in organization and community? 5.How do they contribute to the achievement of mission / vision? PEO MEASUREMENT 1.What are the indicators that show the PEO are being achieved? 2.From whom should we measure the achievement. 3.Where and When to measure? 4.How (Method / Tool) to measure? 5.What are you going to do with the data collected?

MAR 2019

41

PEO PEO 1

EXAMPLE OF PEO Description Graduates are competent, innovative and entrepreneurial in acquiring and applying knowledge towards solving complex civil engineering problems.

PEO 2

Graduates possess leadership qualities, able to work, manage in diverse teams and serve the society in multi-disciplinary environment.

PEO 3

Graduates demonstrate professionalism and uphold ethical values with emphasis on sustainable environment

PEO 4

Graduates are able to communicate effectively, possess strong selfconfidence and recognise the need for life-long learning OBE@RETI

MAR 2019

42

PROCESS TO DEVELOP PEO

PEO Statements  Defined, measurable and achievable.  Linked to Programme Outcomes  Have own niche  Detailed out and documented  Published  Consistent and linked to mission & vision of IHLs and stakeholder needs  Linked to curriculum design  Reviewed and updated OBE@RETI

Processes & Results  Established process for formulating PEOs  Established process for assessing achievement of PEOs  Established process for evaluating achievement of PEOs  Performance target of the PEOs is achieved  Evaluation results are used in the CQI of the programme

Stakeholder  High degree of involvement in defining PEO statements  High degree of involvement in assessing the achievement of PEO  High degree of involvement in assessing improvement cycles (CQI)  Involved in strategic partnership MAR 2019

EAC Manual 2017, BEM

43

PROGRAMME OUTCOMES Statement of necessary ability / attribute / quality (Knowledge, Skills and Attitude) students need to have upon graduation (upon finishing their study) in order to prepare them to be who there are in future. In developing PO, the requirements / inputs from the stake holders must be taken into account including the requirement from accreditation authority (Malaysia – MQA, BEM, MMC and others)

OBE@RETI

MAR 2019

PROGRAMME OUTCOMES

44

PO DEVELOPMENT

PO STATEMENTS

PO STATEMENTS

1.Knowing exactly the expectation that graduate should achieved in each PO 2.Use appropriate verb 3.Identify domain and highest taxonomy level to be achieved. 4.Mapping To PEO 5.PO Distribution – Courses Mapping

PO MEASUREMENT & DELIVERY PO MEASUREMENT & DELIVERY OBE@RETI

1.What, when and where to measure PO 2.How to measure PO 3.What, when, where and how to deliver PO (T&L activities) 4.How to verify the result of PO achievement(Triangulation)

MAR 2019

45

PROCESS TO DEVELOP PO

PO Statements      

   

Covers EAC’s POs Linked to PEOs. Have own wordings. Have own niche. Defined, measurable and achievable. Detailed out and documented. Published. Consistent and tied to Programme Objectives. Outcomes in line with national needs. Reviewed and updated.

OBE@RETI

Processes & Results  Processes for all elements of criteria are quantitatively/qualitatively understood and controlled.  Processes are clearly linked to mission, PEOs, and stakeholder needs.  Systematic evaluation and process improvement in place.  CQI involved support areas.  Processes are deployed throughout the programme, faculty, and IHLs  Sound and highly integrated system.  Common sources of problems understood and eliminated.  Sustained results.  Results clearly caused by systematic approach.  Assessments of each of the 12 EAC’s POs are obtained by explicit direct assessment and not by inference.

Stakeholder  High degree of involvement in defining PO statements.  High degree of involvement in assessing the achievement of PO.  High degree of involvement in assessing improvement cycles (CQI).  Involved in strategic partnership.

MAR 2019

EAC Manual 2017, BEM

46

DISTRIBUTING POs

TOP DOWN 1.PO distribution is determined / dictated by the management team / committee members and the course owners are instructed to implement without argument. OBE@RETI

BOTTOM UP

MIX

Done from the bottom, in which the course owner identify which PO that he/she will be addressing in his/her course. The information is then collected and map to the programme structure. The management has no or very little say.

Its started with course owner (Bottom Up) identifying the POs to be implemented in his/her course. The data is the collected by the management to prepare the draft mapping. Once the draft mapping emerged, the process of balancing / bargaining is carried out to reach a consensus POs distribution to be implemented. MAR 2019

12 PO Trays

47

Engineering Knowledge

Ethics

Individual and Team work

Problem Analysis

Environment and Sustainability

Communication

Design/ Development of Solutions

The Engineer and Society

Project Management and Finance

Investigation

Modern Tool Usage

Lifelong learning

OBE@RETI

MAR 2019

48

EXAMPLE OF PO (EAC) WK

WP/ EA

Apply mathematics, natural science, engineering fundamentals and engineering specialization to the solution of complex engineering problems

WK1, WK2, WK3, WK4

WP

Problem Analysis

Identify, formulate, research literature & analyse complex engineering problems using first principles of mathematics, natural sciences and engineering sciences

WK1, WK2, WK3, WK4

WP

Design/ Development of Solutions

Design solutions for complex engineering problems and design systems, components or processes with appropriate consideration for public health and safety, cultural, societal, and environmental considerations.

WK5

WP

Investigation

Conduct investigations of complex problems using research-based knowledge and research methods

WK8

WP

Modern Tool Usage

Create, select and apply modern engineering and IT tools including prediction and modelling to complex engineering problems

WK6

WP

Characteristics Engineering Knowledge

Apply reasoning to assess societal, health, safety, legal and cultural The EngineerOBE@RETI issues and the consequent responsibilities relevant to professional and Society engineering practice and solutions to complex engineering problems

WK7

MAR 2019

WP

IEA Graduate Attributes 2013

49

EXAMPLE OF PO (EAC) WK

WP/ EA

Understand and evaluate the sustainability and impact of professional engineering work in the solution of complex engineering problems

WK7

WP

Apply ethical principles and commit to professional ethics and responsibilities and norms of engineering practice.

WK7

-

Function effectively as an individual, member or leader in diverse teams and in multi-disciplinary settings

-

-

Communicate effectively on complex engineering activities with the engineering community and with society able to comprehend, write, present, give and receive instructions

-

EA

Demonstrate knowledge and understanding of engineering management principles and economic decision-making, apply to own work, as a member and leader in a team, manage projects and in multidisciplinary environments

-

-

Characteristics Environment and Sustainability Ethics Individual and Team work Communication

Project Management and Finance OBE@RETI

Lifelong learning

MAR 2019

Recognize the need, prepare and engage in independent and life-long learning IEA Graduate Attributes 2013

50

PO ASSESSMENT MODELS

1

• ACCUMULATED MODEL – ALL courses contributing to the PO measurements

2

• DOMINATING MODEL – SELECTED courses contributing to the PO measurements, normally accounted in several CORE courses.

3

• CULMINATING MODEL – SELECTED FEW usually between 3 -5 courses contributing to the PO measurements, normally conducted during the final year of study. OBE@RETI

MAR 2019

PO1

How many courses contributed to PO1? Which assessment tools contributed to PO1?

Engr Activities

Complex Problems Solving

Level of difficulty

51

12 PO Trays

A-Culminating-Enabling B-Accumulated/Average C-Dominant

C P %C/P %C/P/A A TRUE ATTAINMENT OBE@RETI

Explicit Implicit Direct Indirect

PO12

MAR 2019

PO1 PO2 PO3 PO4 PO5 PO6 PO7 PO8 PO9 PO10 PO11 PO12

4 EC400 COURSE D OBE@RETI

5 EC500 COURSE E DEPTH OF KNOWLEDGE

COURSE COURSE CODE NAME

Engr~WP,EA,WK

1 EC100 COURSE A

2 EC200 COURSE B

3 EC300 COURSE C

MAR 2019

COMPREHENSION

COMPREHENSION

COMPREHENSION

ENGINEERING PRACTICE

RESEARCH LITERATURE

PO2

ENGINEERING DESIGN

SPEACIALIST KNOWLEDGE

PO1

ENGINEERING FUNDAMENTAL

COMPLEX ACTIVITIES

MATHEMATIC

NATURAL SCIENCES

SPEACIALIST KNOWLEDGE

ENGINEERING FUNDAMENTAL

WP2 WP3 WP4 WP5 WP6 WP7 EA1 EA2 EA3 EA4 EA5

MATHEMATIC

NATURAL SCIENCES

FAMILIARITY OF ISSUES

CONSEQUENCES TO SOCIETY & ENVIRONMENT

INNOVATION

COMPLEX PROBLEM SOLVING (WP)

LEVEL OF INTERACTIONS

RANGE OF RESOURCES

INTERDEPENDENCE

EXTEND OF STAKEHOLDERS INVOLVEMENT & CONFLICTING REQUIREMENT

EXTENSIVE APPLICABLE CODES

FAMILIARITY OF ISSUES

DEPTH OF ANALYSIS REQUIRED

WP1

RANGE OF CONFLICTING REQUIREMENT

WK8

WK6

52

WK5

WK4

NO WK3

EXAMPLE OF CO-PO MAPPING KNOWLEDGE PROFILE PO3 PO4 PO5 PO6 PO7 PO8

WK1 WK2 WK3 WK4 WK1 WK2 WK3 WK4 WK5 WK8 WK6 WK7 WK7 WK7

53

YR Y1

Course C/P/A C1A

EXAMPLE OF CO-PO MATRIX PO1 PO2 PO3 PO4 PO5 PO6 PO7 PO8 PO9 PO10 PO11 PO12 C C C P P C C C A A C C X X Establish threading courses or lineage of prerequisites

Y1

C1B

X

X

Y2

C2A

X

Y2

C2B

X

Y3 Y3 Y4 Y4

C3A C3B C4A C4B TOTAL

X X X X 8

X

OBE@RETI

3

Be informed of taxonomy domain & it’s appropriate level

X X X X X 4

X 2

X X X X X 4

X 1

X

X 1

X 1

X 1

X 2

MAR 2019

1

X 1

CO At the end of the semester Courses OBE@RETI

Courses

Courses

Courses

Courses

Courses

Courses

Courses

MAR 2019

Courses

PO12

PO11

PO10

PO9

PO8

PO7

PO6

PO5

PO4

PO3

PO2

PO1

PO At the time of Graduation

Courses

Courses

Courses

54

PEO 3 to 5 years after graduation

What if we have the following … 55 Q1 (12.5%) Final Exam 50%

Q2 (12.5%) Q3 (12.5%)

CO1

Q4 (12.5%)

CO2 Mid-Term Test

Q1 (10%)

20%

Q2 (10%)

Assignments

A1 (10%)

20%

A2 (10%)

PO3 CO3

CO4

PO4

Case Study 1 10%

OBE@RETI

This is now DIRECT and EXPLICIT !

MAR 2019

What if we have the following … 56

… use of relevant RUBRICS where applicable! OBE@RETI

Progress Report (20%)

CO8

Oral Prest. (10%)

CO7

Prototype Demo (10%)

Teamwork (10%)

Logbook / MoM (10%)

Proj. Mgt. (10%)

Environment & Sustainability (5%)

Project Report (60%)

CO6

CO5

CO4

CO3

Written Comm. (10%)

CO2

Design (20%)

Literature (5%)

Modern Tools Usage (10%)

CO1

Presentation (20%) MAR 2019

This is now DIRECT and EXPLICIT !

B R E A K

57

OBE@RETI

MAR 2019

T I M E

What are students able to do? C,P,A

How best to help students achieve it?

How to know what the students achieved?

Assessemnt

OBE – KEY QUESTIONS

58

How do you close the loop? CQI

Instruction OBE@RETI

MAR 2019

WHAT Bloom’s Taxonomy 59

Standard/EAC 2017

learning outcomes

ELEMENTS OF COURSE DESIGN

HOW TO HELP

CDIO

Active & cooperative learning

OBE@RETI

Classroom assessment techniques

assessment

instruction Other learning techniques

Formative, Summative, Course, Programme, Tools, Direct and Indirect

STUDENT

Lectures

Problem based learning

HOW TO KNOW

CLOSE LOOP

Instructional Technology

Labs

Course specific goals & objectives

Tests

Surveys Others

OBE requires T&L activities to be CONSTRUCTIVELY ALIGNED to intended learning outcomes

MAR 2019

FELDER & BRENT, JEE, JAN., 2003

60

Edgar Dale (1900-1985) faculty@Ohio State Univ (19291970) – renowned pioneer audio visual in instruction

OBE@RETI

MAR 2019

61

COURSE SYLLABUS Code/Name/Credit Hour/Sem/Perequisite Course Description Course Outcome CO1 : addressing PO?/(C/P/A)/Domain + level of difficulty CO2 : addressing PO?/(C/P/A)/Domain + level of difficulty

Course Content (Topics) – distribution of hrs (LO), SLT (F2F, NF2F) Course Assessment Text book/List of References OBE@RETI

MAR 2019

62

OBE@RETI

lower order

Intermediate

Higher order

MAR 2019

63

OBE@RETI

lower order

Intermediate

Higher order

MAR 2019

64

MAR 2019

OBE@RETI

lower order

Intermediate

Higher order

COURSE COVERAGE

C:P:A RATIO

65

Breadth of coverage is subjected to the required outcomes, (Cognitive (C) = 70%, Psychomotor (P) = 20%, Affective (A) = 10%) C

(70%)

P

(20%)

A

(10%)

12 34 56 56 12 34 12 56 56 56 56 34 34 12 34 56 34 56 12 34 567 34 567 12 34 5

Depth of coverage is subjected to the required level of outcomes OBE@RETI

COURSE ASSESSMENT

MAR 2019

WA3 : Design solutions for complex engineering problems and

1

Identify significance and nature of open-ended problems

2

Identify problem and constraints including health and safety, environmental and societal issues

3

Identify customer or user or client needs

4

Produce multiple potential solutions to meet functional specifications

5

Create and test simulations/models/prototypes

6

Assess design performance based on specification and requirements

OBE@RETI

WP3

WP4

WP5

WP6

WP7

DEPTH OF ANALYSIS

FAMILIARITY OF ISSUES

EXTENSIVE APPLICABLE CODES

EXTEND OF STAKEHOLDERS INVOLVEMENT & LEVEL OF CONFLICTING REQUIREMENTS

INTERDEPENDENCE

Some possible outcomes

WP2 RANGE OF CONFLICTING REQUIREMENTS

No.

WP1 DEPTH OF KNOWLEDGE

CO66

design systems, components or processes that meet specified needs with appropriate consideration for public health and safety, cultural, societal, and environmental considerations. (WK5)

MAR 2019

WA4 : Conduct investigation into complex problems using

1

Reviews the open research literature

2

Identifies the needs for research or investigation

3

Identifies appropriate research or investigation methodologies

4

Designs and executes valid forms of research, experimentation or measurement

5

Calibrates or validates the data collection methods and equipment

6

OBE@RETI Analyses the data including considering sources of error

7

Draws valid conclusions and justifies those conclusions

WP3

WP4

WP5

WP6

WP7

DEPTH OF ANALYSIS

FAMILIARITY OF ISSUES

EXTENSIVE APPLICABLE CODES

EXTEND OF STAKEHOLDERS INVOLVEMENT & LEVEL OF CONFLICTING REQUIREMENTS

INTERDEPENDENCE

Some possible outcomes

WP2 RANGE OF CONFLICTING REQUIREMENTS

No.

WP1 DEPTH OF KNOWLEDGE

CO67

research-based knowledge and research methods including design of experiments, analysis and interpretation of data, and synthesis of information to provide valid conclusion. (WK8)

MAR 2019

WA5 : Create, select and apply appropriate techniques, resources,

1

Select and apply appropriate instrumentation or measurement techniques or models or simulations.

2

Analyse limitations and errors of instrumentation or measurement techniques or models or simulations.

3

Evaluate appropriateness of results from instrumentation OBE@RETI or measurement techniques or model or simulations.

WP3

WP4

WP5

WP6

WP7

DEPTH OF ANALYSIS

FAMILIARITY OF ISSUES

EXTENSIVE APPLICABLE CODES

EXTEND OF STAKEHOLDERS INVOLVEMENT & LEVEL OF CONFLICTING REQUIREMENTS

INTERDEPENDENCE

Some possible outcomes

WP2 RANGE OF CONFLICTING REQUIREMENTS

No.

WP1 DEPTH OF KNOWLEDGE

and modern engineering and information technology tools, including CO68 prediction and modelling, to complex engineering problems, with an understanding of the limitations. (WK6)

MAR 2019

WA7 : Understand and evaluate the sustainability and impact of

CO69

professional engineering work in the solution of complex engineering problems in societal and environmental contexts.

1

Identifies both direct and indirect, and short and long-term impacts on people and the environment

2

Identifies and justifies specific actions required for environmental protection in the event of failure

3

OBE@RETI Undertakes life-cycle analysis to determine the sustainability of any proposed outcomes

WP3

WP4

WP5

WP6

WP7

DEPTH OF ANALYSIS

FAMILIARITY OF ISSUES

EXTENSIVE APPLICABLE CODES

EXTEND OF STAKEHOLDERS INVOLVEMENT & LEVEL OF CONFLICTING REQUIREMENTS

INTERDEPENDENCE

Some possible outcomes

WP2 RANGE OF CONFLICTING REQUIREMENTS

No.

WP1 DEPTH OF KNOWLEDGE

(WK7)

MAR 2019

ENGINEERS IN SOCIETY (COURSE OUTCOMES)

a

Explain the Malaysian legal system (PO8/C2).

b

Explain the role of engineering professional bodies (PO6/C2).

c

Analyse the Code of Ethics and Professional Conduct for engineers (PO8/C4).

d

Explain the local and federal authorities regulation (PO6/C2).

e

Discuss Occupational Safety and Health OBE@RETI Act 1994 and FMM Act 1967 on construction in relation to the

PO7

PO8

PO9

PO10

PO11

PO12

Individual and Team work

Communication

Project Management and Finance

Lifelong Learning

investigation

PO6

Ethics

Design/ Development of Solutions

PO5

Environment and Sustainability

PO4

The Engineer and Society

PO3

Modern Tool Usage

PO2

Problem Analysis

At the end of this course, the students should be able to:-

PO1 Engineering Knowledge

EIS70

This course deals with interaction between engineers and society at large. Regulations and laws relating to civil engineering practice in Malaysia become pivotal, acquire central knowledge prior to working as a registered graduate engineer.

X provide clear and explicit Course Learning Outcomes (CO). COs must be mapped to the appropriate PO

X

X X X

MAR 2019

CQI CQI

CO

CQI

LO OBE@RETI

CQI

PO

PEO

TRANSPARENT & EVIDENT BASED

71

OBE CURRICULA

MAR 2019

OBE IMPLEMENTATION RANACO EDUCATION & TRAINING INSTITUTE 18-19 MARCH 2019

SITI HAWA HAMZAH

Phd, P.Eng, FIEM, Hon. MAFEO ASEAN Engr, MRM, PSWM Associate Director (Civil Engineering), Engineering Accreditation Department, BEM Malaysia [email protected]

SITI HAWA HAMZAH

PhD, P.Eng, FIEM, Hon MAFEO ASEAN Engr, MRM, PSWM Retired Professor (Civil & Structural Engineering), Faculty of Civil Engineering, UiTM, 40450 Shah Alam, MALAYSIA [email protected] Associate Director (Civil) AD, Associate Director (Structural) AD, Past Council Member IEM, Past Excomm IEM, EAD Lead Panel, P.Eng Principle Interviewer, BEM T&E Comm. Member

33 years teaching & academic experience 20 years experience managing engineering programmes Accreditation experience in more than 120 programmes Completed 21 funded research projects Authored 11 books in structural engineering, more than 165 technical papers, accreditation videos on You Tube External examiner to B. Eng Civil programmes at Monash University Malaysia, Universiti Malaysia Sarawak, INTI International University, Universiti Tun Hussein Onn Malaysia. EAC Advisor to Universiti Malaya OBE Advisor to Universiti Industri Selangor IEM Women Engineer Award Honorary Member to ASEAN Federation of Engineering Organisation Three times recipient of UiTM excellence service award More than 20 research accolades *IEM – The Institution of Engineers Malaysia, BEM – Board of Engineers Malaysia, AD – Engineering Accreditation Department, EAC – Engineering Accreditation Council, UiTM – Universiti Teknologi MARA,

75

WORKSHOP LEARNING OUTCOMES

At the end of the seminar, participants are able to; 1. Clearly explain the definition and purpose of OBE and the concept of constructive alignment in enhancing the effectiveness of HE system and the quality of graduates 2. Effectively involve in curriculum development process 3. Write the learning outcomes correctly to the intended students ability and level of taxonomy 4. Identify and adapt the appropriate learning and teaching methods that ensures the development and achievement of the intended learning outcomes 5. Plan and uses appropriate assessment methods that enable the learning outcome to be measured and analysed correctly 6. Contribute to CQI process in the effort of improving learning process and achievement of the learning outcomes of the students OBE@RETI

MAR 2019

CONSTRUCTIVE ALIGNMENT & CQI ASSESSMENT TOOLS DIFFERENTIATE ASSESSMENT TOOLS AS DIRECT AND INDIRECT MEASUREMENTS EVALUATE MEASUREMENTS DATA FOR CQI USE MEASUREMENTS TO TRIANGULATE FINDINGS

PROPOSE COURSE OF ACTIONS FOR CQI TRUE ATTAINMENT OF THE OUTCOMES OF ALL STUDENTS CLOSE THE LOOP

PEO – Programme Educational Objectives

3-5 yrs after graduation

PO – Programme Outcomes / Graduate Attributes

Upon graduation

CO – Course Outcomes / Course Learning Outcomes

Upon course completion

LO – Weekly/Topic Learning Outcomes

Upon weekly/topic completion

LEARNING OUTCOME written based on the highest expectation to be achieved by the students & ASSESSMENT to be aligned to learning outcomes

ASSESSMENT LEVELS

What the students should to be able to do as a result of teaching and learning?

78

learning outcomes CLOSE LOOP What TL activities to adopt in order to achieve the intended LO?

instruction

OBE@RETI

How do you ensure that they have achieved the learning outcomes ?

STUDENT

assessment

OBE requires T&L activities to be CONSTRUCTIVELY ALIGNED to intended learning outcomes

MAR 2019

FELDER & BRENT, JEE, JAN., 2003

CONSTRUCTIVE ALIGNMENT - T&L

79

OBE@RETI

TEACHING – Engaging the student in the verb of the CO

CO – What the student has to learn

ASSESSMENT – How well the student has achieved the CO MAR 2019

When & How to Assess? 80

Individual Individual improvement

Who to Assess?

Course programme improvement

Gatekeeping

Course programme evaluation

Assessment - process of

measuring and collecting the data (mark / score) that enable us to analyse the achievement of the intended learning outcomes and the effectiveness of the learning activities.

What to Assess? C/P/A

Programme/Group Learning / Teaching (Formative) OBE@RETI

Why Assess?

Accountability (Summative)

MAR 2019

PURPOSE OF ASESSMENT

Assessment – process that determines the level of attainment of the outcomes Improvement Accountability Accreditation – curriculum, – institution, – quality instructional public, assurance, practice, resource accrediting student provider bodies services OBE@RETI

MAR 2019 81

3 Interrelated & Complementary Approaches Assessment as Assessment for learning – students Assessment of learning – instructor use self assessment learning – assessing provide feedback student’s learning and instructor’s to students on feedback to against the their learning and acquire learning outcomes how to improve - SUMMATIVE knowledge FORMATIVE FORMATIVE OBE@RETI

MAR 2019 82

ASSESSMENT - a

How can an instructor know his students had learnt the learning outcome to the expected level of performance?

OBE@RETI

systematic process of looking at student achievement within and across courses by gathering, interpreting and using information about student learning for educational improvement (American Association of Higher Education (AAHE))

How did the students know what they have learnt?

MAR 2019 83

ASSESSMENT VS EVALUATION Assessment - a complex process requiring multiple methods of data collection and careful analysis, to inform both the instructor and the student as to how the course is progressing (formative) or how it has ended (summative). Evaluation - to judge the effectiveness of instructional methods, the course contents, and the achievement of course outcomes

OBE@RETI

MAR 2019 84

FORMATIVE & SUMMATIVE Formative Assessments - used to provide feedback to guide learning, helps students to form. FOR learning because educators use the results to modify and improve teaching techniques during an instructional period Summative Assessments - used to contribute marks toward final grades or for the determination of final grades. OF learning because educators evaluate academic achievement at the end of an instructional period. OBE@RETI

MAR 2019 85

86

Common Assessment Tools

1. 2. 3. 4. 5. 6.

FINAL EXAM TESTS ASSIGNMENTS PROJECT REPORTS CLASS PRESENTATIONS ????? OBE@RETI

MAR 2019

WHAT TO ASSESS COGNITIVE DOMAIN

PSYCHOMOTOR DOMAIN

AFFECTIVE DOMAIN

• Knowledge outcomes core concepts and material knowledge of a particular discipline; intellectual/mental ability • Knowledge, Comprehension, Application, Analysis, Synthesis, Evaluation

• Behavioral and Skills outcomes - what a student can do; physical ability; behavior crucial to the curriculum’s impact • Perception, Set, guided Response, Mechanism, Complete Overt Response, Adaption, Organisation

• Attitudes and values outcomes - those educators believe to be important; habits of mind • Receiving, Responding, Valuing, Organisation, Internalising

OBE@RETI

MAR 2019 87

Bloom’s Domain (1956)

ASSESSMENT METHODS AND TOOLS Range of assessment produce valid decisions

methods

Assessment tool consist of instruments & instructions to gather & interpret evidence

Assessment instruments are tasks

administered to the students with criteria used to judge quality of performance OBE@RETI

MAR 2019 88

Activity 1 Produce a list of assessment methods used in your PROGRAMME Produce a list of assessment methods used in your COURSE

OBE@RETI

MAR 2019 89

Activity 2 From Activity 1, identify whether the assessment method is FORMATIVE or SUMMATIVE or BOTH.

OBE@RETI

MAR 2019 90

Activity 3 Think-Pair-Share – Design an assessment tool to measure a. COGNITIVE DOMAIN b. PSYCHOMOTOR DOMAIN OBE@RETI

MAR 2019 91

Types of assessment

ASSESSMENTS LEVELS PEO-PO-CO-LO Institutional assessment (PEO) Curricular and programme assessment (PO) Course and learner-centered assessments (CO) OBE@RETI

Weekly or topic outcomes (LO)

MAR 2019 92

PEO ASSESSMENT To establish graduates career pathway aligned to the qualification acquired and intended educational objectives attained

EMPLOYER SURVEY INDIRECT

ALUMNI SURVEY

PEO ASSESSMENT

AFTER GRADUATION OBE@RETI

PROFESSIONAL MEMBERSHIP DIRECT

GARDUATES CAREER DATA

MAR 2019 93

Assessment methods valid to measure the PEO achievements :

Employer Survey, Alumni Survey, Professional Membership, Graduates Career Data, Professional Advancement OBE@RETI

MAR 2019 94

PO ASSESSMENT 1

• Accumulated model – ALL courses contributing to the PO measurements

2

• Dominating model – SELECTED courses contributing to the PO measurements, normally accounted in several CORE courses.

3

• Culminating model – SELECTED FEW usually between 35 courses contributing to the PO measurements, normally conducted during the final year of study. OBE@RETI

MAR 2019 95

OBE@RETI

TRUE ATTAINMENT

UPON GRADUATION

PO attainment can be collected from few to several courses

TRIANGULATION important to ensure TRUE attainment – EE Report, Moderation of Examination Paper, Stakeholder Feedbacks

MAR 2019 96

Assessment methods and tools valid to measure and triangulate PO attainments : External Examiner Report, Moderation of Final Examination Paper, Industry Advisory Panels Feedback, Benchmarking at Programme Level, Comprehensive Design Projects, Portfolios, Exit Exam/Interviews OBE@RETI

MAR 2019 97

PO ASSESSMENT

DIRECT

OBE@RETI

INDIRECT

• DIRECT examination or observation includes assignment, test, final exam, report & presentation where CO-PO measured directly and explicitly. • INDIRECT ascertained perceived value of learning experiences usually from survey or questionnaire MAR 2019 98

CO ASSESSMENT

FORMATIVE

OBE@RETI

SUMMATIVE

• Gauging students attainment of learning outcomes during and at the end of the semester

• Measurements contributed to the measurement attainments at the end of the course and programme MAR 2019 99

Formulate statements of LO

Discuss & use assessment results to improve learning

OBE@RETI

EFFECTIVE ASSESSMENT Create experiences leading to outcomes

Develop & Select assessment measures

MAR 2019 100

HUBA & FREED (2000)

101

FULL CYCLE

Programme Educational Objective

CQI Actions

CQI heart of OBE

OBE@RETI

PEO Assessment Methods

PO setting and T&L activities CO setting and T&L activities

CQI actions

every semester, year, full cycle

CQI LOOPS

CQI actions

PO assessment methods CO assessment methods

Data Analysis

Data Analysis

Data Analysis

every semester MAR 2019

FOUNDATION OF CQI IS ASSESSMENT

 exists only when it is done systematically  dynamic behavior of an organization  does NOT result from external factors, comes from within an institution  can’t be achieved in isolation  exists when the continuous pursuit of excellence motivates and guides the philosophies, planning, policies and processes of the organization  focus of CQI in education is NOT on the curriculum  requires integration of defined objectives, performance metrics, & regular assessment  cyclical, assessment of performance is OBE@RETI the baseline for future assessment

FACTS ON CQI

102

 essential to synchronize educational objectives, mission of IHL and needs of stakeholders to achieve CQI  CQI can be proven through well documented report and data.  success is achieved through meeting the needs of those we serve.  most problems are found in processes. CQI does not seek to blame, but rather to improve processes.  possible to achieve continual improvement through small, incremental changes using the scientific method  most effective when it becomes a natural part of the way everyday work is MAR 2019 done.

Know the assessment levels (PEO/PO/CO/LO)

CQI

In summary Choose reliable assessments – Direct vs Indirect Formative vs summative

T&L activities – Constructive Alignment (C/P/A)

104

OBE@RETI

2 0 1 9

MAR 2019

INDUSTRIAL TRAINING - PO ATTAINMENT & TRIANGULATION Knowledge Sharing on Accreditation Tips : Malaysian Experience SITI HAWA HAMZAH

PhD, P. Eng, FIEM, Hon MAFEO ASEAN Engr, MRM, PSWM [email protected]

LIEW CHIA PAO P. Eng, MIEM

[email protected]

106

TRAINING PROVIDERS ACT A1479 REA 1967 (REV 2015) ACT A1158 REA 1967 (REV 2015) Series Eight

ENGINEERING WORKS – all works which include any publicly or privately owned public utilities, buildings, machines, equipment, processes, works or projects that require the application of engineering principles and data. PROFESSIONAL ENGINEERING SERVICES – engineering services and advice in conjunction with any feasibility study, planning, survey, design, construction, commissioning, operation, maintenance and management of engineering works or projects and include any other engineering services approved by the Board. August 2018

107

EAC MANUAL 2017 has made INDUSTRIAL TRAINING as a QUALIFYING REQUIREMENT prior to any accreditation visit can be conducted MINIMUM DURATION of 8 WEEKS CONTINUOUS TRAINING

ONE (1) CREDIT is allocated for EVERY TWO (2) WEEKS of TRAINING subjected to a MAXIMUM OF SIX (6) CREDITS THE TRAINING SHALL BE ADEQUATELY STRUCTURED, SUPERVISED and RECORDED IN LOG BOOKS/REPORT IT MUST BE CONDUCTED BEFORE THE FINAL SEMESTER Series Eight

August 2018

108

PO1

PO2

PO3

PO4

PO5

PO6

PO11

PO12

Which PO? PO7 Series Eight

PO8

PO9

PO10

August 2018

109

Noyes et al. (2011) performed an analysis with data from 9,870 undergraduate students enrolled in six engineering majors at a Carnegie Research University (very high research activity) with an extensive history of student involvement in cooperative education (co-op). Some of the findings include: • Experiential learning obtained through co-op is a superior method of inculcating communication skills (both oral and written) as well as teamwork skills. • Courses that include “soft skills” are more likely to demonstrate a co-op effect than courses that stress hard skills only.

Other Soft Skills

Key Skills PO10

PO9

PO6

PO7

PO8

PO12

Communication

Teamwork

Engineer & Society

Environment & Sustainability

Ethics

Lifelong Learning

Baber and Fortenberry (2008) reported that the following outcomes can be strengthen by co-op:

110

PO3

PO2

Design or Development

PO6 Engineer & Society

Problem Analysis

Ethics

PO9

PO1

Engineering Knowledge

PO8

PO10

Teamwork

Communication Series Eight

August 2018

The attributes of complex engineering activities, some of which might reasonably be encountered by a professional engineering undergraduate (eg. during 111 capstone design or a period of industry experience) (IEA, 2014) Range of resources

EA1: Involve the use of diverse resources (people, money, equipment, materials, information and technologies).

Level of interactions

EA2: Require resolution of significant problems arising from interactions between wide-ranging or conflicting technical, engineering or other issues.

Innovation

EA3: Involve creative use of engineering principles and research based knowledge in novel ways.

Consequences to society and the environment

EA4: Have significant consequences in a range of contexts, characterised by difficulty of prediction and mitigation.

Familiarity

Series Eight

August 2018 EA5: Can extend beyond previous experience by applying principles-based approaches.

112

POs THAT COULD BE ASSESSED IMPORTANCE

PO10 Communication

PO6

PO8

PO9

Engineer & Society

Ethics

Teamwork

PO X Series Eight

PO Y August 2018

113

PO10:COMMUNICATION ~ COMPLEX ENGINEERING ACTIVITIES

COMMUNICATE effectively on complex engineering activities with the engineering community and with society, able to comprehend, write, present, give and receive instructions Series Eight

WRITTEN

PO10

report, prototype, drawings, video ENGINEERING ACTIVITIES

VERBAL

presentation, viva, video

August 2018

114

EXAMPLE : Aligning LEARNING OUTCOMES & ASSESSMENT (COMPLEX ENGINEERING ACTIVITIES)

Learning Outcomes (Verbs)

QUESTION

PROPOSE DEFEND RELATE

SHARE EXPLAIN PRESENT

RECITE CHOOSE NAME Series Eight

PO10 Assessment Methods WRITTEN – Log book, Report, Video VERBAL – Presentation, Viva, Video

EA1 EA2 EA3

EA4 EA5

EA Range of resources Level of interactions Innovation Consequences to society and the environment Familiarity August 2018

EXAMPLE : Aligning LEARNING OUTCOMES & ASSESSMENT (AFFECTIVE)

HIGH BLOOM LEVEL

Series Eight

PROPOSE

DEFEND RELATE SHARE EXPLAIN

PRESENT RECITE CHOOSE

NAME

LOGBOOK INTERVIEW

QUESTIONNAIRE PRESENTATION ROLE PLAY OBSERVATIONS

Which PO?

Learning Outcomes (Verbs)

LOW BLOOM LEVEL

QUESTION

Assessment Methods

115

August 2018

116

REFERENCES

 EAC Manual 2017, Board of Engineers Malaysia  IEA Graduate Attributes and Professional Competency Profiles, Version 3: 21 June 2013  25 Years of the Washington Accord, International Engineering Alliance, June 2014  Noyes, C. R., Gordon, J. and Ludlum, J. (2011). The academic effects of cooperative education experiences: Does co-op make a difference in engineering coursework?. 2011 ASEE Annual Conference & Exposition  Baber, T. and N. Fortenberry (2008). The academic value of cooperative education: A literature review. 2008 ASEE Annual Conference & Exposition

POINTS FOR CLARIFICATIONS  [email protected]  [email protected] Series Eight

August 2018

117

Just sharing – worthy points to consider in designing RUBRICS OBE@RETI

MAR 2019

118

QQT

Quantitative  Quantitative learning focus on amount of a course a student learn where learning is about adding new pieces of information. Nature of learning outcome is quantitative & assessment focus on REPRODUCING knowledge.  Quantitative – related to number, amount or size, capable being measured OBE@RETI

Qualitative  Qualitative learning focus on CHANGING & DEVELOPING students understanding of a course & learning is a question of combining, relating & interpreting new material with what is already known. Nature of learning outcome is qualitative & assessment focus on the LEVEL of qualitative understanding of a course.  Qualitative – comparisons based on qualities, satisfaction levels

Taxonomy  Taxonomy – C, P, A (Blooms, 1956). Useful in planning & evaluation of teaching & assessment to investigate and evaluate student’s intellectual and ethical development  Check out the verbs

MAR 2019

RUBRICS

Why? Mechanism to judge quality work Clarity Diagnose Assessment for learning vis-a-vis Assessment of learning Reduce subjectivity OBE@RETI

Rating scale to provide good judgement towards the award of a grade

119

Where? Essay

Research paper Response item Performance assessments – debate, speech

Presentations – skills (psychomotor ) Procedures

++++ MAR 2019

TYPES OF RUBRICS

120

HOLISTIC – one general descriptor for performance as a whole RUBRIC HAS EVALUATIVE CRITERIA, INDICATORS & SCORING SCALES

ANALYTIC – multi dimensional corresponding to each independent dimension OBE@RETI

INDICATORS OF RUBRICS QUANTITATIVE QUALITATIVE PROXY INDICATORS

INDICES MAR 2019

121

EXAMPLE – HOLISTIC RUBRIC

SCORE

DESCRIPTION

5

Demonstrates complete understanding of the problem. All requirements of task are included in response

4

Demonstrates considerable understanding of the problem. All requirements of task are included

3

Demonstrates partial understanding of the problem. Most requirements of task are included

2

Demonstrates little understanding of the problem. Many requirements of task are missing

1 0

Demonstrates no understanding of the problem OBE@RETI

No response or task not attempted

MAR 2019

122

EXAMPLE - QQT RUBRICS QQT rubrics aid good assessment conduct because :

 it is measurable and effective,  provides true attainment of the course and programme outcomes,  shows students’ cognitive, psychomotor and affective skills development, and  fulfils OBE requirements (achievement must be measurable, evident and continually improved). OBE@RETI

MAR 2019

123

E

Marks %

S

40

C/Report/50 + P/lab or comp/30 + A/consultation/20

E1

20

C/Report/50 + (C+A)/viva/(25+25)

E2

20

C/Report/50 + (C+A)/viva/(25+25)

I1

10

A/communication & poster presentation/100

I2

10

A/communication & poster presentation/100

Distribution/Weightage %

Construct rubrics and performance criteria for each tool OBE@RETI

C – Cognitive P – Psychomotor A - Affective

ASSESSMENT TOOLS & WEIGHTAGE (FYP)

E

Marks %

Tools

C

100

report

P

100

lab or comp

A

100

consultation

C+A

50+50

A

100

viva communication & poster MAR 2019 presentation

124

OBE@RETI

EXAMPLE – ANALYTIC RUBRIC FOR REPORT WRITING

MAR 2019

125

OBE@RETI

EXAMPLE – ANALYTIC RUBRIC FOR OTHER ASSESSMENT

MAR 2019

12 PO Trays

MAR2019

How many courses contributed to PO1?

Which assessment tools contributed to PO1?

EA/TA/NA Engr Activities

C P A

WP/BD/WD Problems Solving

Level of difficulty

PO1

A-Culminating-Enabling B-Accumulated/Average C-Dominant

%C/P %C/P/A TRUE ATTAINMENT OBE@RETI

Explicit Implicit Direct Indirect

PO12

MAR2019

Engineering Knowledge

Ethics

Individual and Team work

Problem Analysis

Environment and Sustainability

Communication

Design/ Development of Solutions

The Engineer and Society

Project Management and Finance

Investigation

Modern Tool Usage

Lifelong learning

OBE@RETI

SUMMARY • Acquire in-depth knowledge the expectations of the Programme OUTCOMES; • Comprehend the requirements of OBE and the CONSTRUCTIVE ALGINMENT; • Conduct systematic assessments according to the TAXONOMY and LEARNING DOMAINS; • Verify the moderation process; and • Justify TRUE attainment of the Programme OUTCOMES

MAR2019

OBE@RETI

129

ADDITIONAL REFERENCES

 EAC Manual 2017, Board of Engineers Malaysia  Engineering Technician Education Programme Accreditation Standard (2019), Board of Engineers Malaysia;  Engineering Technology Programme Accreditation Standard (2019), Board of Engineers Malaysia;  http://www.mqa.gov.my/  IEA Graduate Attributes and Professional Competency Profiles, Version 3: 21 June 2013  Siti Hawa Hamzah, Knowledge Sharing on Accreditation Tips: Malaysian Experience, You Tube Channel, 2016 & 2018

OBE@RETI

MAR 2019

130

OBE@RETI

MAR 2019