Outcomes Assessment (OA)

– Assessing PILOs – a Process


1. Background

  • Under OBTL, learning outcomes at courses/activities, programmes and Graduate Attributes are aligned through curriculum mappings, in which the TLAs and AMs all match the CILOs; i.e. constructive alignment (Figure 1).
  • Therefore, assessments at course/activity, programme and institutional levels are not isolated, but rather, integrated.
  • The University’s QA Mechanism principally relies on the assessments at course or activity level to provide evidence in ascertaining how well the Programme Outcomes and Graduate Attributes have been achieved.

Figure 1. OBTL, Constructive Alignment & QA Mechanism



2. Definition and Purpose

  • Outcomes Assessment (OA) can be described as a systematic and ongoing method of gathering, analysing and using information from various sources about an education programme to ascertain how well the programme and institutional learning outcomes have been attained by students in that programme.
  • Definition of OA: Assessment of students’ achievement of the programme intended learning outcomes (PILOs), and by constructive alignment, the Graduate Attributes (GAs)(Figure 2).
  • This continuing exercise can help the University, Faculties and Programmes identify and solve the key issues in teaching and learning in the respective programmes, so as to continuously enhance the overall quality of the programmes to improve the learning and development of our students.

Figure 2. Curriculum Maps



3. Planning and Actual Assessment

  • Based on the curriculum maps of GAs – PILOs – courses/CILOs, the Programme Director decides which PILO/GA will be evaluated each semester, and completes the PILOs Assessment Review Cycle (Form 1)(Figure 3).

    Figure 3. PILOs Assessment Review Cycle

  • Appropriate assessment methods and measures, direct and indirect, should be considered and provided in the PILOs Assessment Plan (Form 2).

- Direct measures – the direct assessment of student works, performance or behaviours, including course embedded assessments (CEA: coursework and course examination), projects, designs/creative works, University Academic Test, external tests (e.g. IELTS), etc.

- Indirect measures – the evaluation of student learning experiences or stakeholders’ perceptions of student learning outcomes, including Formative Review Exercise (FRE), graduate/alumni surveys (or interviews), employer surveys (or interviews), external examiners’ comments, etc.

。Measuring tools for CEA are the 17 GA rubrics recommended. Programmes can generate their own rubrics for their PILOs based on these 15 GA Rubrics for generic skills, plus one for discipline-based knowledge and one for general knowledge, so 17 GA Rubrics in total.

The 17 GA Rubrics mentioned in the previous section are already setup in Bb Learn. These can be imported into your own Bb course(s) to adapt for your use. Please contact the CHTL (email:chtl@hkbu.edu.hk; phone:3411-5348) if you require further assistance.

  • Programme Directors are the overall persons-in-charge of the OA exercise, Course Leaders/Instructors of the courses designated for specific PILO(s) assessment in each semester will be involved during the relevant semester. Course Leader/Instructor for each designated course has to complete the Individual Course-PILO Assessment Report (Form 3) at the end of the appropriate semester. Bb Learn and Outcomes can produce this report if they are used.
  • Programme Directors will aggregate the reports from the Course Leaders/Instructors and complete the respective PILOs Assessment Report (Form 4). If the individual courses involved have make use of Bb Learn for their assessments, Bb Outcomes can be used to produce a consolidated report for each PILO/GA assessed.

Figure 4. Outcomes Assessment Framework



4. Outcomes Assessment in Blackboard (Bb) Learn and Outcomes Systems

Figure 5. Outcomes Assessment in Blackboard

  • The Blackboard Outcomes system (Bb Outcomes) which is part of the Bb Learn system is a tool that can be used to collect data/evidence of student works, as well as for carrying out the OA. Bb Outcomes can generate reports for analysis and follow-up actions if deployed appropriately.
  • To fulfill the requirement of the Assessment Policy on keeping student works for ACP visits and related purposes, it is desirable to ask students to submit their work (artifacts) electronically into Bb Learn and Bb Outcomes wherever possible.
  • Bb Learn and Bb Outcomes can generate reports for Forms 3 & 4 (see Figure 4) at the end if these systems are used in the process.