Washback study of an outcome based english language assessment on student learning

Abstract

The assessment reform around the world has culminated in the adoption of various assessment systems that focus on student learning, and in the case of higher Malaysia, Outcome Based Assessment (OBA) was adopted. Since OBA was claimed to be an effective system to enhance student learning, it is interesting to note that learning may be debilitated by poor assessment practices. Thus, the present study investigated how OBA has thus far influenced student learning and their learning outcomes by means of sequential exploratory mixed methods with item development. Specifically, this study investigated the intended and unintended washback of an OBA, i.e., Integrated Language Skills III test battery, focusing on its washback effect on students’ learning and learning outcomes. A survey named WOLOS was developed and validated by means of qualitative (semi-structured interview) and quantitative analysis (Item Objective Congruence (IOC) and Rasch Measurement Model). Drawing on an argument-based validation approach, i.e., Assessment Use Argument (AUA), the evidence about the washback effect of Integrated Language Skills III test battery were linked to construct validity. To better understand how the students experienced Integrated Language Skills III test battery, students who have taken this test battery were selected as the participants of this study. Document analysis, semi-structured interviews, focus group discussion and survey data were collected. The document analysis provides information on the intended washback of Integrated Language Skills III test battery. Semi-structured interviews and focus group discussions involving 22 students were analysed using NVivo12, by means of phenomenology approach. Rasch Measurement Model was employed to analyse the survey data from 309 respondents. A range of factors were found to have mediated the intended washback, both positively and negatively. In short, it was found that Integrated Language Skills III test battery brought about more positive washback owing to its 10 years of implementation. Further directions may involve studies which include various stakeholders’ perspectives to perceive the consequences of a test and focusing on the impact of technology in teaching and learning.