Exploring the use of screencasts in a postgraduate research methodology class : a case study

Abstract

This mixed-methods case study of a sequential explanatory design was conducted to examine the effects of screencasts on students’ learning of advanced statistics in a postgraduate Research Methodology course. The screencasts were designed based on Gagne’s events of instruction and Mayer’s multimedia learning principles. Therefore, the case study also explored the postgraduate students’ views of the effectiveness of the screencasts in terms of their instructional design. The participants were thirty-three (N = 33) local and international postgraduate students undertaking a course in educational research methods at the Kulliyyah of Education, International Islamic University Malaysia (IIUM). Data were first collected in a pretest-posttest single group experiment, and then in a focus group discussion involving five participants. Students’ learning of the advanced statistical content was measured using an open-ended test comprising 15 questions totalling 80 marks, while their views of the screencasts were captured using a 33-item screencast evaluation questionnaire and FGD protocol. Intra-rater and inter-rater consistency estimates were calculated for the students’ posttest scores in advanced statistics, yielding reliability indexes of .96 and .99 respectively. The screencast evaluation questionnaire demonstrated acceptable Cronbach’s alphas ranging from a low of .75 (for the “gaining attention” design element) and a high of .95 (for the “providing learning feedback” element). The case study acquired both numerical and narrative data. Descriptive statistics, Pearson correlation (r)and paired samples t-test were used to analyse the numerical data, while the narrative data were analysed using inductive thematic content analysis. A major finding of the study was that screencasts significantly improved students’ learning of advanced statistics with an average learning gain of 56.24 marks from a pretest average of 1.70 marks from a total of 80. The effect size of the treatment was very large (Cohen’s d = 5.96). The Pearson’s correlation analysis pointed to a significant but moderate relationship between academic achievement (CGPA) and student learning. Moreover, almost all participants (94%) supported the idea of studying with screencasts. In terms of instructional design, the participants positively rated the screencasts, especially their clear learning objectives (94.8%), embedded guidance tools (91%) and the chunking of the content (89.2%). The quantitative results were corroborated by the narrative data from which ten themes emerged. Students explained how the screencasts facilitated their learning cognitively and psychologically by increasing their motivation to learn more and empowering them to learn autonomously. The findings provide strong empirical support for the utilisation of screencasts in higher education courses and establish the importance of instructional design in screencast development.