Assessment of the use of preliminary concepts test and rubrics in practical learning at university

Abstract

The high number of students and teachers in the laboratory practices of the course Theory of Machines and Mechanisms, shared by several engineering degrees at Universitat Jaume I, makes the teaching planning and evaluation challenging. Some issues were observed in past years, such as differences in the student knowledge of basic concepts, underpreparation before practical classes, poor quality of the written reports, absence of common objective assessment criteria among teachers and low success rate. That led us to the implementation of an action for educational improvement during the academic year 2018-2019. The aim was to improve the students' level of training before attending laboratory practices, to standardize the evaluation among teachers and to involve students in the process of assessment in order to improve learning outcomes and the quality of the reports delivered. The improvement action included, on one side, carrying out an initial evaluation with a diagnostic test. The results of this test were used to estimate the basic knowledge of the students and proposing specific tutorial sessions in order to unify their level of knowledge. This method should increase the students involvement and improve the success while performing the laboratory practices. On the other side, rubrics were created in order to standardize the correction criteria. These rubrics were also made public to the students, before the classes, in an attempt to improve the quality of their reports. In addition, for some of the reports the students were asked to self-assess their own reports following the public rubric. The results indicate that teachers found the rubrics useful for objective and fair evaluation. However, the standard deviation across teachers has not been apparently reduced with respect to the previous year by the use of them, although the differences in the student cohort have to be taken into account. The effect on grades of the factors teacher and session, as well as their interaction, was significant (p<.05) according to the ANOVA tests, as already was prior to the application of the action. It was also found that the student grades in the practical classes did not improved significantly. A low implication in the preparation before classes and a low tutorial attendance were observed in the collective of students, possible explaining in part the discrete improvements obtained. The self-assessment improved the grades on average where it was applied. We concluded that the involvement of both students and teachers is necessary to promote learning quality using these techniques.