Using videos to improve oral presentation skills in distance learning engineering master's degrees

ABSTRACT In this paper we present a pilot study to improve the learning process on distance learning Engineering master's degrees. We propose an activity where the students individually produce a video in which they explain the solution to one of their assignments. They then receive the teacher's feedback and are acquainted with the assessment rubric. Finally, they produce a second version of the video, the assessment of which is the basis for their overall mark. The results of the study show that students improved their ability to give oral presentations.


Introduction
Distance education has grown a lot in the last decades with the development of an increasing number of training programmes. For instance, 360,000 foreigners studied distance learning courses at Harvard University in 2017. Furthermore, the MIT (Massachusetts Institute of Technology) offers over 2400 distance courses, with over 300 million visitors (https://ocw.mit.edu/about/, accessed in May 2019). The platform edX (https://open.edx.org) gathers more than 20000 courses deployed around the world, in 32 languages with more than 40 million learners reached. These facts and figures have been impacted by the important advances in Information and Communication Technologies (ICTs) over the last decade.
The overall goal of this work is to introduce an active learning activity for students in online Engineering master's degrees, and show that it helps improve students' oral presentation skills. In particular, we ask students to record a video where they appear in person and explain in detail the solution to one of their assignments. A similar study for teachers was described in Kersting (2008).
Students need a stronger commitment and gain greater mastery of a task when they have to produce a video where they explain in person (and not only do) the solution to the task. They also deploy a key skill that is rarely developed in scientific subjects and is important for the students' professional development: the ability to give oral presentations. This skill has been addressed in several studies: Gray (2010) considers that most recent accountancy CONTACT P. Gregori gregori@uji.es Instituto Universitario de Matemáticas y Aplicaciones de Castellón, Departamento de Matemáticas, Universitat Jaume I de Castellón, Campus Riu Sec, E-12071 Castellón de la Plana, Spain graduates lack oral communication skills; Chan (2011) analyses its importance in scientific subjects; Živković (2014) presents the specifics of designing oral presentations for language students; and van Ginkel, Gulikers, Biemans, and Mulder (2015) identify and classify relevant studies with the aim of deducing a set of design principles with underlying conceptual and empirical argumentation for developing oral presentation skills.
The Tuning Project was an initiative of most of the European countries designed to define common terminology and concepts for the planning, maintenance and description of programmes in European higher education. This project has helped enhance the overall quality of educational programmes offered by European universities, in particular regarding society and learners' needs (dalle Rose & Haug, 2013). Our proposal allows us to put in practice the generic competences 3 (' Ability to communicate both orally and through the written word in first language'), 5 (' Ability to plan and manage time') and 27 ('Skills in the use of information and communications technologies') of the Tuning Project (Bultitude, 2011;Tuning, 2017).
According to an OECD's report (see OECD, 2013), this 'competence approach' is an educational proposal that aims to form constructive, committed and reflective citizens. In particular, this proposal allows the students of Engineering and master degrees to identify and understand the role played by mathematics in the world. Indeed, it provides the mathematics curriculum with a structure aimed at developing mathematical processes such as reasoning, representation, calculation, modelling, problem-solving and communication (Solar, Azcárate, & Deulofeu, 2012). In this regard, the authors' experience as mathematics teachers reveals that students' oral communication of scientific knowledge has room for improvement. This is one of the motivating reasons for this research.
We have conducted our research in the School of Technology and Experimental Sciences at Universitat Jaume I in Castellón (Spain). Our school awards bachelor, master and doctoral degrees in several STEM fields. In particular, the Master's Degree in Computational Mathematics programme offers 600 hours in two semesters, and combines courses in mathematics and computer engineering. We are implementing this study as a pilot study in two courses of this programme (with distance students): firstly, because there are fewer students, and secondly because, due to their age and formation, they are generally more mature than undergraduate students. Nonetheless, we are hopeful that the application of this experience in undergraduate Engineering courses is also feasible.
Our research yields two important benefits. Firstly, it helps students develop their ability to give oral presentations on scientific subjects. Secondly, it enables us to identify the student and gives more clues about the authorship of the work.
The description of our experiment, together with a rubric, are presented in Section 2 of this paper, while the results are shown in Section 3. The improvements resulting from this procedure are shown in Section 4, where we draw conclusions about the study.

Proposal and purpose
In distance learning courses, students may be far from the teaching centre and it is usually difficult for them to attend examinations or tests in person. In this paper we propose that students produce videos, as an active learning activity within the context of Engineering master's degrees. Our proposal has a two-fold purpose: firstly, to improve the oral skills of students on the aforementioned courses, and secondly, to provide an effective method of assessment that reduces the risk of impersonation. Additionally, it increases the students' engagement in the learning process (Guo, Kim, & Rubin, 2014;Helme & Clarke, 2001).

Previous research
Among the recent references in the related literature, we have the ones cited in the third paragraph of the introduction. In addition, Bhattacharyya (2014) identifies the existence of interdisciplinary and interpersonal competences, in particular the ability to communicate, which are essential for engineers to be leaders in their professional fields. She defends that correct communication in oral technical presentations is a requirement that employers usually demand. Synergy and collaboration between academics and professionals are, according Bhattacharyya (2014), essential to ensure that future graduates are encouraged to speak and engage in communications and work meetings in an effective manner. Another such study was described by Greculescu, Todorescu, Popescu-Mitroi, and Fekete (2014) with a sample of 250 students in their third academic year of Computer Science, Electronics and Telecommunications, Electrical Engineering, Chemistry or Mechanics degrees. The purpose of their study was to identify the weaknesses and strengths of the Engineering teaching staff when making oral presentations. The authors reported that their experience helped to improve speech, avoid repetitions and increase motivation. Dunbar, Brooks, and Kubicka-Miller (2006) used a performance-based rubric to assess the oral communication competencies suggested by the National Communication Association. In addition the authors offered suggestions of such a type of rubric in other educational settings, for assessing other student skills and for training new teachers. Murillo-Zamorano and Montanero (2017) undertook a thorough study comparing the effect of peer and teacher feedback in oral presentations by Economics and Business students.

Description of the experiment
In this study, students are given an extra assessment item to complete. We ask them to record a video in which they make a presentation about the proposed assignment and describe in detail how they solved it.
Initially, the teacher gives general advice on how to make effective presentations, and students produce and submit their videos. The videos are then assessed according to a rubric and the teacher sends the feedback (comments, plus the rubric used with their marks) to the students. Finally, the students produce a second version of their videos and they are graded according to the mark obtained, again using the same rubric.
To determine how our proposal would work in practice, we performed a pilot study at our university, with students of the Master's degree in Computational Mathematics at Universitat Jaume I in Castellón, Spain. We chose the Fall 2017 courses 'Data mining' and 'Mathematical methods for partial differential equations' because their teachers were interested in the proposed innovation. There were 11 and 13 students in these groups, respectively. The usual methodology of these two courses consisted of master lectures (recorded on video for distance-learning students), supplemented by video tutorials made by the teaching staff. The students were graded by assessing an activity book containing a certain amount of assignments.
This time, the students had the additional work of recording a video about the topics: 'Association Rules' and 'Solution of the Heat Equation by Separation of Variables'.

Aims
The main goal of our contribution is to encourage the acquisition and development of the competence 'oral communication of scientific works'. Nonetheless, the activity that we propose, the production of a video, also compels the students to gain a better command of their assignments, since they have to explain it in public. The nature of the activity also leads to an increase in student-teacher interaction, because of the feedback session. Notice that student-teacher interaction in online courses is occasional. It is worth mentioning that the students' physical appearance in the videos, and their speech, reduces the risk of plagiarism or impersonation, a potential failing in distance education. Finally, we have also adapted the assessment rubric of Huba and Freed (2000) for a more objective evaluation in distance learning courses.

Steps of the experiment
We started by announcing the new activity in the forum of each course. The activity consisted of completing one of the usual assignments (a list of problems about the topics and courses mentioned in Section 2.3) and producing a 12-15 minute video in which the students had to explain how they had solved the proposed problems. They had a deadline of two weeks to submit the video. The description of the task included a list of general guidelines. The first set of guidelines concerned the planning of the presentation: prepare the presentation considering the audience (teachers); establish the purpose; research the topic; select support material; organize the speech; motivate the audience; and state clear conclusions (Fischhoff, 2013;Gareis, 2006;Jucan & Jucan, 2008;Lortie, 2017;Penn State University, 2017). The second set of guidelines concerned the effective communication: speak at an appropriate rate; use pauses for emphasis and give the audience time to understand; maintain eye contact; only present one major point per slide; do not make excessive use of lengthy text; use simple visuals; use the final slide for links to additional resources; and finally, review the production and be self-critical. There were no guidelines about technical resources for video recording, as we did not want to impose restrictions in this respect and we were willing to find out the different solutions that the students would provide.
After the deadline, the teacher of each course assessed their students' productions using the rubric described in Section 2.6. The time spent assessing the students videos was, in general, longer than the usual time for assessing written assignments. However, the teachers were satisfied because they had more elements to decide whether the students had internalized the instruction. One viewing was needed to assess the scientific correctness (accuracy, text, and verbal language), followed by a second viewing to assess communication skills (organization, image and sound quality and body language).
The students then received their teacher's feedback, as well as the assessment rubric. In accordance with De Grez, Valcke, and Roozen (2009), and because of the difficulty of the mathematical tasks, peer feedback was not considered. Although Murillo-Zamorano and Montanero (2017) concluded that a single step of a feedback session is not effective in the long term, we were hopeful that in the different context of our study (master's degree students; self-recorded video instead of a live presentation, so it can be repeated until the student is satisfied; mathematical subjects) the single step of a teacher's feedback session would lead to significant improvements.
The teachers' feedback emphasized errors in terms of not being sufficiently rigorous or not expressing scientific concepts with the necessary clarity. Feedback also suggested possible points for improvement: scientific accuracy, clarity of image and sound, choice of fonts, organization of the presentation and management of body language, among other aspects. In particular, we wanted to make the students aware of the importance of both speech and body language in an oral presentation. It is not just the message that is important, but also the structure, meaning and level of agreement with the interlocutor. Body language can convey much more information than words alone (Pease, 2006). Our gestures are constantly giving information: appearance, posture and facial expression convey a certain degree of reliability, level of knowledge and understanding of what one is saying (see the seven principles covered in van Ginkel et al., 2015).
The technical aspect of the video production was discussed specifically with every student. All students were asked about the tools they had used and teachers became acquainted with free software under all the main operating systems. Students with poor results in image and sound quality were asked about their resources and were guided for the choice of available devices, free software or free cloud services.
Finally, after the feedback was given, students had one week to produce and submit a second version of their videos. These videos were assessed using the same rubric, yielding the students' overall mark. It is worth mentioning that the activity (with its assessment) was optional, and that all the students agreed to participate. Table 1 shows the assessment rubric used in our study, based on the one introduced in Huba and Freed (2000, pp. 156-157) and Allen and Seaman (2015). The rubric assesses three levels of achievement for the following categories: organization of the presentation, image and sound quality, and communication tools (consisting of text, accuracy of content, verbal communication and non-verbal communication). We have checked that the chosen categories meet the requirements specified in Moskal and Leydens (2001) with regard to reliability. In order to check its validity, the rubric has been used on video lectures by teachers who are good communicators (according to the official students evaluation of teaching quality at our university), yielding high marks.

The rubric
The rubric was shared with the students before they prepared the second version of their videos. They knew its result would be used for the final assessment of the activity. We have performed formative assessment as suggested in Fook and Sidhu (2014). The total mark of the student j, denoted by M j , was obtained by using the formula:  where w i is the weight of the category i (see Table 1) and q ij is the mark obtained by student j in category i (1 for 'Not Yet Competent', 2 for 'Competent' and 3 for 'Sophisticated'). Therefore, the total mark is a value between 1 and 3, with the possibility of students obtaining intermediate values between 'Not Yet Competent', 'Competent' and 'Sophisticated'. Sharing the rubric with the students has some advantages. It allows the students to have a hand in their own improvement. In addition, they feel the assessment is more objective (Andrade & Du, 2005;Reddy & Andrade, 2010) and they also tend to deploy better oral communication skills, because the rubric describes the importance of those aspects for the final mark.
After the second version of the video was assessed, an analysis was carried out in order to test the effect of the proposed innovation. The dataset of students' marks per component and version is shown in Table 2. A summary of this data appears in Figure 1.  respect to the first version, we can see that students get higher marks in text and accuracy, average marks in verbal communication and lower marks in body language and organization. In summary, master's students are good in terms of quality of content and writing. We believe that this is because master's students have already achieved an undergraduate degree and have a good command of the presented contents, moreover some of them are secondary school teachers, a fact that explains their rigour. However their speech is markedly dull and discouraging for listeners. We consider this type of experiences worthwhile in a context of undergraduate programmes, expecting worse results in the first version of the video, and possibly also in the second. The distribution of marks for the second version of the video (right bar of each paired bar in Figure 1) shows that all components have improved, except Verbal Language, as an effect either of the teachers' feedback or of becoming aware of the rubric, or simply due to experience. Figure 2 shows the evolution of the students' median and mean marks from the first to the second version of the video for each aspect of the rubric. After the teacher's feedback, only the mean marks corresponding to the 'Organization' and 'Body Language' categories have improved substantially. Indeed, we performed a statistical hypothesis test (the Wilcoxon test, see Conover (1999, p. 352)) for every aspect, which confirmed, according to the p-values shown in Figure 2, that there was a statistically significant improvement in the aforementioned aspects, as well as in 'Text'.

Results
In addition, an online forum was opened in order to share the video and encourage peer comments. Those comments showed the students' enthusiasm for the experience and satisfaction with the final results. The activity also gave students greater opportunities to socialize with each other, which is important in the case of distance courses, since students are usually isolated and unaware of their peers. A wide range of software and hardware was used by students to produce their videos. A few students recorded themselves writing on a blackboard and used their own Android smartphones, except one student, who used a professional camcorder. The rest recorded their talk using their computers (mostly laptops under Microsoft Windows, but also Apple iMac laptop, and only one in Linux). The desktop showed the slides (generally in PDF format or PowerPoint), and the webcam captured their image, which was overlaid in a corner of the desktop. The software programmes used for the desktop recording were Camtasia Studio and Debut Professional NHC Software in Windows, Quicktime in MacOS or Windows, and webcamoid together with recordmydesktop in Linux.
With regard to the academic results, Figure 3 clearly shows a positive evolution of the overall marks after the students received guidance from the teaching staff. The activity was marked according to the rubric, using the weights assigned in Table 1 to highlight the most important aspects.

Conclusions
The students and teachers involved in the experience expressed their satisfaction with the activity. The main goal of the assessment was to develop a skill: the ability to give oral presentations. This skill is usually neglected in the curricula of scientific and engineering courses, nevertheless very important from a professional point of view. To do this, we designed an 'active learning' activity, where students produced videos, teachers provided quality feedback and the assessment rubric, and students had a second chance. In addition, the videos reduced the possibility of students' impersonation. In this regard, we should mention that a Gallup poll in 2013 (Saad, Busteed, & Ogisi, 2013) showed that about 50% of employers felt that the assessment in distance learning was less reliable than in face-to-face courses.
A significant improvement was observed in the 'Organization', 'Text' and 'Body Language' categories (see Figure 2, with the low p-values). We cannot ascertain whether the improvement was due to the teacher's feedback (then an effect of the present innovation) or not; the sample was too small for an experimental design with a control group. We propose such analysis for a future experience with a larger sample.
In the 'Image & Sound Quality' category, a great diversity of resources were used. For the second version of the video, the teachers took into account the resources available for the students, and put relative marks, so that students could achieve the maximum regardless of the technical means they had chosen. The marks increased in general, but not in a statistically significant way.
With regard to 'Verbal Language', despite having an acceptable level, no evolution was observed. In both cases all the marks were 2. These results encourage us to continue with this type of activities, increasing the number and diversity of problems, as there is room for improvement.
There is some controversy about the effectiveness of distance learning programmes in Mathematics, as discussed in Matzakos and Kalogiannakis (2018, Section 3.3). Based on personal communication with students, we have appreciated that they were satisfied with the proposed activity. Furthermore, the involved teachers expressed that they felt more confident about the authorship of the students' works We conclude by listing some suggestions for improvement, for the different agents involved in distance education, in future applications of our proposal. Firstly, institutions should engage in these kind of active learning practices and provide their staff with sufficient human and technological resources. Secondly, teachers should select highly motivating problems. Thirdly, we believe that it is more worthwhile setting a few short assignments (with shorter videos) along the course, than setting just one assignment related to a longer video. Furthermore, teachers should be approachable and encourage students to ask them for assistance at any time, as well as reminding them to work harder on the first version of the video. We have noticed that students usually work on their own before the first feedback session, instead of attending previously the designated (distance) consultation hours. Lastly, another suggestion is to introduce gamification in this activity, by establishing, for instance, a peer system of mark feedback assessment, attractive milestones, and some other means, as in Zhu, Pei, and Shang (2017).