Introduction
Methods
Results
Discussion
Limitations
Conclusion

INTRODUCTION

Medical education is a rapidly evolving field that has been using new forms of media and technology to enhance the learning of medical students across the U.S. One of the more prevalent and extensively used advancements is the use of video recording systems. In previous studies, medical students have reported greater subjective benefit from video-recorded lectures than from live lectures.^1,2,3 Some of the subjective benefits that students reported included faster knowledge acquisition, better retention of material, more focus, and easier access to additional information.¹

The advantages to using video-recorded lectures include the ability to rewind, pause and return to finish a lecture later, and watch lectures at faster speeds. In a medical school setting where knowledge of minutiae and comprehension of concepts is paramount to success, the added flexibility that video-recorded lectures provide could be extremely important. As medical education demands countless hours of studying, the biggest advantage may be the ability to watch lectures at faster speeds. As an example, for every hour of material in the traditional classroom setting played at 1.5× speed, a student could save 20 minutes. Therefore, in a typical four-hour morning lecture scenario, a student could save 80 minutes by watching the video at 1.5× speed.

Though the benefits may seem numerous and prior survey studies have shown subjective benefits with lectures played at faster speeds, no quantitative studies to date have measured information retention. The objective of this pilot study was to determine if watching video-recorded lectures at faster speeds compared to the original recording had any effect on the immediate retention of novel learning material.

METHODS

This was a prospective, single-center, randomized controlled trial, pilot study that presented a novel curriculum to medical students and tested information retention with a short examination. Material presentation and assessments were all done in a single day. The study was approved by the university institutional review board (IRB), as well as by the administration of the medical school involved.

Video selection and assessment creation

The two new educational subjects chosen were transducers and artifacts. The presented videos were recorded by a nationally recognized emergency ultrasound educator. These two specific videos were chosen because these topics are not covered in the medical school curriculum and, therefore, we believed that the material presented was novel. Subjects were also asked if they had exposure to the material; if they had, they were excluded from the study. Additionally, these videos were similar in length (transducers, 12 minutes; artifacts, 15 minutes) and “factoid heavy,” or in other words, had a wealth of material that could be used to assess student learning.

Population Health Research Capsule

What do we already know about this issue?

Previous survey studies have shown a subjective increase in the rate of knowledge acquisition from sped-up lectures. However, no quantitative studies have measured objective retention.

What was the research question?

Would students’ mean test scores differ significantly after watching video lectures at 1.5× speed compared to 1.0× speed?

What was the major finding of the study?

We found that watching a lecture at a faster speed may have a detrimental or no significant effect on learning novel material.

How does this improve population health?

This data is important to modern learners as it challenges the assumption that faster podcast speeds lead to potential time savings for learners.

We created a 23-question multiple-choice assessment for the transducers video, and a 20-question assessment for the artifacts video. Given the pilot nature of the study, the assessments were tested by the two medical students involved in the study as well as by the video creator and content experts to ensure that the two tests were of comparable difficulty levels. To avoid different interpretations of a correct answer, the tests asked about definable facts explicitly stated in the video. The investigators created an answer key for both assessments, which were subsequently reviewed and proofread before submission to the IRB.

Subject recruitment and privacy

We conducted the experiment in August, at the beginning of the school year, when first-year (M1), second-year (M2) and third-year (M3) medical students had limited exposures to emergency medicine (EM). Subject recruitment was conducted by first sending an email to all four medical school classes. This was then followed by live announcements to each class. All students were told that results on the assessments would have no effect on medical school evaluations or grades and that participation was voluntary. Students who participated were given a $5 Starbucks gift card as a token of appreciation.

Inclusion criteria for subjects included being a medical student and being over the age of 18. The primary exclusion criterion was having been exposed to the presented material before, since prior knowledge of the subjects could skew the results. Additionally, medical students with prior ultrasound experience and students rotating in EM were excluded from the study.

To enforce the exclusion criteria, each participant was asked to enroll via Google Docs and sign an informed consent prior to the study. Each student was asked to answer “yes” or “no” to having previously seen either the transducers video or the artifacts video. Those who answered “yes” received the $5 Starbucks gift card and the opportunity to sit in on the study, but they were excluded from examination and data analysis. To protect the privacy of student performance, each student was identified by his/her student ID number. Demographic data including ethnicity, gender, and year in medical school were also collected. Assessment performance was not shared with the medical school.

Presentation and assessment

Of the 81 students who signed up for the study, 63 showed up on the day of the experiment. Two were excluded from the examination for having prior knowledge of the material, and seven were excluded from assessment for arriving late. As a result, 54 medical students were included in the final data analysis. Participants were randomized into group A or B by converting the Google docs sign-up document to a Microsoft Excel spreadsheet, and then using the “randomize” function to assign participants into either group A or B. On the date of the experiment, participants could see their group assignment at the check-in desk.

Group A watched the artifacts video at 1.5× speed first and then took the artifacts assessment immediately after the video. Following this, Group A watched the transducers video at normal speed and immediately took the transducers assessment. Group A served as the experimental group for the artifacts video and the control group for the transducers video. Group B watched the transducers video at 1.5× speed first and took the corresponding assessment immediately following the video. Group B then watched the artifacts video at normal speed and took the assessment. Group B served as the experimental group for the transducers video and the control group for the artifacts video.

Scoring and statistical analysis

Since the two assessments had an unequal number of questions, the scores were converted into a percent correct score. Then the means and standard deviations (SD) of the variable of interest and percentage score were calculated for each assessments (Group A Artifacts, Group A Transducers, Group B Transducers, and Group B Artifacts). Group A Artifacts at 1.5× speed was compared to Group B Artifacts at normal speed using a t-test. Similarly, Group B Transducers at 1.5× speed was compared to Group A Transducers at normal speed using a t-test. To compare the demographic information, such as year in medical school, gender, and ethnicity, the Fisher’s exact test was used. The p-values were used to determine if there was a significant difference in performance for both videos. Statistical significance was set at p < 0.05. All analyses were completed in R version 3.4.1 (R Core Team; Vienna, Austria).

RESULTS

Subject data

A total of 54 students participated in the study. Of these, 21 were M1s, 27 were M2s, two were M3s, and four were M4s. Thirty participants were female and 24 were male. Finally, seven participants self-identified as Asian American, one as African American, 45 as Caucasian, and one participant chose not to provide ethnicity (Table 1).

There was a similar distribution of M1, M2, M3, and M4 in each group. Among the 21 M1s, 11 were in Group A and 10 in Group B. Among the 27 M2s, 14 were in Group B and 13 in Group B. Each Group A and B had three M3 and M4 participants.

Assessment results

The average performance of Group A on the assessment for Artifacts at 1.5× speed ± SD was 61.4 ± 19.3% (95% CI [−53.9, 68.9]). On the Transducers assessment at normal speed, Group A’s mean performance was 73.8 ± 15.6% (95% CI [−67.7, 79.8]) (Table 2).

The average performance of Group B on the assessment for transducers at 1.5× speed was 66.9 ± 17.6% (95% CI [−59.8, 74.0]). For artifacts at normal speed, Group B averaged 72.7 ± 14.6% (95% CI [−66.8, 78.6]).

For both videos, the performance of the control and experimental groups were compared using the t-test, and estimated the effect sizes using Cohen’s d. For the artifacts video, there was a significant difference between the performance at 1.5× speed compared to 1.0× speed (p= 0.0188), along with a moderate effect size (Cohen’s d = 0.654). For the transducers video, a statistically non-significant (p= 0.1365) difference was found in the performance between the two groups along with a small effect size (Cohen’s d = 0.414).

We compared the performance of the control groups of artifacts and transducers videos using the t-test and found no significant difference (p= 0.7965), suggesting that the tests did not differ in difficulty.

DISCUSSION

We chose to conduct a quantitative analysis of students’ information retention after viewing a sped-up video compared to retention at normal speed because there was a lack of literature regarding the topic. To assess retention, novel education material was presented to the test subjects one at normal speed and another at 1.5× speed and assessed comprehension after each video. Participants overall performed worse on assessments after learning from 1.5× speed compared to 1.0× speed. For the artifacts video, the average test score was 72.7 at 1.0× speed compared to 61.4 at 1.5× speed. For the transducers video, the average test score was 73.8 at 1.0× speed compared to 66.9 at 1.5× speed. Our findings were contrary to previous studies that reported subjective, accelerated learning when learning from videos at faster speeds.

The difference in performance on the artifacts quiz at 1.5× speed compared to the control group was significant. Although the difference in performance of the 1.5× group compared to the control group was not significant for the transducers video, the difference was equivalent to a letter-grade difference.

The discrepancy in relative performance between control and experimental groups for each of the videos may be explained by confounding factors. The main confounding error that may have led to the difference in results was that the artifacts video and assessment may have been more inherently complex in nature compared to the transducers video and assessment. Although the video selection was done deliberately to ensure videos were similar in length and in the amount of fact content covered, retrospectively we realized that for a novice learner, a few of the questions on the artifacts video may have been more conceptual compared to the transducers questions.

For example, many students answered correctly to the question asking about the A line on the artifacts video, much in the same way they answered correctly about the linear transducer having a higher frequency. However, with no prior knowledge of how ultrasound works, many students answered incorrectly on posterior acoustic shadowing vs. enhancement. As one participant later remarked, as a new learner she focused all her cognitive energy learning to associate that stones cause sound waves to reflect back, which made her associate stones with the word “enhancement.” However, someone with a basic concept of ultrasound would have easily picked up that because of this reflection of sound waves off the stone, there would be shadowing of the structures lying posterior to it. We theorize that when learning multiple-step processes such as these, playback speed plays a more significant role than when learning a rote memorization fact.

This data is important to modern learners as it challenges the assumption that faster podcast speeds lead to potential time savings for learners. This time savings is only realized if the retention of the material is comparable.

LIMITATIONS

Among this study’s limitations was the small sample size of 54 students that limited the power of this study. Also, even though students with prior exposure to subject material were excluded from the study, it was impossible to ensure that the entire study population was naïve to the material. Another limitation was the inability to establish that the two novel subject matters presented were equivalent in complexity, as discussed above. Final noteworthy point was our decision to play the 1.5× speed video before the 1.0× speed. This may have affected performance and impacted the study results.

Future studies

This study was designed to examine the immediate recall of information after watching a video at 1.5× speed vs. at normal speed. However, to emulate the full utility of video-recorded lectures, students must be given the ability to rewind parts of the lecture they did not understand or re-watch a lecture a second time. A potential study design to examine this could involve giving both the experimental and control groups the same allotted time to learn a lecture while using 1.0× speed or 1.5× speed and comparing their performance. Long-term information retention is another variable that should be assessed. This study only tested immediate recall.

CONCLUSION

Our study suggests that watching a video lecture at a faster speed may have detrimental or no significant effect on learning novel material. Contrary to previous studies showing subjective improvement in performance with sped-up, video-recorded lectures compared to normal speed, our data showed that immediate retention of novel material at 1.5× speed was worse compared to normal speed.

Footnotes

Section Editor: Andrew W. Phillips, MD

Full text available through open access at http://escholarship.org/uc/uciem_westjem

Address for Correspondence: Christopher Doty, MD, University of Kentucky, Department of Emergency Medicine, M53 Wm R Willard Medical Education, Lexington, KY 40536. Email: christopher.doty@uky.edu. 1 / 2018; 19:101 – 105

Submission history: Revision received August 20, 2017; Submitted October 20, 2017; Accepted October 11, 2017

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.

REFERENCES

1. Cardall S, Krupat E, Ulrich M. Live lecture versus video-recorded lecture: are students voting with their feet?. Acad Med. 2008;83(12):1174-8.

2. Lovell K, Plantegenest G. Student utilization of digital versions of classroom lectures. JIAMSE. 2009;19(1):20-5.

3. Bacro TR, Gebregziabher M, Fitzharris TP. Evaluation of a lecture recording system in a medical curriculum. Anat Sci Educ. 2010;3(6):300-8.