Challenges and requests
To get at these issues, Polly was interested in how video interactions in Kaltura activity logs for Introductory Anatomy lecture and laboratory components matched with in-class Top Hat and grade data. Issues accessing these data led her to seek out the eLearning Lab's assistance.
I was struggling, trying to sort through all of these data myself. One student at a time, one video or one question at a time. I have a pretty high tolerance for tedious tasks, but it was taking hours upon hours to sort through all of that data. But I knew I wanted to do more with this. I knew there was more here that I could look into that could really help inform my classroom.
The eLearning Lab was able to provide a merged analytic file between the Top Hat responses, final grades, assessment outcomes, and the Kaltura activity data, thereby allowing Polly to address such questions as:
- What is the relationship between students' earliest access to the video resource and their performance in exams and the course?
- How does the triangulation between students' beliefs about their studying and the activity logs in Kaltura relate to their performance in assessments and the course?
Preliminary findings, changes to the course, and future analyses
While Polly's work is ongoing, preliminary analyses have already revealed some insights that have led to changes in the course.
Some of the feedback that we've gotten from students and some of what we've seen is that [the videos were] overwhelming at the level we were doing it. We're asking them to do quite a bit of stuff outside of class. And so we are planning to adjust that for this fall. Keep that model to some extent, but require or ask a little less of them outside of the classroom.
Specifically, these changes include the following:
- Limiting the video resources that students were required to watch to one hour per week rather than per day to allow for more students to keep up and more questions during activities and lectures in class.
- Sharing observed relationships between engagement in class resources and assessment and course outcomes with students, instructors, and the field.
Further, Polly plans to interview students on how their perceived study strategies and observed behaviors may relate to their performance in the course. These data will allow for further triangulation between students' perceptions, their behaviors, and how they may be struggling with particular course resources.
The next thing that I would really love to do is to interview some of the ones that don't keep up and ask them about their thought processes. How do you think this is going to work out? How do you think the exam is going to go? That sort of thing. Because we have some suspicions, but this is another place we'd like to collect some data.
Working with the eLearning Lab enabled Polly to more readily analyze these data and make targeted changes to her course and study plans. Without the lab's support, Polly noted it would have taken an excessive amount of time to work and access these data. Instead, she was able to jumpstart her planned research and make more immediate changes to her course structure and practices.
