Rob Elliott's case

Initial scope

Canvas log data (one semester for two courses) showing operating system, which frequently tells you which device a student was using when they performed a particular action.

Frankly, I just requested way too much. Once I explained what I was trying to do and how I was trying to look at the log data, the eLearning Lab was able to provide it to me in a way that made things a lot easier. It cut out so much legwork for me. Otherwise, it's literally every click somebody makes in Canvas 24 hours a day.

Using the data curated by the eLearning Lab, Elliott was able to run simple queries like how many mobile sessions were there? What time of day were the mobile sessions? How many times did a student log on to Canvas on their laptop and their phone at the same time?

Revised scope

Two years' worth of Canvas data for all of IU, very tightly refined and formatted by the eLearning Lab to match what Elliott needed, paired with Student Information System (SIS) data for demographic information about the students without any personally identifiable information.

Now Elliott can pursue questions like how much more likely female students are to use their phones to access Canvas, or how much more likely 18- to 22-year-old students are to access Canvas on their mobile phones from 11pm to 3am (in both cases, the answer is a lot more likely). But it took the eLearning Lab's assistance to make the data usable. They worked with Elliott to narrow down exactly what he was looking for and ensure it made sense.

It required vast knowledge of all the different data sets that we have at IU—everything from enrollment data to faculty data to student demographic data to Canvas data (all those different pieces that exist in all the different systems we use every day)—not to mention extracting data in such a way that we could make connections between them.

For Elliott, data that were comprehensive, and data that aligned despite having been captured in separate places, would not have been possible without the eLearning Lab. He needed it to know exactly what was happening, especially in terms of the big picture beyond his own classes. Previous surveys and interviews provided student responses he could compare to the data, then go back to the students to ask much more pertinent questions.

Some of Elliott's hypotheses were way off the mark, but he would not have known that without being able to look at all of these data. Some of his key findings included:

• Students on IU's Bloomington and Indianapolis campuses are the most into using their phones, but they are also the youngest. Traditional college-age students are the ones who use their mobile devices quite a bit.
• Female students are significantly more likely to use their mobile devices, tablets, and phones to login to Canvas.
• Full-time students work on their phones more than part-time students. And a significant number have only ever looked at Canvas on their phones (they never login to Canvas on a computer).
• Students check Canvas frequently, meaning four or five times a day, but they are doing short, ad-hoc things. Anytime there is an announcement or a new grade, they respond to push messages on their phones and go right into Canvas.

Based on these realizations, Elliott has made some changes to his courses. Instead of creating PDF files of documents, he links directly to those files in a native format like a Word doc or a Google doc. On a mobile device, the PDF comes up looking like a postage stamp, but the Word or Google doc flows naturally and responds to student preferences like font size or dark mode.

Also, now that he knows how frequently and quickly students check Canvas on their phones, Elliott makes sure everything appears on the Canvas to-do list. He treats it as a neatly organized checklist of everything they need to do, meaning not just assignments and tests but also reading for the next class (along with a link to whatever it is, so they can read on the go).

Essentially, Elliott is trying to choose approaches and formats that are a better fit with the student behavior revealed by his research data. And that means really focusing on whether course content is mobile friendly -- in other words, focusing on creating a day-to-day environment that helps students succeed.

Working with the eLearning Lab helped Elliott arrive at meaningful insights based on IU data from multiple systems. Even if faculty only have an inkling of a research question, Elliott thinks the most valuable part of working with the eLearning Lab is the back and forth of working that question into something answerable with IU data. The lab provides a crucial consultation role, which helps faculty identify and get the most out of the available data.