See All Posts
As adaptive learning has become more common in U.S. colleges and universities in recent years, there have been many studies of learning outcomes, and there have been many surveys of faculty and what they do and don’t find useful about the technology.
However, there have been fewer — if any — surveys of student perceptions of adaptive learning and what they do and don’t find effective about their experiences with it.
In 2017, Patricia O’Sullivan, then the project coordinator on a grant from the Association of Public and Land-grant Universities (APLU) to implement and scale adaptive courseware in gateway courses at the University of Mississippi (UM), decided to fill in that gap.
One of the goals of the Accelerating the Adoption of Adaptive Courseware grant was to increase student learning so more students can complete their degrees. Between September 2017 and April 2019, O’Sullivan and her co-investigators collected data through four end-of-semester surveys and 16 course-based focus groups. (Their results were published in Current Issues in Emerging eLearning in 2020.)
“Because the student experience is essential in assessing promising but untested educational initiatives, we felt it is important to understand how students are experiencing adaptive courseware, and whether or not they find it adds value to their education,” says O’Sullivan, who is now the Content Manager at Every Learner Everywhere.
The findings of the survey and focus groups made one thing clear to O’Sullivan: Students overwhelmingly said they wanted effective, efficient learning solutions that would give them good value for their money and offer them more control over their learning experience.
Setting up the study at UM
APLU included eight universities in the grant, but O’Sullivan knew managing it would be unique at UM because numerous individual faculty and departments were already using adaptive courseware with a lot of variation in the product, the course subjects, and the use cases. She knew she would not be able to limit faculty to an approved vendor list, as other universities in the grant that started from scratch were doing.
To find faculty who were using adaptive courseware, O’Sullivan scanned the shelves of the campus bookstore, pored through posted course materials, and contacted vendors. At the most active period of the grant, says O’Sullivan, the project included 13 different adaptive products and 82 faculty members, teaching 420 sections of 28 different courses in almost as many different disciplines.
Many students encountered adaptive learning more than once, adding up to more than 18,000 uses across the four semesters of the study.
The importance of integrating courseware within a class
The wide-ranging nature of the project nevertheless revealed some common reactions to the courseware, says O’Sullivan. Students enjoyed adaptive learning courseware features that gave them more agency when it came to learning and grades.
The most popular feature was the ability to take quizzes and other low-stakes assessments more than once, enabling students to learn from their mistakes without jeopardizing their grades. It gave them the chance to practice daily and to self-remediate using the courseware’s immediate feedback.
Due-by dates, an alternative to due dates, were also popular. For example, students enjoyed being able to complete and turn in all work by Sunday evening every week rather than on one particular day. This allows students to get the work done when it’s convenient for them.
Students most appreciated adaptive courseware when it was well integrated into a class, the survey found. By “well integrated,” students meant five things, says O’Sullivan:
- The textbook embedded in the courseware is the same textbook faculty teach from.
- The faculty use the courseware in significant and meaningful ways, such as for quizzes, homework, learning practice, and preparation for high-stakes assessments.
- Student performance in the courseware contributes significantly (15% or more) to their final grade in the class.
- Faculty use information from the courseware to adjust lectures or to reach out to struggling students.
- Faculty align class lessons with courseware modules such that students are learning about the same topic in class and in the courseware during a particular period of time.
When those criteria weren’t met, students were often frustrated.
“It is like taking two classes,” said one student of a misaligned class in a focus group. “One is the book and the homework, and one is lectures and the test.”
Other students complained that the courses were organized for them to do the hardest work on their own and to complete the easiest work in class. This was not necessarily what was intended by faculty members, who were using class time to teach the concepts and new material and assigned practice activities to out-of-class time on the courseware, but sometimes students felt abandoned and turned to other sources of help such as Google searches, online answer keys, and student forums.
The high cost of adaptive courseware
One prevailing complaint from students was that every courseware product they encountered was sold differently. Sometimes the courseware was covered by the course fees, and other times it was an extra expense like a textbook. In that case, the courseware might be bundled with other products, but not always.
For example, some products were bundled with a homework system, tutorials and an ebook, which students often perceived as a good deal. However, sometimes those bundles required purchasing a print textbook. That could be frustrating for students who knew they were paying more than if they’d been given the option to purchase the courseware on its own. In other cases, students felt they needed a textbook rather than the ebook.
“I can’t comprehend anything from e-text or anything so I spend the extra 120 bucks on a physical book that I can highlight,” said one student in the study.
Another complicating factor: Not all the courseware was sold through the same channel. Students sometimes purchased products from the bookstore, sometimes from vendors, and sometimes bought access through their departments.
Meanwhile, the access they were buying varied from a single semester to a year or more.
“It turned into a confusing situation for students because they never knew when they entered a class: Does my course fee pay for this, do I have to buy this, and if I have to buy this, what am I getting, and how long does it last,” says O’Sullivan. “There was no standard access period, point of sale, or package.”
One interesting aspect of this finding is that student concerns around value emerged from a question about the cost, rather than pricing models. They used the question about cost to make observations about getting value for their money, says O’Sullivan.
Students in some science courses, for example, were often required to buy expensive courseware packages, but they knew they could use it more than once for different courses throughout their major.
The same expense, however, annoyed students who had to buy expensive courseware for introductory courses for non-majors. Students didn’t feel the expense was justified when they were fulfilling a general education requirement.
“You are taking this course for one semester and then never use it again,” says one student. “You drop a hundred dollars for a semester to get a good grade in a class you are paying way too much money for.”
Students value courseware that helps them improve
More students had positive than negative views of digital learning platforms. For example, students were less likely to mind the cost of courseware if it helped them improve their scores and understanding of course materials, particularly in courses in their major.
O’Sullivan recalls one accounting student from a focus group who had been consistently earning B grades. Curious what the A students were doing differently, he checked the anonymized course-level learning data provided on the courseware and saw the difference that time spent on their lessons made in the outcomes. He started copying those habits, and his grade moved up.
“He was amazed that the data gave him information that would change his behavior,” she says. “He just didn’t know how to get a better grade before.”
While not every adaptive platform allows students to learn more about the activity of their peers, students across all the courses liked adaptive features that gave them actionable insights or tasks they could complete to improve their grades.
One particular student, says O’Sullivan, called the courseware her “backup instructor”; her actual instructor didn’t break down math problems in a way that she could understand them, but she was able to revisit lessons using the adaptive courseware.
“It was like a safety net for her,” she says.
Other students appreciated how courseware allowed them to return to a lesson without having to rely on their own notes taken during class. Students, for the most part, appreciated any feature that gave them autonomy over their learning experience.
“They were making choices, rather than participating in the model of passive learning they were led to expect from large lecture classes,” says O’Sullivan.
Students occasionally pushed back on active learning. First-year students didn’t like the perception that they had to teach themselves, but third- and fourth-year students weren’t concerned about that, says O’Sullivan. By the end of college, their perceptions had changed; those students valued having agency in the learning process.
Students value authentic work with faculty
One of O’Sullivan’s biggest takeaways from this research is that students value learning experiences and tools that are efficient — feedback that applies to both adaptive courseware and traditional classroom learning and assignments.
“They don’t want to read 50 pages of text when all they really need to reach are those critical two pages,” she says.
“They want courseware to target their gaps in mastery rather than reviewing content they already know, and they want opportunities to get help when they need it rather than having to wait on office hours, a response to an instructor’s email, or the next time the class meets.”
Despite students’ overall positive view of digital learning platforms, they do not view courseware as a substitute for what they value more in their learning: authentic relationships with skilled and caring instructors. Even with the support of adaptive courseware, which adjusts to the learner, learning is a social activity, O’Sullivan says.
“Overwhelmingly, students value a good instructor,” she says. “Someone who tries to make a personal connection with them. Someone who cares about them.”