AAC&U News, August 2016
Facts & Figures

New Report Shows Mixed Results from Adaptive Software

Adaptive learning technology, in which software tracks the progression of individual students through a class and tailors coursework to suit each student’s needs, had no significant effect on the rate of course completion and a slight positive effect on student grades, according to a report on its usage at fourteen colleges and universities. The report, issued in April 2016, was commissioned by the Bill & Melinda Gates Foundation to analyze the effects of the Adaptive Learning Market Acceleration Program (ALMAP), also funded by the Gates Foundation. The ten bachelor’s-degree-granting programs and four associate’s-degree-granting programs involved in the program—reaching a total of more than 19,500 unique students and more than 280 unique instructors—implemented the software in ways that were organized into three categories: lecture vs. blended adaptive, in which “adaptive courseware was used as part of a shift from traditional lecture to blended instruction”; online vs. online adaptive, in which “adaptive courseware was used as an enhancement to existing online courses”; and blended vs. blended adaptive, in which “adaptive courseware was swapped into face-to-face courses already using blends of classroom-based and online approaches to support learning” (SRI Education 2016).

Participating institutions compared learning outcomes and course costs between courses that used adaptive software and courses that did not. Evaluators found that although adaptive courseware implementations led to slightly higher grades in four of the fifteen courses for which sufficient data existed, in the remaining eleven there was no identifiable impact on students’ grades, according to the report. Of the sixteen courses for which there were sufficient data on course completion, the use of adaptive courseware did not seem to have any substantive impact; still, many students, especially those in two-year institutions, reported having positive experiences with the software.

The Effect of Adaptive Courseware on Student Performance

  • In seven controlled side-by-side comparisons of scores on common learning assessments, adaptive courseware was found to have a “modest but significantly positive” effect (SRI Education 2016).
  • Moving to adaptive blended instruction from a traditional lecture format had a positive impact on student learning; using adaptive learning software instead of nonadaptive software in fully online courses had a “small positive effect on course grades” (SRI Education 2016).
  • Overall, no significant average impact on course completion was found in any of the available data.

Cutting Costs?

  • Costs typically increased during the first term in which adaptive courseware was implemented, although “the adaptive courseware was associated with lower ongoing costs” in seven out of the ten cases with adequate cost data (SRI Education 2016).
  • Of the eight cases in which there were sufficient data on both learning outcomes and cost, five cases had cut costs but only one of those saw improved learning outcomes.

Student and Faculty Response

  • Sixty-seven percent of remedial and developmental course instructors planned to use adaptive courseware again in the future, while only 49 percent of gateway general education course instructors indicated that they would.
  • Overall, 74 percent of instructors reported being satisfied with the adaptive software they used.
  • Seventy-seven percent of students at two-year colleges said that their learning had improved after using the adaptive software, while 51 percent of students at four-year institutions reported making positive learning gains.
  • Fifty-six percent of students at two-year institutions reported being satisfied with their overall adaptive software experience, while 33 percent of students at four-year colleges and universities reported the same.

Did You Know?

  • Adaptive software seemed to have the most significant impact in mathematics and biology courses.
  • The impact of adaptive software on Pell Grant students seemed to be the same—not more or less beneficial—as its impact on non-Pell Grant students.
  • Ninety-five percent of students in developmental courses reported making learning gains with adaptive software, while only 35 percent of students in gateway general education courses reported the same.

The full report, Lessons Learned From Early Implementations of Adaptive Courseware, prepared by Louise Yarnall, Barbara Means, and Tallie Wetzel and published  by SRI Education in April 2016, is available online. Readers interested in approaches to student learning and new digital technologies should see also the new AAC&U publication Open and Integrative: Designing Liberal Education for the New Digital Ecosystem.

 

About AAC&U News

AAC&U News is written and edited by Ben Dedman. If you have questions or comments about the newsletter's contents, please e-mail dedman@aacu.org.

MEMBERSHIP

Join AAC&U's Associates Program
Please contact Dennis Renner at memberservices@aacu.org for questions about AAC&U membership.