Maintaining Validity in Online Assessments

For the most part, the same principles that apply to assessments designed for use in-class also apply to assessments designed for the online environment. The most important consideration in any assessment design is validity, which is not a property of the assessment itself but instead describes the adequacy or appropriateness of interpretations and uses of assessment results. Below, I explore three considerations about validity that faculty and assessment professionals should keep in mind as they design curricula, assignments, and assessments in their new teaching environments.

Align Learning Outcomes with Assignments and Assessments

Alignment is one of the most important considerations when addressing the validity of assessment results. There should always be mutually supportive connections among an assignment’s intended student learning outcomes, instructional activities that help students meet those outcomes, and the assessment instruments used to test the results. For example, assessments can erroneously deflate or inflate performance scores if they measure proficiency on material that hasn’t been covered in class, or, conversely, if they do not cover content that is representative of the class.

Backward design is one way to ensure alignment. Instructors start with the specific learning objective for a class or a unit and work backward to create lesson plans to achieve those desired goals or objectives. Assessments can then be adapted and linked to these lesson plans. Moreover, using a test blueprint can help faculty ensure that there is alignment between test items and instructional objectives.

Assess All Levels of the Cognitive Domain

Educators also need to be sure that assignments and assessments match the various ways that students learn, avoiding an overreliance on tests that include trivial questions to examine low levels of the cognitive domain. For example, if assessments measure only the lower levels of the cognitive domain (e.g., by calling for students’ recognition of facts), then students will most likely adapt their learning strategies to meet these lower levels (e.g., through memorization). There are multiple authentic assessment approaches (e.g., using AAC&U’s VALUE rubrics to evaluate student projects) that can be adapted to the online format instead of relying on simple multiple choice or recognition tests. Relatedly, test security is often cited as a challenge associated with online learning, and assessment formats that measure learning at higher levels of the cognitive domain discourage cheating, since more open-ended approaches make it more difficult for students to give the same verbatim response as their peers.

Provide Equitable Opportunities to Learn

For assessment results to be valid, all students need to have adequate opportunities to learn the material and prepare for assessments. The COVID-19 pandemic has highlighted persistent achievement disparities across income levels as students who do not have access to digital technology or internet service struggle to complete their work. Faculty and university leaders need to step in to determine appropriate accommodations to ensure that all students can learn.

As the COVID-19 pandemic continues, more changes to assessment are on the horizon. While the assessments we use to measure student learning may change, issues of quality such as validity, reliability, and fairness should always be constant considerations.

Video Discussion on Assessment of Online Learning

This multimedia series is coordinated by M. David Miller (University of Florida), Tammie Cumming (Brooklyn College, CUNY), Gladys Palma de Schrynemaker (CUNY School of Labor and Urban Studies), and Terrel Rhodes (AAC&U).

Have an idea for a blog post? Write to [email protected].