Student Evaluation of Teaching: COVID-19 Considerations for Validity and Fairness

The onset of the COVID-19 pandemic has brought unprecedented change to higher education. In March, many college campuses closed their classrooms, and nearly all in-person courses were moved to a remote environment. Faculty were given the not-so-simple task of transferring their in-person courses to a remote learning modality within days.

The implications of moving pedagogy online raise significant concerns about traditional student evaluations of instructors at many institutions. These surveys, usually administered at the end of each semester, allow students to provide feedback to help faculty inform teaching practices. In addition, these surveys provide institutions with information that can be used, in part, for annual reappointment, promotion, and tenure decisions. If instructors are evaluated in classes that are substantially different from the ones they’d planned, it may be unfair to administer these evaluations in an environment that is primarily beyond their control.

Instructors spend a significant amount of time preparing for their courses well in advance of the start of instruction: they develop instructional materials, create student engagement activities, and construct course-appropriate assessments. According to online learning specialists, preparation of a semester-long course for online instruction using a learning management system takes at least three months; many instructors had just days to prepare. Naturally, challenges are to be expected with such a rapid transition. While this does not mean that students did not receive the instruction required to meet course or program learning objectives, they still underwent a markedly different instructional experience from what instructors originally planned.

As a case study for how the COVID transition has affected the learning experience, let’s look to the American University of Paris (AUP), located in the heart of France’s capital. On March 18, AUP switched all of its courses to remote learning. The university worked swiftly and diligently to support faculty, students, and staff in ensuring continuity in teaching and learning. Several of the university’s academic, administrative, and student support centers were entirely repurposed to assist faculty and students. Like many US-based colleges and universities, AUP made platforms such as Microsoft Teams and Blackboard available. Faculty members wanting to maintain synchronous online learning opportunities opted to schedule additional evening lectures and contact hours. Tests and quizzes were offered through Blackboard or substituted with other assessment measures such as final projects.

As faculty are grappling with moving courses online, students are experiencing challenges with technology access, learning conditions, or financial difficulties—not to mention health concerns for themselves and their loved ones. In this current environment, institutions must seriously reconsider the continued administration of a formal student evaluation of teaching and its use for promotion and tenure decisions.

Furthermore, the evaluation instruments themselves need to be evaluated prior to use in this new learning environment. According to educational measurement experts and generally accepted assessment standards, any assessment or survey instrument constructed and used for evaluation purposes should be developed with an eye toward validity, reliability, and fairness. Providing evidence of these three properties is a nontrivial, multi-step process. In fact, a recent article in the American Association of University Professors Academe Blog questions the validity of using survey items developed for in-person courses in evaluating teaching in a wholly online environment.

In light of these considerations, some institutions have decided to forego administering their student evaluations during the COVID-19 transition. Other institutions have continued their administration with significant modifications to data usage policies. AUP, for example, has temporarily modified the policies that govern the access to, and use of, course student evaluations. Results of student evaluations for individual faculty will be visible only to the faculty member and the office of the provost, and not to the department chair. Faculty will not be asked to include any data from student evaluations conducted during the transition in their applications for promotion, sabbatical, or course release. (French labor laws do not allow for the American tenure system.) The office of the provost will also reserve the option to use these evaluations in aggregate—and only for institutional research and educational assessment.

While it is recommended that formal administration and usage of the student evaluations be put on hold, it is still important for students to be able to provide feedback to their instructors regarding their learning experience in the new remote environment. A formative assessment of the instructor by the students—conducted several times throughout the semester at the end of each instructional unit in an anonymous manner—could be extremely useful for improving pedagogy in the new environment. Many institutions have learning management systems with built-in survey capabilities that allow students to provide anonymous feedback to their instructors.

If this move to a new teaching and learning environment persists, institutions should strongly consider modifying their current evaluations of teaching or building new ones to pilot and evaluate for validity, reliability, and fairness. In the interim, instructors may gain valuable formative feedback from their students using tools already available through their institutions.

Video Discussion on Student Evaluation of Teaching during the COVID-19 Pandemic

Have an idea for a blog post? Write to [email protected].