Handbook for Improved Assessment of Student Learning Outcomes in Short-Term Faculty-Led Study Abroad

The following report is an analysis of Roosevelt University’s Office of International Programs (hereafter referred to as the “OIP”) assessment and evaluation for short-term faculty-led study abroad programs. The document is comprised of three main sections: (1) the Description of Client and Project; (2) the Assessment Proposal; and (2) the Design Proposal. The Description of Client and Project section details the context of the project: students at the Middlebury Institute of International Studies (Courtney Jackson, Jacob Jallah, Tobias Joel hereafter referred to as “the team”) have sought out collaboration with Roosevelt University in order to gain practical industry experience. The team conducted a needs assessment to understand how best to contribute to the OIP. The most pressing need identified was improving the OIP’s assessment and evaluation of short-term faculty-led programs. Following the Description of Client and Project section, the Assessment Proposal introduces the research questions of the project: - Research Question #1: How might we improve the OIP’s assessment and evaluation techniques? - Research Question #2: What type of programmatic changes can increase participants’ global awareness? The Assessment Proposal goes on to address Research Question #1 by analyzing the OIP’s current assessment and evaluation techniques. First, the team introduces our general approach by describing our Theory of Change and Project Logic Model. These graphics provide a general roadmap to our consultation and, on the logic model in particular, a suggested timeline for implementation. Following that, key programmatic documents—the Study Abroad Program Evaluation survey and the Faculty-Led Program Proposal Form—are analyzed to capture the nature of OIP’s current assessment and evaluation measures. The Design Proposal addresses Research Question #2. This section consists of recommendations to improve the OIP’s assessment and evaluation. The recommendations are divided into three sections: (1) pre-program changes; (2) in-program changes; and (3) post-program changes. The unifying theme of the recommendations is that effective assessment comes from institutional behaviors, not simply tools. Improving assessment requires placing it at the base of the OIP course design. The Design Proposal discusses using AAC&U’s rubrics to clarify learning outcomes. With more transparent goals, the OIP can pursue targeted techniques to make assessment a natural, ongoing part of its programs. The Design Proposal details this broader context as well as concrete techniques to improve the OIP’s assessment and evaluation. Finally, a concrete timeline of proposed changes is referred to in our program logic model (see Assessment Proposal). The Additional Resources & Recommendations section provides additional background regarding measuring global awareness. Implementing these additional recommendations is beyond the scope of the current study, but should comprise a useful reference for future OIP decisions. The team’s third-party perspective can help the OIP improve its strategies, but realizing programmatic changes involves institutional issues beyond the team’s influence or knowledge. This is a limitation of the study. Nevertheless, the team’s distance from the OIP’s institutional culture also gives the team a new perspective on OIP challenges. The goal is to share this fresh perspective with the OIP and promote new strategies for increasing students’ global awareness through short-term faculty-led programs.