The LEAP Challenge Blog
They'll Love You Just the Way You Are: Tiptoeing Toward Transparency
“When you come to a fork in the road, take it” (Yogi Berra)
A few years ago, the National Survey of Student Engagement (NSSE) and USA Today arranged for schools to post their NSSE benchmark scores on the USA Today college website. While the Community College Survey of Student Engagement had championed public reporting since its inception, this NSSE-USA Today relationship marked the first time that several hundred four-year schools took the leap of faith and made their student engagement results available to a national audience. More recently, national associations developed templates for their member schools to use to report cost, NSSE or other student experience measures, and—in some instances—selected student learning outcomes. To my knowledge, no institution has closed or been otherwise adversely affected by making public these kinds of student or institutional performance measures.
It's almost certain that in the future, colleges and universities will be expected to provide much more information about what and how much students learn during college. Institutions are not of one mind, of course, about whether and how to do this. There are justifiable worries about people drawing erroneous conclusions from data. Another risk is that making visible all our laundry—some clean and some not so clean—will have the adverse effect of stifling candid internal discussions about where improvements need to be made and will discourage efforts to address such shortcomings. These concerns are real and not trivial.
My colleagues at the National Institute for Learning Outcomes Assessment (NILOA) are proponents of public reporting, provided it is done appropriately. To advance this cause, NILOA just last week made available a Transparency Framework. It’s a tool that institutions can use to evaluate how well their websites are communicating learning outcomes assessment information to the intended audiences. Unlike association-branded templates or state-level accountability platforms that array student and institutional performance results on external sites (which makes comparisons possible), the Transparency Framework is a guide for individual institutions to examine and rethink how they are communicating what they consider to be relevant information via their websites.
We realized that something like the Transparency Framework could be helpful after reviewing one thousand two- and four-year institutional websites over the past couple of years, looking for information offered by colleges and universities about their student learning outcomes assessment activities. We found much less information about these matters on websites than what chief academic officers at the institutions told us was happening on their campuses. In fact, much of the information about student learning assessment was located on pages intended for internal audiences, such as the Institutional Research or Chief Academic Officer pages. Many times we had to search through long reports or self-studies to find references to assessment findings. At the same time, we also found institutions that apparently have worked through many of the challenging questions associated with providing information they determined would be of legitimate interest to different constituencies on and off the campus.
Toward this end, the Transparency Framework is intended to guide schools in systematically reviewing the structure and content of their websites related to measuring student learning and using the results to improve. The goal is to provide meaningful, understandable information about student accomplishment and how the institution obtains such data, while at the same time being true to the institutional mission and values. The framework is made up of six components:
- •Student learning outcomes statements
- •Assessment plans
- •Assessment resources
- •Current assessment activities
- •Evidence of student learning
- •Use of student learning evidence
By examining the extent to which the institution’s website is consistent with what each of these components represents, a college or university can determine what it needs to do to be transparent with regard to student learning assessment. To illustrate what transparent websites look like, the NILOA website has examples for each of the transparency components of institutions that appear to being doing a good job of clearly communicating their student learning assessment activities to both external and internal audiences. We’ll add more examples as they become available. A quick glance at those posted now makes plain that schools take different approaches to sharing meaningful information—a function of their different missions, educational programs, and nature of assessment activities and results. In this vein, it makes sense that schools adapt the components of the framework in ways that match their circumstances.
Because we intend to refine the framework based on the experiences of institutions that use it, we welcome suggestions to improve it as well as other ideas for what NILOA can do to help advance the assessment agenda.