Select any filter and click on Apply to see results
Table of Contents
Connecting the Dots between Learning and Resources
For the better part of the last fifteen years, leaders in American higher education have been working to improve student success, focusing particularly on assessment of learning outcomes and getting more students to degrees. But progress in translating good intentions into improved educational practices is at risk of being erased by eroding fiscal support. In response to current financial pressures, leaders are asking how to allocate scarce resources for the greatest payoffs in student learning and degree attainment. This is a particularly critical issue for institutions serving at-risk students, as they work to meet low-income students' educational and financial needs with far fewer dollars than their wealthier counterparts.
To better understand what is known and what remains to be discovered about cost effectiveness and the differences money either does or does not make in getting students to degrees with acceptable levels of learning, I created an occasional paper titled "Connecting the Dots Between Learning and Resources" for the National Institute for Learning Outcomes Assessment. In the paper, I present a conceptual approach for analyzing the relation of spending to student success, followed by an examination of existing research on the topic. While the full paper is available online at www.learningoutcomesassessment.org/documents/
Wellman.pdf, I will summarize some key points for Diversity & Democracy readers here.
A Conceptual Framework for Thinking about Cost-Effectiveness
Ideally, to look at cost-effectiveness, one would look at the role of funds in producing educational value added, or the translation of inputs into outputs. Such a look would require better ways of evaluating learning than are currently available, as well as better ways to look at how funds are used within institutions. In the absence of ideal measures, a number of proxies can be created that measure different dimensions of spending in relation to types of outcomes:
- Spending per student or total revenues from all resources
- Spending per student from unrestricted resources only
- Spending per student by functional area (e.g., for instruction, student services, financial aid)
- Spending per student for education and related expenses only (excluding sponsored research and public service and auxiliary enterprises)
- Trends over time in spending and by functional area
- Credits earned
- Completion of twelve units or more
- First-year retention
- Degrees completed
- Certificates completed
- Transfer from a two-year to a four-year institution
- Job placement rates
- Pass rates on examinations such as the GRE or licensure examinations
- Lifetime earnings
Using this framework, one can imagine a number of ways to connect topics on the cost side of the matrix to the outputs side, to look at spending in relation to different measures of performance.
Spending on Academic and Student Services
A few studies have focused on spending and performance as measured by student degree attainment and effective teaching practices. For example, researchers Patrick Kelly and Dennis Jones of the National Center for Higher Education Management Systems (NCHEMS) looked at data for public institutions to see whether spending makes a difference in access to higher education, in state-level degree productivity, or in research funding (2005). Recognizing that these measures say nothing about quality, the researchers found no consistent relationship between levels of spending and measures of performance. Although their major finding was that funding levels overall do not explain differences in performance, Kelly and Jones found that spending on student support services does correlate with higher levels of degree attainment. Their research suggests that the way resources are used may matter as much as or more than the absolute level of funds available.
In another study, NCHEMS researcher Peter Ewell (2003) compared spending per student for instruction and related expenses at two sets of institutions: one set identified by the National Survey of Student Engagement as outperforming its peers in student engagement and retention, and a second set of peer institutions. Ewell found that the more effective institutions did not spend more per student than their peers, but they did spend differently, putting proportionately more money into academic and student support. A follow-up study from Iowa State University found the same thing: total spending levels evidently mattered less to effective educational practices than did the distribution of the resources within the institution (Gansemer-Topf et al. 2004).
Another recent study provides additional support for the theory that student services expenses make a difference in low-income students' retention and degree completion. Using panel data developed by the Delta Cost Project, Cornell Higher Education Research Institute researchers Webber and Ehrenberg (2009) examined how spending in instruction, student services, and other areas influences the graduation and first-year persistence of undergraduate students. The strongest influence they found was from student service expenditures, with the highest marginal effects for students in institutions with low admissions selectivity and high proportions of Pell grant recipients. Simulations of the effect of reallocation from instruction to student services showed enhanced persistence and graduation rates.
Spending on Student Aid
Some consistent themes relevant to the role of student aid in increasing access and student success appear in the research. One finding is that achievement gaps between low-income students and other students are far wider in terms of college completion than in terms of various measures at the point of initial college entry (Kane 2004; McPherson, Schapiro, and Winston 1993; Mortenson 1998; Pell Institute 2004). Most student financial aid programs, however, make access and not degree attainment their primary goal. Work-study programs are an exception, and the research shows that these programs have had some success in increasing degree persistence among low-income students. In addition to providing financial help, these work opportunities increase student interaction with university staff and faculty and cultivate the student's identity as a member of the campus community.
A second consistent finding in the research literature relates to student financial aid. Institutions have rapidly increased their resources for student aid, through both "tuition discounts" and grants. Research shows that the majority of this has gone to "merit" aid, which is distributed primarily on the basis of academic merit rather than financial need (College Board 2008). Since merit aid goes to students who would go to college without it, institutions would better increase college access and persistence by spending their limited dollars in grant aid, in work-study programs, or on programs to enhance student success.
Spending on Faculty and Teaching
As institutions work to contain costs, the use of part-time and contingent faculty has increased precipitously. Some researchers have looked at the relationship between the use of part-time and contingent faculty and measures of student learning. Umbach (2007) surveyed faculty using data from the Faculty Survey of Student Engagement to evaluate the relationship between faculty appointment status and institutional engagement with effective teaching practices. He found that contingent faculty showed less use of active and collaborative teaching techniques, less likelihood of challenging students academically, less likelihood of spending time preparing for class, and less likelihood of interacting with students.
Examining first-to-second-year persistence and graduation rates and use of part-time or nontenured full-time instructors, Ehrenberg and Zhang (2005) found that increases in nontenured and part-time faculty reduced both rates. Using data from Ohio public institutions, Bettinger and Long (2004) also looked at how adjunct faculty affect student interest and course performance. This study found different effects depending on discipline, with an overall slight loss of student interest in subjects for courses taken from graduate students and adjuncts compared to full-time faculty, and with a slight positive effect from use of adjunct professors among disciplines with an occupational or vocational focus.
A promising area of work has been in what Bruce Johnstone calls "learning productivity"--ways to increase learning with either less time or less costly inputs (1995). A good example comes from Carol Twigg (1999) and her colleagues at the National Center for Academic Transformation, who assessed learning and costs for courses delivered through a traditional lecture/discussion session format and compared them to those of courses delivered using distance-learning technology. The primary cost difference came from less time spent in course and materials preparation and from labor savings by substituting low-cost "coaches" for faculty and teaching assistants. The researchers found superior results at reduced costs for the technology-enhanced courses.
The revenue meltdown of 2008 and 2009 has meant that "learning productivity" efforts are expanding rapidly through the efforts of several state systems and other college leaders. In the University of Maryland system, a comprehensive initiative to tackle inefficiency and to increase effectiveness included limitations on reimbursements for credits earned above the 120 units required for the degree, and a requirement that all students earn at least one semester's worth of credits through some form of off-campus instruction: credit by examination, study abroad, or distance-based courses.
Reconnecting the Threads
Although more research is sorely needed, threads of an emerging consensus begin to appear in what is known and what might reasonably be surmised from the research:
- Intentionality matters as much as, or more than, money alone. Institutions whose leaders put resources behind instruction and student services show greater rates of persistence and graduation per dollar spent. If the national priority is to increase academic attainment, more can and should be done to focus on student learning and degree attainment as the first priority for resource use.
- Focusing resources on instruction and student services helps to increase learning, retention, and degree attainment. Investments in faculty resources make a difference in student learning. Student services investments are especially important for increasing retention at institutions serving large proportions of at-risk students. Yet the pattern among public institutions across the country has been to disproportionately reduce funding in instruction and student services. Deliberate efforts must be made to protect funding for these resources.
- Student financial aid programs need to be restructured to support the goal of student degree attainment as well as access. Student aid makes the biggest difference in access for low-income students, but less of a difference in student success. Grant aid allowing students to attend college full time and increased funding for on-campus work programs can help improve retention and graduation.
- Excess credits and student attrition cost money and do not help students get to the finish line. Curriculum realignment, aggressive academic counseling, and attention to course scheduling can all help increase student success at reduced cost, both to the student and to the institution. Redesigning curricula to ensure coherence and to focus on learning results can be cost-effective if done with an eye on spending as well as on student success, and if accompanied by attention to student and academic support services aligned with the goal of increased learning success.
Our country is faced with the daunting challenge of increasing educational attainment despite negative funding patterns that threaten access and quality. We cannot meet these attainment goals unless we are successful in closing achievement gaps all across the educational pipeline. We need an investment strategy to do this, one that supports our educational and attainment goals. While the findings from this research are still quite tentative, they can help point the way to spending decisions that will make a difference in increasing success for all students.
This article has been adapted with permission from the National Institute for Learning Outcomes Assessment.
Bettinger, E., and B. T. Long. 2004. Do college instructors matter? The effect of adjuncts and graduate assistants on students' interest and success. NBER Working Paper No. 10370. Cambridge, MA: National Bureau of Economic Research.
College Board. 2008. Trends in student aid. New York: College Board. professionals.collegeboard.com/profdownload/trends-in-student-aid-2008.pdf.
Ehrenberg, R. G., and L. Zhang. 2005. Do tenured and tenure-track faculty matter? Journal of Human Resources 40 (3): 647-59.
Ewell, P. 2003. Do DEEP institutions spend more or differently than their peers? Boulder, CO: National Center for Higher Education Management Systems.
Gansemer-Topf, A., K. Saunders, J. Shuh, and M. Shelley. 2004. A study of resource expenditures and allocation at DEEP colleges: Is spending related to student engagement?Ames, IA: Iowa State University, Educational Leadership and Policy Studies.nsse.iub.edu/pdf/DEEP_Expenditures_Schuh.pdf.
Johnstone, B. 1995. The imperative of productivity in higher education. Learning Productivity Network. gse.buffalo.edu/org/LPN/lpn.htm.
Kane, T. J. 2004. College going and inequality. Social inequality, ed. K. M. Neckerman, 319-54. New York: Russell Sage Foundation.
Kelly, P. J., and D. Jones. 2005. A new look at the institutional component of higher education finance: A guide for evaluating performance relative to financial resources. Boulder, CO: National Center for Higher Education Management Systems. www.nchems.org/
pubs/docs/Policy Guide Jan2007.pdf.
McPherson, M., M. O. Schapiro, and G. Winston, eds. 1993. Paying the piper: Productivity, incentives, and financing in U.S. higher education. Ann Arbor, MI: University of Michigan Press.
Mortenson, T. 1998. Students from low-income families and higher educational opportunity.Postsecondary Education Opportunity 78, 1-2.
Pell Institute for the Study of Opportunity in Higher Education. 2004. Indicators of opportunity in higher education. Fall 2004 status report. Washington, DC: Pell Institute for the Study of Opportunity in Higher Education. www.eric.ed.gov/ERICDocs/data/ericdocs2sql/
Twigg, C. A. 1999. Improving learning and reducing costs: Redesigning large-enrollment classes. Troy, NY: National Center for Academic Transformation.www.thencat.org/Monographs/ImpLearn.html.
Umbach, P. D. 2007. How effective are they? Exploring the impact of contingent faculty on undergraduate education. The Review of Higher Education 30 (2): 91-123.
Webber, D. A., and R. G. Ehrenberg. 2009. Do expenditures other than instructional expenditures affect graduation and persistence rates in American higher education? Ithaca, NY: Cornell Higher Education Research Institute. www.ilr.cornell.edu/cheri/