Talking Points: AAC&U 2009 Member Survey Findings

Talking Points on Trends in Learning Outcomes, Assessment, and General Education

While the public and the media are focused primarily on cost issues in higher education, American colleges and universities are also in the midst of a significant reinvention of the undergraduate curriculum to provide all students with a broad set of skills and areas of knowledge that will prepare them well for success in today’s competitive global economy and to contribute to building a successful participatory democracy.

Nearly 80 percent of colleges now have a broad set of learning outcomes that apply to all students. These stated learning outcomes include a wide array of cross-cutting skills and areas of knowledge, including many on which earlier surveys suggest employers want colleges to focus. The skills most widely addressed in college and university goals are writing, critical thinking, quantitative reasoning, oral communication, intercultural skills, information literacy, and ethical reasoning. The knowledge areas most often required for all students are humanities, sciences, social sciences, global cultures, and mathematics.

Contrary to public and policy perceptions, colleges and universities are also not resisting the move to assess learning outcomes beyond the use of grades. They are, however, developing multiple ways to assess outcomes, and moving toward qualitative approaches rather than standardized measures—especially content-knowledge tests.  College leaders are embracing forms of assessment that they perceive as effective in evaluating college-level achievement of higher order skills. They also are embracing assessment approaches that measure student progress over time rather than just at one or two individual points in time.

This survey shows that many colleges of all sorts are also in the midst of reforming their general education programs—the part of the curriculum required of all students—and are developing new curricular approaches and ways to assess key learning outcomes in general education.

Colleges are changing their curricula to meet the needs of a changing world and they are working hard to assess students’ achievement of a wide array of important learning outcomes.

Assessing Learning Outcomes
Despite what you might think from the last year or two of public debate about accountability, colleges and universities have overwhelmingly embraced the need to assess learning outcomes across the curriculum beyond just grading. More than 7 in 10 (72 percent) of AAC&U member institutions now assess learning outcomes across the curriculum and an additional one in four (24 percent) say they are planning for this assessment.

Most colleges assess learning outcomes at the departmental level rather than in general education. Nonetheless, nearly half (48 percent) are assessing at both the departmental level and in general education.

Colleges and universities are using a variety of approaches to assessment. Forty percent use rubrics applied to student work; 37 percent use capstone projects; 35 percent use student surveys. Relatively few are using standardized national tests of general knowledge (16 percent) and about a quarter (26 percent) use standardized national tests of general skills, such as critical thinking.

High-Impact Educational Practices
More and more campuses are focusing attention on providing students with integrative and engaged forms of learning. Interest has especially grown in the areas of undergraduate research, capstone projects, and electronic portfolios.

Nearly all institutions now offer capstone projects, with most making them available in departments rather than in general education. Nearly 40 percent of those polled require capstone projects of all or most students in departments.

More than half of those surveyed now use electronic portfolios but few are requiring them of all students. Fifty-seven percent use electronic portfolios in some form but only 42 percent use them for assessment purposes.

Many institutions surveyed are placing more emphasis on practices that educational research has shown to be particularly effective, and many are doing so as part of revisions to general education. Seventy-eight percent are placing more emphasis on undergraduate research; 73 percent are placing more emphasis on first-year experiences; and 52 percent report placing more emphasis on learning communities.

Trends in General Education
Colleges are moving away from a purely cafeteria-style general education requirement model, with only 15% percent now using this model exclusively.

Most institutions (more than two-thirds) now use a model that combines course choice with other integrative features like learning communities (used by 24 percent) or thematic required courses (used by 36%).

Eighty-nine percent of institutions are in some stage of either assessing or modifying their general education program.

Fifty-two percent are currently assessing learning outcomes in general education, with another 42 percent planning for such assessment.

Over the past five years, campuses are placing increased emphasis on a variety of practices including undergraduate research (78 percent more emphasis), first year experiences (73 percent), study abroad (71 percent), service learning (68 percent) and internships (62 percent).

Those surveyed rate their own campuses’ general education program well in the following areas: includes global courses (60 percent); includes first-year seminars (58 percent); includes diversity courses (56 percent); includes interdisciplinary courses (51 percent); includes civic learning (38 percent).

Those institutions using a general education model that includes some integrated features (e.g. learning communities, first-year seminars, etc.) are much more likely to have stated learning outcomes for all students, to report greater integration of general and major requirements, and to use capstones or other high-impact practices.

Areas in Need of More Attention
While colleges have taken the first step to articulate their learning outcomes for all students, they recognize they still have work to do in communicating the importance of these outcomes to all students. Of those with outcomes, only 5 percent say that they think almost all their students understand their institution’s intended learning outcomes.

While many of the trends documented in this survey suggest that campuses are moving in the direction suggested by educational research and in ways consistent with what employers recommend, there are still areas where colleges could do more to ensure that students have the skills and knowledge they need, particularly for success in a volatile global economy. For example, while earlier AAC&U surveys of employers indicate that they wish colleges would place more emphasis on learning in real-world settings, only 36 percent of academic administrators currently rate their own general education programs well (4 or 5 on a 5-point scale) for including experiential learning opportunities. These institutions may, indeed, be providing some students with these experiences either through independent service learning programs or through internships. But this study suggests that most institutions are not moving to incorporate these practices into general education—that part of the curriculum that all students must complete.

While many academic administrators give their general education programs high marks for having clear goals and having requirements linked to those goals, fewer rate their programs highly in terms of assessing student achievement of general education goals. They are least likely to describe their general education programs as having a coherent sequence of courses. Less than half of member institutions feel that their general education programs are well integrated with students’ major requirements.

We know that issues of civic engagement and ethics are highlighted in many campuses’ mission statements, but this data suggests that for too many campuses, this mission-level commitment is not yet fully addressed in general education. About 59 percent of those surveyed, for instance, name ethical reasoning as a common learning outcome and even fewer (53 percent) name civic engagement as a common learning outcome important for all students.

The diversity of our nation and the increasingly interconnected global communities of which we are all a part continue to suggest the importance of diversity, global knowledge and intercultural skills as essential outcomes for all college students. Despite this reality and the endorsement of these particular outcomes as essential by educators, policy makers, and the business community, too many colleges and universities still have not prioritized these learning outcomes for all students.  Only 57 percent of colleges and universities name understanding diversity in the United States as a learning outcome for all students. Slightly more (62 percent) name intercultural skills as an important outcome for all students.

Survey Administration and Sample Profile
The sample for this survey is representative of AAC&U’s total membership and includes chief academic officers at public and private institutions, 2-year and 4-year institutions, and small and large institutions. Forty-four percent of those surveyed were at public institutions and 55 percent were at private institutions (including independent and religious institutions).

From November 19, 2008, to February 16, 2009, Hart Research conducted an online survey among 433 Chief Academic Officers or designated representatives at AAC&U member institutions to measure the prevalence of specified learning outcomes in higher education institutions today and to document recent trends in curricular change, specifically in the areas of general education and assessment. The margin of error is ±4.7 percentage points for the entire sample, and it is larger for subgroups. The total population for the survey included 906 AAC&U member institutions that were invited to complete the survey, and thus the response rate for the survey is 48%.