Membership Programs Meetings Publications LEAP Press Room About AAC&U
Association of American Colleges and Universities
Search Web Site
AAC&U
Resources on:
Liberal Education
General Education
Curriculum
Faculty
Student Success
Institutional Change
Assessment
Diversity
Civic Engagement
Women
Global Learning
Science & Health
PKAL
Connect with AACU:
Join Our Email List
RSS Feed
Facebook
Follow us on Twitter
LEAP Blog
LEAP Toolkit
YouTube
Podcasts
Support AACU
Online Giving Form
 
Peer Review Spring 2009 Cover  

Spring 2009, Vol. 11, No. 2

Connecting Beliefs with Research on Effective Undergraduate Education

By Ross Miller, senior director of assessment, Berkeley College, and former senior director of assessment for learning, Association of American Colleges and Universities


In support of its mission, AAC&U’s projects and publications, institutes, and network meeting all focus on improving undergraduate student learning. Both the recent Greater Expectations initiative and the current Liberal Education and America’s Promise (LEAP) campaign advance the belief that higher education should strive for higher levels of learning for all students. But reports of low graduation rates and of graduates with inadequate knowledge and skills may cause some people, both on and off campus, to question whether holding idealistic beliefs such as those associated with Greater Expectations and LEAP is reasonable. Some skeptics ask whether higher education is hopelessly out of touch with a world in which some students are “college material” and others simply are not.

Faced with difficult choices, we often rely upon deeply held beliefs, attitudes, or philosophies to help us make decisions. When our decisions affect other people, as decisions in education often do, it is important that the beliefs behind those decisions are conscious and have been examined in order to avoid both unwise and arbitrary choices. From setting goals and expectations for learning to addressing the hard realities of the classroom, our beliefs about teaching, learning, and the purposes of education have a profound influence.
AAC&U’s advocacy for higher levels of learning for all students is not misplaced. Education research demonstrates that we can improve student learning—sometimes dramatically—and that maintaining idealistic and inclusive beliefs about and goals for higher education is perfectly reasonable.
Beliefs about who should be in college

Far-reaching global, economic, and technological developments have converged to make postsecondary learning an imperative for almost everyone.
—AAC&U

Through efforts like Greater Expectations and LEAP, AAC&U has argued consistently that all students should have access to excellence in higher education, regardless of background or intended field of study. But questions arise, both on and off campuses, about whether all students can learn at the college level and whether everyone should attend college. Based upon an extensive review of education research, Gardiner (1994, 98) concluded that all students can be educated to high levels.

Research…coupled with modern educational methods and quality improvement principles, can enable us for the first time in human history to educate all of the people to a high level. We will, however, have to use, rather than ignore, research.

Studies Gardiner reviewed also showed that “Using mastery learning…the researchers consistently achieved a full one-sigma increase in assessed learning over conventional instruction.” (98) A full standard deviation increase (i.e., one sigma) is very significant, equivalent to moving from the 50th to the 84th percentile. (With mastery learning, students and teachers focus on current topics until students reach a high level of achievement, moving to subsequent lessons only after “mastery” is achieved. Mastery learning is supported by careful curricular sequencing and feedback from frequent assessments that helps students reach the desired level of learning.)

Additionally, Gardiner (97) reported that mastery learning techniques were especially helpful to students with the lowest initial achievement: all students improved, the lowest achievers improved the most, and the learning gaps among students decreased. George Kuh (2008, 18-19) showed a similar differential boost for initially low-achieving students who experienced engaging pedagogies such as learning communities, internships, and senior capstone seminars.

Two of the practices used in mastery learning include pretesting and monitoring students’ progress during a cycle of learning—powerful instructional practices deserving of more consistent use in higher education. Other effective practices suggested in Walberg and Paik (2000) include graded homework and cooperative learning. Research on mastery learning and selected effective practices confirms that holding high expectations for students actually helps them achieve.

But what about the should question? According to Carnevale (2000), from 1998 to 2008, 14.1 million new jobs will require a bachelor’s degree or some form of postsecondary education—more than double those requiring high school level skills or below. Given those data, it makes sense to encourage all students to continue their education past high school. Consistent high expectations for all students to take a challenging high school curriculum and prepare for college (or other postsecondary education) benefit everyone. Our current practices of holding low expectations for many students result in far too many dropouts or graduates unprepared for college, challenging technical careers, and lives as citizens in a diverse democracy.

Beliefs about Goals

The first object of any act of learning…is that it should serve us in the future.
—Jerome Bruner

It is no secret that much material covered in college courses is neither remembered nor used. Bransford, Brown, and Cocking (2000, 16) write of college physics students whose performance in a game requiring an applied understanding of momentum was no better than that of elementary school students. Such an outcome suggests that these students’ learning in physics was not serving them even in a simple problem-solving context, let alone as a foundation for future sophisticated learning or performance.

Bruner implies that, in setting goals, we should consider the question, “to what end?” As part of its LEAP initiative, AAC&U advocates for a broad set of “essential learning outcomes” (AAC&U 2007) in areas of knowledge, intellectual and practical skills, personal and social responsibilities, and integrative learning. These outcomes prepare students to be effective workers and citizens in a diverse democracy—an end to college education worthy of our efforts.

Revealing objectives to students is more than just a courtesy—it improves achievement. Gardiner (1994, 24) reports that

Knowing clearly what desired outcomes should be and having specific and timely knowledge of actual results achieved contribute powerfully to improving performance….Students need to know what they should know and be able to do and, on a regular basis, how well they have succeeded in their efforts.

Giving students detailed assignment descriptions and/or rubrics with criteria for success can clearly inform them of the goals for work they are to complete.

Bransford, Brown, and Cocking (61 ) add another dimension:

Learners of all ages are more motivated when they can see the usefulness of what they are learning and when they can use that information to do something that has an impact on others—especially their local community.

Whether you think you can or think you can’t—you are right.
—Henry Ford

Closely related to goals for learning are the expectations for learning that teachers hold for students and that teachers hold for their own teaching ability. When teachers communicate expectations to students—whether high or low—students tend to meet those expectations. (Cotton 1989 citing Good 1987). Schilling and Schilling (1999) document an increase in the time seniors spend on academic work when their institutions established high expectations during their first year in college.

Teachers’ opinions of their own teaching skills also are critical. “Teachers who produce the greatest learning gains accept responsibility for teaching their students. They believe that students are capable of learning and that they (the teachers) can teach them.” (Alkin 1992, 1375) Teachers who doubt their own efficacy often exert little effort in reshaping instruction to help students.

Teachers with a high sense of efficacy adapt instruction to student characteristics and show a high level of tolerance for students’ varied learning approaches. A commitment to high expectations for students, then, requires ongoing faculty development to enhance teachers’ own ability and confidence to assist students with varying levels of preparation.

Students and teachers attribute successes and failures either to changeable or unchangeable factors such as effort and ability (respectively). Lumsden (1997) notes that

In the U.S….innate ability is viewed as the main determinant of academic success. The role played by effort, amount and quality of instruction, and parental involvement is discounted…Poor performance in school is often attributed to low ability, and ability is viewed as being immune to alteration, much like eye or skin color. Therefore, poorly performing students often come to believe that no matter how much effort they put forth, it will not be reflected in improved performance.

It is a peculiarity of American culture that we assign so much of a learner’s success to his or her innate ability. Education research in Japanese and Chinese schools shows that teachers and students alike attribute their successes and failures more to training received and effort expended than to ability (Stevenson and Stigler 1992, 94–112). When teachers believe that hard work is important to success and consistently convey this belief to students, they communicate both high expectations for student learning and pull an effective lever to improve results.

Beliefs About Teaching and Assignments

You learn what you do and damn little else.
—Charles Leonhard

Improving student learning relies on alignment—planning assignments, assessments, and revisions to ensure that goals are achieved. Since “you learn what you do,” it makes sense to require students to actually practice what we expect them to achieve. For example, if critical thinking is a goal, students need to do more than read an article in which the writer displays critical thinking. Students must be led to think critically for themselves. If consistent, high-quality achievement is expected, students must think critically multiple times and at increasing levels of sophistication. Students’ experiences must be consistent with their goals. As Gardiner (23) puts it, “students learn what they study.”

“Doing” is a kind of engagement and has been shown to be positively associated with student achievement. In their massive review of how college affects students, Pascarella and Terenzini (1991, 610) write, “One of the most inescapable and unequivocal conclusions…is that the impact of college is largely determined by the individual’s quality of effort and level of involvement in both academic and nonacademic activities.”

Bruner writes that “Teaching that emphasizes structure is probably even more valuable for the less able student than for the gifted one” (1960, 9). He advocates for teaching “structure” at all levels of learning, and Bransford, Brown, and Cocking (2000, 16-17) confirm that students benefit from being able to place knowledge within a conceptual structure. Research on transfer and novices shows that

Experts…always draw on a richly structured information base…(K)nowledge of a large set of disconnected facts is not sufficient…A key finding in the learning and transfer literature is that organizing information into a conceptual framework allows for greater “transfer”; that it allows the student to apply what was learned in new situations and to learn related information more quickly.”

Learning by doing, studying central concepts, and structured support are the focus of “authentic learning”—having students learn through activities and experiences very similar to the kinds of things that experts in a field of study do, even if at a simplified level. Undergraduate research, internships, and student teaching are good examples of this kind of learning. In each case there are challenging problems similar to the problems that experts face to which the students respond, and expert guidance to help students place experiences into conceptual frameworks.

Teachers, both prospective and experienced, are often advised to plan their teaching so that students spend a significant amount of time at the highest levels of the cognitive domain, and to use positive reinforcement whenever possible. Most colleges and universities as well as individual departments and programs advocate for students to learn at the highest levels of the cognitive domain—to analyze, to create, to evaluate. In spite of nearly universal support for such outcomes, studies show that the time spent on these challenging levels of learning is low. Audiotapes from 155 classes at four institution were collected and

Questions asked in class were analyzed for the level of thinking skill required for students to answer them. Memory level…questions accounted for 89.3 percent…Evaluation level thinking…occurred only 0.3 percent to 2.5 percent of the time…As the cognitive level of instructors’ questions rose, the level of students’ responses also rose (Gardiner 1994, 43).

While the small percentage of high-cognitive-level questions is disappointing, the fact that the level of student responses corresponded to the level of questions shows that higher level thinking is not beyond students’ capabilities. Professors, however, could certainly ask more questions (or give more assignments) that elicit high-level thinking—especially if they expect such thinking to be an outcome of their courses.

The term “reinforcement” may dredge up memories of Skinner boxes and dancing pigeons but interestingly, the simple act of giving feedback to students can have a huge impact on learning. Research shows an “effect size” of 1.17 for reinforcement— equivalent to moving an “average” student (50th percentile) up to the 87th percentile (Walberg 1984). Building a learning culture in which students regularly receive feedback and reinforcement has great potential for improving student learning.

Beliefs about Assessment and Improvement of Student Learning

A common mantra about assessment is that its primary purpose is to improve student learning. While assessment can serve a number of different purposes, many efforts to develop campus assessment practices over the last two decades have been defined as assessment for improvement—with advocates promoting a variety of possible approaches and skeptics pointing out real and potential problems. The word “assessment” often means different things to different individuals and within different disciplines, creating fundamental problems in shared understanding across campuses and institutions. Given that developing local assessment methods is quite new for higher education (with serious attention beginning only a decade or two ago), it is not surprising that campuses and individuals still struggle with assessment.

There are, nonetheless, several important research studies that support the development and use of assessment to teach and improve learning. One ideal tactic is to have formative assessments embedded within a teaching-learning-assessment-improvement cycle in such a way that they become a powerful part of teaching—not a separate process added on as an afterthought.

Formative assessment is a process in which critical feedback is given to the learner while she is still in the process of completing learning activities—whether writing a paper, practicing a sonata, or synthesizing a new compound. Bransford, Brown, and Cocking (24) found that

Formative assessments—ongoing assessments designed to make students’ thinking visible to both teachers and students—are essential…formative assessments help both teachers and students monitor progress.

Bransford, Brown, and Cocking conclude (154)

Students may receive grades on tests and essays, but these are summative assessments that occur at the end of projects; also needed are formative assessments that provide students opportunities to revise and hence improve the quality of their thinking and learning.

Formative assessment has been recognized as especially beneficial for students who are experiencing difficulties in learning (Black and William 2-3).

Teacher-education programs commonly suggest that educators teach their students to self-assess. Teaching students to self-assess advances a number of positive goals: as students evaluate their own work they move into the highest level of the cognitive domain, they complete their own formative assessment, and they assume some of the teacher’s assessment load. Bransford, Brown, and Cocking cite developing students’ self-assessment capacities as part of effective teaching (140):

Effective teachers also help students build skills of self-assessment. Students learn to assess their own work, as well as the work of their peers, in order to help everyone learn more effectively.

Data from many different assessment efforts, both formative and summative, can also be analyzed to help improve programs or entire institutions and thereby future student learning outcomes. A summative assessment of student learning (such as a senior capstone project) may become a formative self-assessment for a program, clearly contributing to “the primary purpose of assessment” but on a different level.

Coping with the Realities of the Classroom

Idealistic teachers may get discouraged when confronted with the thorny problems of teaching real students unless, as Gardiner says, they “use, rather than ignore, research.” The research is supportive whether one is motivated by his or her beliefs or simply trying to solve educational problems. Research with potential to address several common classroom frustrations has already been cited. For example, frustrations with low student motivation and too little time spent on assignments may be helped through the use of teaching practices that engage students and through institution-wide efforts to raise expectations for amount of time spent studying. Uneven levels of preparation and/or achievement among students in a class may be addressed through mastery learning techniques and targeted formative assessment. Negative attitudes about requirements (“Why do I have to take this course?”) may improve if we carefully explain program goals, purposes, and rationale. Solutions for many other problems exist in the literature.

While some issues of campus culture may respond to wise application of education research, situations that contradict well-established principles of teaching and learning may require structural, not instructional, changes. For example, many professors have neither studied teaching nor become familiar with educational research. Thus, fulfilling institutional and professional commitment to student learning requires substantive faculty development and an approach to promotion and tenure that rewards time spent on improving teaching to advance student learning outcomes. Similar issues include classes with too many students and faculty with too many courses to teach.

Final Thoughts

Within the college classroom, teachers and students make thousands of decisions that affect learning. It is especially important for both teachers and students to believe that all students can learn at high levels, because—from setting expectations and goals, to choosing and responding to assignments, to shaping assessments—believing otherwise will trigger decisions and actions that result in lower achievement. Having convincing evidence from multiple sources that our chosen actions have the possibility to succeed can help sustain our efforts and result in achieving the important goals in which we believe.

References

Alkin, M. C., ed. 1992. Encyclopedia of educational research. Sixth edition. New York: Macmillan.

Association of American Colleges and Universities. 2007. College learning for the new global century: A report from the National Leadership Council for Liberal Education and America’s Promise. Washington, DC: Association of American Colleges and Universities.

——. 2002. Greater expectations: A new vision for learning as a nation goes to college. Washington, DC: Association of American Colleges and Universities.

——. 2004. Taking responsibility for the quality of the baccalaureate degree. Washington, DC: Association of American Colleges and Universities.

Black, P., and D. William. 1998. Inside the black box: Raising standards through classroom assessment. Phi Delta Kappan, 80 (2). www.pdkintl.org/kappan/ kbla9810.htm.

Bransford, J. D., A. L. Brown, and R. R. Cocking, eds. 2000. How people learn: Brain, mind, experience, and school. Washington, DC: National Academy Press.

Bruner, J. S. 1960. The process of education. Cambridge, MA: Harvard University Press.

Carnevale, A. P. 2000. Help wanted…college required. Leadership 2000 Series. Princeton, NJ: Educational Testing Service

Cotton, K. 1989. Expectations and student outcomes. (School Improvement Research Series, Close-Up #7). Portland, OR: Northwest Regional Education Laboratory. www.nwrel.org/scpd/sirs/4/cu7.html

Gardiner, L. F. 1994. Redesigning higher education: Producing dramatic gains in student learning. Washington, DC: The George Washington University.

Kuh, G. 2008. High-impact educational practices: What they are, who has access to them, and why they matter. Washington, DC: Association of American Colleges and Universities.

Lumsden, L. 1997. Expectations for students. ERIC Clearinghouse on Educational Management (ERIC Document Reproduction Service No. ED 409 609).

Pascarella, E. T., and P. T. Tenenzini. 1991. How college affects students. San Francisco: Jossey-Bass.

Schilling, K. M., and K. L. Schilling 1999. Increasing expectations for student effort. About Campus 4 (2):4–10.

Stevenson, H. W., and J. W. Stigler. 1992. The learning gap. New York: Summit Books.

Walberg, H. J., and S. J. Paik. 2000. Effective educational practices. Brussels: UNESCO International Academy of Education. www.ibe.unesco.org/en/services/publications/educational-practices.html

Walberg, H. 1984. Improving the productivity of America’s schools. Educational Leadership 41 (8): 24.

A NOTE ON EPIGRAPHS:

Sources for the epigraphs found in this piece are as follows: Association of American Colleges and Universities, College Learning for the New Global Century; Jerome Bruner, The Process of Education; Henry Ford, quote commonly attributed to Henry Ford, www.quotationspage.com; Charles Leonhard, personal communication with the author, attributed by Leonhard to his study of John Dewey’s writings.

spacer