Are new approaches to transforming undergraduate learning in science, technology, engineering, and mathematics (STEM) making a difference? If so, how? How do we know? And what next? These are the questions explored in a 1999 report from Project Kaleidoscope, which concluded by making predictions and recommendations for the coming decade (Rothman and Narum 1999). Now that that “coming decade” is here, it is timely to ask how accurate those predictions were and to offer some new recommendations for the next decade.
The predictions made in 1999 addressed a broad range of issues, from faculty to facilities and more. In each of the scenarios for the future that were developed then, the underlying theme was that attention to learning and assessment would be pervasive in the undergraduate STEM learning environment on campuses across the country. One reason we thought this would be the case had to do with the anticipated impact of How People Learn: Brain, Mind, Experience and School, a seminal report published that year by the National Research Council. The report called for the development of academic cultures where deep understanding about how students learn determines how courses and curricula are planned, technologies selected, spaces designed, and faculty recognized and rewarded. Further, it was a report that could be used as a resource for shaping and sustaining such cultures.
There were several other compelling reasons for basing our future scenarios upon the expected emergence a new kind of learning culture. In 1999, there was growing external pressure—from public agencies, accrediting agencies, funding agencies, and the business community—for greater transparency with regard to student learning outcomes. New accreditation practices for engineering education programs were challenging that community of professionals, and many other STEM communities were giving new or renewed attention to student learning outcomes in their specific disciplines. Moreover, the National Survey of Student Engagement was piloted in 1999.
Equally important was the increasing visibility and maturity of the work of pedagogical pioneers—agents of change whose efforts had been supported by the National Science Foundation (NSF) since the late 1980s. Their experiences and expertise were beginning to inform a generation of what dissemination literature calls “early adapters.” There was a growing body of research-based theory and practice about what works in the iterative cycle of exploring, examining, addressing, and assessing undergraduate student learning goals.
So, where are we today? Where and when are conversations about students and student learning taking place within institutions or scholarly communities? How widespread among STEM faculty and their administrative colleagues is awareness of the work of pedagogical pioneers and of the growing body of research on learning and cognitive science? Is it now possible to articulate a general set of goals for student learning in STEM fields on which local efforts can be built and against which they can be compared? And if so, what recommendations can be made for the next decade?
Current conversations covering the range of issues related to student learning are dramatically different from those of a decade ago. There is a growing national consensus about what students should know and be able to do as an outcome of their careers as undergraduate learners. The work of the Association of American Colleges and Universities (AAC&U)—most importantly through its Liberal Education and America’s Promise initiative—has been a significant catalyst in engaging communities in discussions about the kinds of learning needed for a complex and volatile world. Explicit and remarkably consistent goals for student learning have been articulated and promulgated by greatly diverse groups both within and beyond the academy.
The Council on Competitiveness (2005, 76), for example, has called for preparing “a whole generation with the capacities for creative thinking and for thriving in a competitive culture, able to work in multidisciplinary teams, . . . be comfortable with ambiguity, recognize new patterns within disparate data, . . . [and] to be inquisitive and analytical.” The undergraduate neuroscience community has outlined learning goals, including critical thinking and independent thought; communicating effectively in written and oral form as well as with figures, graphs, and through presentation software; and an appreciation of the value of diversity and the ability to work with colleagues from a variety of backgrounds and perspectives (see Wiertelak 2003). The American Chemical Society’s Committee on Professional Training has stated that the outcome of laboratory experiences should “give students hands-on experience with chemistry and the self-confidence and competence to . . . interpret experimental results and draw reasonable conclusions, analyze data statistically and assess reliability of results . . . and communicate effectively in small groups and teams. . . ” (2003, 10).
Are the conversations taking place within national groups changing the dialogue on campuses? One indication that they are is the fact that colleges and universities across the country are beginning to make public their explicit visions of student learning by publishing them on their Web sites. The public announcement of the specific learning outcomes established by Miami Dade College is one of the most recent and most visible examples of the mainstreaming of attention to setting and assessing student learning goals both within and beyond STEM fields (see the article by Eduardo J. Padrón in this issue of Liberal Education).
This brings me to the first of my recommendations for the coming decade, namely that leadership teams on campuses gather and distill lists of learning outcomes articulated by leadership associations such as AAC&U, as well as by other professional and disciplinary societies and corporate leaders, and translate those statements into a coherent and institutionally-appropriate set of goals for student learning that serves their vision for the future. Make those goals, and the actions that advance them, public and transparent.
Pedagogies of engagement
The most compelling evidence of the mainstreaming of conversations about how people learn comes from the field. Just-in-Time Teaching, Problem-Based Learning, and Student-Centered Activities for Large Enrollment Undergraduate Programs are three examples of what Russell Edgerton (2001) has described as “pedagogies of engagement,” and they demonstrate the ways in which attention to how people learn is beginning to transform the undergraduate STEM learning environment.
Just-in-Time Teaching (JiTT) involves dialogue between student and student as well as between student and instructor, much of which occurs outside of the classroom—thanks, in part, to the maturation of electronic technologies. Gregor Novak, one of the JiTT pioneers, says that at the heart of the JiTT pedagogy are pre-instruction, Web-based assignments called “warm-ups.” These are short, thought-provoking questions that, when fully discussed, often have complex answers. Students are expected to develop the answers as far as they can on their own, and then the job is finished by working together in the classroom. These warm-ups are submitted electronically just a few hours before class, giving the instructor (just) enough time to incorporate into the upcoming lesson the insights gained from student submissions. Exactly how the classroom time is spent depends upon a variety of issues such as class size, classroom facilities, and student and instructor personalities.
In a JiTT classroom, students construct the same knowledge as in a passive lecture, but with two important added benefits. First, having completed the Web assignment very recently, they enter the classroom ready to engage actively in their learning. Second, they have a feeling of ownership of their learning because the interactive lesson is based on their own wording and understanding of the relevant issues. “Our goal,” Novak explains (pers. comm.), “is to create and sustain team spirit. We all, students and faculty, work together toward the same objective, that students pass the course with the maximum amount of enduring knowledge, skills, and habits of the mind that are critical for success in STEM communities of learners and practitioners.” Does JiTT work? Consider the following testimony from a student:
It is easy to feel disconnected from a science course as a student. Each day can seem as a new set of notes to take from the instructor’s monologue, another chapter to read, and another problem set to work on, but each unrelated to the previous day—that is, until the exam. The situation changes if the assignments are designed to pose questions that require some real effort and interaction with other students ahead of class, but providing the assurance that the toughest points will be cleared up in the class makes that work worthwhile. Just-in-Time Teaching offers the kind of day-to-day motivation that drives the course forward for me. (Project Kaleidoscope 2007, 2)
Problem-Based Learning (PBL) simulates workplace projects that require mastery of a range of content knowledge as well as the development and application of process skills in an integrative and interesting format. Faculty of the biomedical engineering program at Georgia Institute of Technology, for example, arrived at PBL as the foundation for planning a program from scratch. Theirs was a discipline without a history of pedagogies or a tradition of textbooks. (Admittedly, this can be seen as a luxury when trying to incorporate research on learning into the process of curricular change!) The inherent interdisciplinarity of biomedical engineering drew them into research on cognitive flexibility—that is, the ability to look at problems from a variety of perspectives. Their program was designed to challenge students with the right kind of problems, and the goal was to produce integrative problem-solvers who have the cognitive flexibility to apply engineering analysis and synthesis to problems in the biosciences.
On the impact on student learning, Wendy Newstetter, one of the founding members of the PBL team at Georgia Tech, reports that
solving problems on the frontiers of science that other experts are trying to solve at the same time does two things: it motivates students tremendously, and has a very interesting impact on identity. A major problem with students going into the sciences and being sustained is that they don’t identify with the kind of activity they are being asked to do. They don’t see their own personal identities or lives aligned with science. Whereas, when you give them complicated, multi-dimensional, interdisciplinary problems from the real world, their imagination is sparked. They begin to say, “I can see myself doing this.” So problem solving is about motivation and identity, about engaging students through the excitement and fantasy of trying to solve those problems. (Project Kaleidoscope 2006.
Student-Centered Activities for Large Enrollment Undergraduate Programs (SCALE-UP) is based on the conviction that, in order to understand the science, students must actually do the science as a central activity in studying the science. SCALE-UP overcomes the many barriers to “doing” science in the traditional large lecture class: the isolation of individual students in the crowd of strangers, the competitive atmosphere, and the little one-on-one contact with the instructor. SCALE-UP uses cooperative learning pedagogical techniques with classes of approximately one hundred students, with lecture and lab integrated in technology-rich settings. Active group learning is promoted within consistent groups, and the grading system requires teamwork to ensure that each student in the group—even the really bright ones—benefits from working together.
SCALE-UP pioneer Robert Beichner reports that substantive evaluation—video and audio recordings, interviews, focus groups, pre- and post-tests, and student profiles—has revealed improvements in students’ ability to solve problems and their attitudes toward science and learning, as well as increases students’ conceptual understanding. And SCALE-UP reduces failure rates. When asked what impression a visitor would get from walking into a SCALE-UP classroom, Beichner says,
their impression would be that the learning space looks more like a restaurant than a classroom, or perhaps more like a banquet hall, because there is much noise from the visibly engaged students. They would see the realization of our idea that social interactions between students and their teachers is the “active” ingredient that works for us. From our own experiences and from research on learning, we knew that as students collaborate on interesting tasks they become deeply and personally involved with what they are learning. The doors of the closets and the walls of the classroom are covered with whiteboards—public thinking spaces—to help them share their learning with each other and their instructor. (Project Kaleidoscope 2005)
What we find in examining the core principles of these and other pedagogies of engagement is that each of them can be seen, in some way, as a grandchild or great-grandchild of undergraduate research, which is surely the epitome of a pedagogy of engagement. Each of these contemporary pedagogies is designed to introduce students to the community of practice in STEM fields and to enhance their personal understanding of how scientists, engineers, and mathematicians make sense of our world. In other words, each invites students to assume the identity of the STEM professional. Each seeks to give students confidence in their ability to pursue study in STEM fields, based on successive (and presumably successful) social interactions within a collaborating community. These are communities in which expert and novice learners come together in the same spaces—sometimes physical, sometimes virtual—to engage in the process of discovery as they move from what is known to what might be known. These are pedagogies designed to help students understand the importance of being a part of a collaborating community with a sense of shared purpose.
Another similarity is that, although most began with support from the National Science Foundation, these pedagogies work for all disciplines, serve all institutional types, strengthen the learning of all students, and reflect societal and disciplinary goals for undergraduate learning. They all call upon instructors to reflect deeply on how to get students to take ownership of their learning, how to transform the learning environment from one of passive transmission of information to active construction of knowledge and skills, and how to monitor the progress of their students’ learning.
Despite the recognized national need for the kind of problem-solving, critical-thinking risk-takers formed through these pedagogies of engagement, their adoption is not yet widespread. One significant barrier to the spread of contemporary pedagogies of engagement is institutional culture. The compelling research conducted by Charles Henderson and Melissa H. Dancy (2007, 1) examines why “proven strategies are slow to integrate into mainstream instruction” even though, as they document, STEM faculty understand some of the problems with traditional ways of teaching. Henderson and Dancy conclude that situational characteristics consistent with traditional instruction account for the major impediments. Their suggestion, directed at the broader STEM research community, is really for us all: identify both the situational barriers faculty face—bolted-down chairs, large enrollment classes, the need to “cover” content, student lack of experience with active learning, etc.—and the means to removing those barriers. As they say, “after all, getting the chairs unbolted is often a non-trivial task.”
This brings me to the second of my recommendations for the coming decade, namely that leadership teams gather critical information about interest and expertise relative to pedagogies of engagement within their campus community and about local barriers to promoting and adapting those approaches; design and implement an action plan to overcome barriers in the process of adapting pedagogical innovations to serve learning goals for the students for whom they are responsible.
An oft-cited barrier to the mainstreaming of pedagogies of engagement is the lack of evidence of their efficacy. Over the past decade, however, this barrier has been largely surmounted by building on efforts to link research on STEM learning to STEM teaching—efforts that go back many years, at least to the work at the University of Washington by Arnold Arons and Lillian McDermott (see Narum and Rothman 1999). Although these efforts were present and becoming visible in 1999, the past ten years have seen a significant increase in efforts like those of Nobel Laureate Carl Weiman (2007) to take a scientific approach to pedagogical and institutional transformation.
Among those leading these efforts are a cadre of assessment pioneers, who, with support from the NSF’s Assessing Student Achievement (ASA) program, have been deeply involved in research-based initiatives designed to get inside the learning process and gather evidence to document what works and for which students. Some start from what disciplinary content students should know (the Calculus Concept Inventory, the Geoscience Concept Inventory, and Measuring What Students Know about How to Learn Chemistry). Others approach assessment directly from what skills students should acquire (Assessing Problem-Solving Strategies in Chemistry and Assessing Critical Thinking Skills) or from the STEM literacy perspective (Assessing Students’ Value for Science and Math Literacy).
At a meeting of the ASA community in 2006, project principal investigators were invited to share insights from their experiences with colleagues across the country. They urged STEM faculty to recognize that students know and understand less when they emerge from courses than most faculty think they do; that what we teach, despite our best efforts, is not what students learn or how they learn; that student achievement can be increased with effective assessment; and that you can teach better and enjoy it more if your students are demonstrably learning better. This group of experienced practitioners also thought that the situational barriers that keep others from exploring, adapting, and extending their work could be overcome if three conditions were met. First, STEM faculty and their administrative colleagues would need to realize there is no need to reinvent the assessment wheel—an effort costly in both time and energy. Second, faculty would need to identify themselves as members of the community of STEM pedagogical and assessment practitioners in the same way they feel an identity as a member of a STEM research community. And third, formal opportunities would need to be available at the local level—campuswide, within departments or programs—for conversations about how difficult it is to teach certain concepts or students with different learning styles, conversations that capitalize on the expertise and interest resident in their community by engaging that broader community in asking, what works for our students, and how do we know?
I hesitate to predict what the STEM world will look like in 2019, especially given how different it is today from the way the 1999 Project Kaleidoscope report predicted it would be. And so I will end instead with the third of my recommendations for the coming decade, namely that those with a stake in a robust twenty-first-century undergraduate STEM learning environment step back, take time to work with the communities of which they are a part to agree on outcomes from undergraduate engagement in STEM learning, determine their individual and collective responsibility for ensuring their students achieve those goals, and make it happen.
American Chemical Society Committee on Professional Training. 2003. Undergraduate professional education in chemistry: Guidelines and evaluation procedures. Washington, DC: American Chemical Society.
Council on Competitiveness. 2005. National innovation initiative summit and report: Thriving in a world of challenge and change. Washington, DC: Council on Competitiveness.
Edgerton, R. 2001. Education white paper. http://www.pewundergradforum.org/wp1.html.
Henderson, C. and M. Dancy. 2007. Barriers to the use of research-based instructional strategies: The influence of both individual and situational characteristics. Physical Review Special Topics: Physics Education Research 3 (2): 1–14.
Narum, J. L., and F. G. Rothman. 1999. Then, now and in the next decade. Washington, DC: Project Kaleidoscope.
National Research Council. 1999. How People Learn: Brain, Mind, Experience and School. Washington, DC: National Academies Press.
Project Kaleidoscope. 2005. Student-centered activities for large enrollment undergraduate programs (SCALE-UP): An interview with Robert Beichner. www.pkal.org/documents/Beichner_SCALEUP_Interview.cfm.
———. 2006. Problem-based learning at Georgia Tech: An interview with Wendy Newstetter. www.pkal.org/documents/PBL_GeorgiaTech.cfm.
———. 2007. Just-in-time teaching. What Works: Reports from the Community 4: 2. www.pkal.org/documents/All%20Pedagogies%20in%20one%20document%20for
Rothman, F. G., and J. L. Narum. 1999. Then, now and in the next decade: A commentary on strengthening undergraduate science, mathematics, engineering and technology education. Washington, DC: Project Kaleidoscope.
Weiman, C. 2007. Why not try a scientific approach to science education? Change 39 (5): 9–15.
Wiertelak, E. 2003. Neuroscience education: Goals for the undergraduate program.
Project Kaleidoscope (PKAL) is a leading advocate for what works in building and sustaining strong undergraduate programs in the fields of science, technology, engineering, and mathematics (STEM). An informal alliance, PKAL takes responsibility for shaping undergraduate STEM learning environments that attract students to STEM fields and inspire them to persist and succeed. Such environments give students personal experience with the joy of discovery and an awareness of the influence of science and technology in the world. Resources derived from the work of the extensive PKAL community are available for adaptation by leaders on campuses across the country.
Visit www.pkal.org for more information.
To respond to this article, e-mail firstname.lastname@example.org, with the author’s name on the subject line.