About the PKAL Series
Intended to challenge the higher education community to think strategically about how best to advance the learning and success of all students in science, technology, engineering, and mathematics (STEM), this series of articles presents a broad array of perspectives on cutting-edge issues affecting contemporary undergraduate education in the STEM fields.
The full series is available online at www.aacu.org/liberaleducation/pkal.
In the National Council on Education and the Disciplines’ (NCED) 2001 Mathematics and Democracy, Lynn Steen vividly declares, “The world of the twenty-first century is a world awash in numbers” (2001, 1). In that volume, Steen and his collaborators articulate a clear call for broad reforms to prepare students for the ubiquitous need for quantitative literacy (QL)—in citizenship, education, professional life, personal finance and health, and even culture. Making certain not to understate the case, Robert Orill warns in his preface, “If individuals lack the ability to think numerically they cannot participate fully in civic life, thereby bringing into question the very basis of government of, by, and for the people” (2001, xvi).
While built on a foundation of mathematical skill, the QL needed by our students is distinct from traditional mathematics in several key dimensions. Where mathematics is intentionally abstract, QL drives toward the specific context of problems. As Steen explains, “The power of mathematics lies in its generality and abstraction, in its ability to rise above specifics. Quantitative literacy, on the other hand, is anchored in real world data” (2004, 34). In addition, QL involves rhetorical choices necessary to communicating arguments (Lutsky 2007). Thus, mathematics training is insufficient for the complete development of applied QL skill, just as QL instruction cannot instill the abstract, formal reasoning skills imbued by the mathematics curriculum. These two endeavors truly complement one another.
Indeed, effective science requires both the abstract qualities of math and the real-world attributes of QL to operate in concert. To take just one high-profile example, consider the Centers for Disease Control and Prevention’s 2004 study, published in the Journal of the American Medical Association (JAMA), estimating that 400,000 American lives are lost each year to obesity. The debate that followed led ultimately to a retraction (and a new estimate of just over 100,000 obesity-related deaths) the following year. While the academic debate focused on formal statistical methods (the original study effectively failed to account for the effects of age in cause of death), the broader public debate centered on a QL question that might have saved the scientific community from a high-profile embarrassment: Is this estimate reasonable? In 2008, the US Statistical Abstract reports just over 2.4 million deaths. Of those, 200,000 were of people younger than age 45, and another 1.4 million were 75 or older. The former seem too young, while the latter seem too old for many to have been related to obesity. That leaves approximately 850,000 deaths in the age group for which obesity would seem to be a major cause of death. In this context, is it reasonable that 400,000 people die of obesity-related deaths each year? Does the simple comparison of 400,000 to 850,000 at least advise a reexamination of methods before publication in JAMA?
But QL is not just about preparing scientists. Its reach extends to the public, professional, and personal lives of all our students. Whether making informed political decisions when voting on candidates who advocate differing economic policies, weighing the costs and benefits of expanding or shrinking a firm, or determining an appropriate treatment plan for a recently diagnosed cancer, all of us must navigate a sea of numbers on a daily basis. As we have learned through much collective pain in the last few years, in a highly integrated economy it is exceedingly risky to accept as inevitable the innumeracy of many of our fellow citizens.
The Association of American Colleges and Universities (AAC&U) demonstrated its support for the QL movement when it announced the Essential Learning Outcomes as part of its Liberal Education and America’s Promise (LEAP) initiative. In addition to being explicitly listed among the key intellectual and practical skills students need, QL and its values are represented implicitly among other outcomes including inquiry and analysis, critical thinking, written and oral communication, and information literacy (AAC&U 2007). For while these skills obviously apply to qualitative as well as quantitative work, they exhibit themselves in distinctive ways in the quantitative context.
Sadly, the clear understanding of QL and its role in twenty-first-century lives was not initially matched by robust resources to support institutions seeking to develop curricula to meet student needs. In fact, at the end of 2001, the NCED, along with the Mathematical Association of America (MAA) and the Mathematical Sciences Education Board, hosted a forum on QL that arrived at the following stark findings: (1) Most higher education students graduated without sufficient QL training; (2) faculty in all disciplines needed professional development support to enhance QL in their courses; (3) QL was not part of assessment activity; and (4) education policy leaders were insufficiently aware of the increasing need for QL (Steen 2004, 11).
Fortunately, over the last decade, groups like AAC&U, Project Kaleidoscope (PKAL), the MAA’s Special Interest Group on QL (SIGMAA-QL), and the National Numeracy Network (NNN) have worked diligently to address these gaps. We now have free collections of materials that can support curricular development in all divisions of the academy, assessment tools that match a range of QL definitions, and a robust community to come alongside institutions taking action. What follows is a brief description of these newly developed resources.
In Our Underachieving Colleges, Derek Bok argues that “numeracy is not something mastered in a single course. The ability to apply quantitative methods to real-world problems requires a facility and an insight and intuition that can be developed only through repeated practice. This quantitative material needs to permeate the curriculum, not only in the sciences but also in the social sciences and,
in appropriate cases, in the humanities. . .” (2006, 134). This inspiring call is consistent with what we know about student learning: deep understanding follows from repeatedly encountering material in a diverse range of contexts. But in 2006, it was not clear whether the reform Bok championed was feasible. It is much easier to theorize about infusing QL throughout the curriculum than to identify those authentic curricular connections, particularly in the arts, literature, and the humanities. Happily, the QL movement has answered this call and can now point to extensive resources to support such reforms.
PKAL has published a series of “what works” documents, outlining key elements of curricular reform for QL and the related “statistical literacy” (see http://www.aacu.org/pkal/resources/ teaching/quantitative.cfm). The documents emphasize several recurring themes. While mathematics and other departments play an important role in promulgating the QL movement, true success demands a collaborative approach that involves partners in and beyond the natural sciences. The contextual nature of the QL endeavor simply cannot be learned solely in the intentionally abstract mathematics environment. Moreover, the mastery of quantitative skill by itself does not prepare students for the world they will enter. They need to be given opportunities to practice the communication and analysis of quantitative arguments.
The NNN houses multiple, free collections of example assignments and class activities to support the dissemination of this cross-cutting approach (see http://serc.carleton.edu/nnn/teaching). Each is explained in detail. In addition to providing specific learning goals, the context for use, and the teaching materials, the descriptions include “teaching notes and tips”—the electronic equivalent of the collegial advice we share with each other when passing on assignments at the water cooler. While the examples can be used “as is,” the rich background information included makes it easy to think through necessary revisions for application in different contexts.
Example activities are drawn from across the curriculum. For instance, the collection of roughly seventy quantitative writing assignments includes offerings in foreign languages, history, classics, American studies, and the fine arts, in addition to the natural and social sciences. Similarly, a collection of fifty-five assignments that use spreadsheets to teach quantitative analysis includes examples in education, library and information science, and the humanities. Whether used with students in courses or as prompts for professional development conversations, the assignment collections are designed to speed curricular development from the course to the institutional level.
About Project Kaleidoscope
Since its founding in 1989, Project Kaleidoscope (PKAL) has been a leading advocate for building and sustaining strong undergraduate programs in the fields of science, technology, engineering, and mathematics (STEM). With an extensive network of over seven thousand faculty members and administrators at over one thousand colleges, universities, and organizations.
PKAL has developed far-reaching influence in shaping undergraduate STEM learning environments that attract and retain undergraduate students. PKAL accomplishes its work by engaging campus faculty and leaders in funded projects, national and regional meetings, community-building activities, leadership development programs, and publications that are focused on advancing what works in STEM education.
In 2008, the Association of American Colleges and Universities (AAC&U) and PKAL announced a partnership to align and advance the work of both organizations in fostering meaningful twenty-first-century liberal education experiences for all undergraduate students, across all disciplines. This new partnership represents a natural progression, as nearly 75 percent of campuses with PKAL community members are also AAC&U member institutions. Together, AAC&U and PKAL apply their collective expertise in undergraduate learning, assessment, leadership, and institutional change to accelerate the pace and reach of STEM transformation.
For more information, visit www.aacu.org/pkal.
In 2004, Steen painted a bleak picture of QL assessment: “QL is largely absent from our current systems of assessment and accountability” (11). This void in the resource base was not due merely to neglect. In fact, the very nature of QL poses challenges for traditional assessment approaches. In short, the context-rich nature of QR runs contrary to typical test design, which seeks to “standardize” prompts to avoid confounding factors. As Wiggins (2003) argues, “The implications of contextualized and meaningful assessment in [QL] challenge the very conception of ‘test’ as we understand and employ that term. Test ‘items’ posed under standardized conditions are decontextualized by design. . . . [A]ssessment must be designed to cause questioning (not just ‘plug and chug’ responses to arid prompts); to teach (and not just test) which ideas and performances really matter; and to demonstrate what it means to do mathematics” (2003, 125; italics in original). Fortunately, many scholars have been busy filling this gap.
Steen and Wiggins suggest one path forward: the development of new, standardized tests that target the QL learning outcome. With support from the National Science Foundation (NSF), James Madison University has taken this approach and developed the Quantitative Reasoning Test, which targets two key dimensions of QL:
the ability to “use graphical, symbolic, and numerical methods to analyze, organize, and interpret natural phenomenon”
the ability to “discriminate between association and causation, and identify the types of evidence used to establish causation”
A twenty-five-minute multiple-choice exam, the Quantitative Reasoning Test provides results that are easy to score and nationally comparable (Sundre 2008).
To date, the test has been used at over fifty institutions with more than twenty thousand students. The results are encouraging to those of us working to develop QL among our students. In particular, James Madison students who had completed more credit hours in the “Natural World” cluster—the course grouping designed most explicitly to teach QL—exhibit greater QL facility. In other words, deliberate instruction increases student performance.
At around the same time, Carleton College’s Quantitative Inquiry, Reasoning, and Knowledge (QuIRK) initiative took a very different approach. Even as he advocated for the creation of better standardized tests, Wiggins noted that, “as in book literacy, evidence of students’ ability to play the messy game of the [QL] discipline depends on seeing whether they can handle tasks without specific cues, prompts, or simplifying scaffolds from the teacher-coach or test designer” (2003, 134). With support from the Fund for the Improvement of Postsecondary Education and the NSF, QuIRK responded to this challenge by developing a rubric to assess the use of quantitative evidence in the context of papers written by students throughout the general education curriculum and submitted to a writing portfolio. (For a detailed description of the instrument, see Grawe, Lutsky, and Tassava 2010.) One of the great benefits of the portfolio approach is the development of community on campus, as faculty readers discuss specific papers and issues. As Kezar notes, “Studies have shown that deliberation and discussion among professionals commonly lead to authentic change” (2012, 43).
QuIRK’s assessment work quickly documented that there are indeed opportunities for QL instruction across the curriculum. Among papers written for courses in the arts, literature, and the humanities, 30 percent were found to include claims for which quantitative evidence would be appropriate (Grawe 2011b). This finding confirmed that Bok’s theory of QR across the curriculum was, in fact, possible to achieve in the context of the current academy. Of equal importance, while QuIRK’s definition of QL pushed its relevance beyond the natural sciences, Carleton found results that paralleled those at James Madison: a program of intentional QL instruction—even if diffused across the curriculum—produces documentable improvements in student outcomes (Grawe 2011a).
As part of its Valid Assessment of Learning in Undergraduate Education (VALUE) project, AAC&U also pursued a portfolio approach in developing a rubric for assessing the LEAP goals, including QL. The resulting assessment tool provides language for assessing a broad range of QL dimensions: interpretation, representation, calculation, application/analysis, assumptions, and communication (see http://www.aacu.org/value/rubrics/Quantitative Literacy.cfm). While the VALUE rubrics can be applied to some student work samples without revision, the project recognizes in the rubric framing language that “the core expectations articulated in all fifteen of the VALUE rubrics can and should be translated into the language of individual campuses, disciplines, and even courses” (Rhodes 2010, 21). Boersma et al. (2011), Dingman and Madison (2011), and Pusecker et al. (2012)provide detailed examples of how this adaptation can take place in both course- and institution-level assessment.
As much as has already been completed to address the void articulated in 2004, activity continues. For example, a recently funded NSF project centered at Bowdoin College is refining a nonproprietary test incorporating short-answer and multiple-choice prompts (as a means of assessing the efficacy of both approaches) that can be taken in one hour. It is hoped this compromise between open-ended essays and multiple-choice questions will provide insight into complex, problem-solving QL skills while retaining cross-institution comparability. Although it is clear that more work must be done in coming years, multiple options for both formative and summative assessment now exist.
PKAL has argued from its inception that community is a key ingredient in fostering national reform movements. All the resources noted above will likely amount to little, unless an engaged group of teacher-scholars brings them to life through active conversation and innovation. As Kezar puts it, “Scale-up works better when individuals or groups working in local settings are connected to a network of others who are also involved in similar efforts. Through such networks, innovators can support one another and help resolve issues of implementation, motivation, and ownership. Networks can also provide the leadership needed to create and sustain change in particular settings” (2012, 41–42).
In addition to groups like AAC&U and PKAL that, through initiatives like LEAP, endorse the QL movement within the context of broader missions, several organizations have created important community groups to support the advancement of QL specifically. Mathematicians might be particularly interested in the MAA’s Special Interest Group on QL (SIGMAA-QL). Open to members of the MAA, SIGMAA-QL organizes sessions at MAA meetings and hosts an active online conversation throughout the year (see http://sigmaa.maa.org/ql/). This community is especially valuable for those interested in leading QL reform on their campuses through the mathematics curriculum.
Resting on mathematical foundations, effective QL must reach beyond the math department. The National Numeracy Network (NNN) seeks to support this broad effort by supporting members from all disciplines—from the sciences and the social sciences to the humanities and the arts. In addition to providing web access to the curricular resources summarized above, the NNN sponsors an annual meeting and maintains an ongoing online forum to discuss ways that QL can be advanced across the curriculum (see http://serc.carleton.edu/nnn/index.html). NNN leadership can offer suggestions to institutional members who seek experts to assist in QL program evaluation—whether the focus is on basic skills development, student support, QL in writing, QL assessment, or curriculum development. The organization’s open-access journal, Numeracy, offers scholars a peer-reviewed venue for disseminating and reviewing research, best practices, and perspectives in this growing field. With more than eight hundred full-text downloads per month, the journal is providing the QL movement with an intellectual home for scholarship that advances this new “discipline.”
While AAC&U, PKAL, SIGMAA-QL, and the NNN support different facets of the collective work toward a numerate society, each recognizes the value of the others. Significant overlap in leadership ensures the healthy cross-fertilization and coordination necessary to support continued advancement of this Essential Learning Outcome. With your membership and involvement in our growing, collaborative networks, together we can prepare a quantitatively literate citizenry.
AAC&U (Association of American Colleges and Universities). 2007. College Learning for the New Global Century: A Report from the National Council for Liberal Education and America’s Promise. Washington, DC: Association of American Colleges and Universities.
Boersma, S., C. Diefenderfer, S. W. Dingman, and B. L. Madison. 2011. “Quantitative Reasoning in the Contemporary Worlds, 3: Assessing Student Learning.” Numeracy 4, no. 2, http://services.bepress.com/numeracy/vol4/iss2/art8.
Dingman, S. W., and B. L. Madison. 2011. “Twenty-First-Century Quantitative Education: Beyond Content.” Peer Review 13 (3): 15–18.
Grawe, N. D. 2011a. “Beyond Math Skills: Measuring Quantitative Reasoning in Context.” In Assessing Complex General Education Student Learning Outcomes, edited by J. D. Penn, 41–52. San Francisco: Jossey-Bass.
———. 2011b. “The Potential for Teaching Quantitative Reasoning across the Curriculum: Empirical Evidence from Carleton College.” International Journal for the Scholarship of Teaching and Learning 5, no. 1, http://academics.georgiasouthern.edu/ijsotl/v5n1/articles/PDFs/_Grawe.pdf.
Grawe, N. D., N. S. Lutsky, and C. J. Tassava. 2010. “A Rubric for Assessing Quantitative Reasoning in Written Arguments.” Numeracy 3, no. 1, http://services.bepress.com/numeracy/vol3/iss1/art3.
Kezar, A. 2012. “The Path to Pedagogical Reform in the Sciences: Engaging Mutual Adaptation and Social Movement Models of Change.” Liberal Education 98 (1): 40–45.
Lutsky, N. 2008. “Arguing with Numbers: Teaching Quantitative Reasoning through Argument and Writing.” In Calculation vs. Context: Quantitative Literacy and Its Implications for Teacher Education, edited by B. L. Madison and L. A. Steen, 59–74. Washington, DC: Woodrow Wilson National Fellowship Foundation.
Orrill, R. 2001. “Mathematics, Numeracy, and Democracy.” Preface to Mathematics and Democracy: The Case for Quantitative Literacy, edited by L. A. Steen, xiii–xx. Washington, DC: The Woodrow Wilson National Fellowship Foundation.
Pusecker, K. L, M. R Torres, I. Crawford, D. Levia, D. Lehman, and G. Copi.2012. “Increasing the Validity of Outcomes Assessment.” Peer Review 13/14 (4/1): 27–30.
Rhodes, T. L., ed. 2010. Assessing Outcomes and Improving Achievement: Tips and Tools for Using Rubrics. Washington, DC: Association of American Colleges and Universities.
Steen, L. A. 2001. “The Case for Quantitative Literacy.” In Mathematics and Democracy: The Case for Quantitative Literacy, edited by L. A. Steen, 1–22. Washington, DC: Woodrow Wilson National Fellowship Foundation.
———. 2004. Achieving Quantitative Literacy: An Urgent Challenge for Higher Education. Washington, DC: Mathematical Association of America.
Sundre, D. L. 2008. The Quantitative Reasoning Test, Version 9: Test Manual. Harrisonburg, VA: Center for Assessment and Research Studies.
Wiggins, G. 2003. “‘Get Real!’: Assessing for Quantitative Literacy.” In Quantitative Literacy: Why Numeracy Matters for Schools and Colleges, edited by B. L. Madison and L. A. Steen, 121–43. Princeton, NJ: National Council on Education and the Disciplines.
Nathan D. Grawe is associate dean of the college and associate professor of economics at Carleton College.
To respond to this article, e-mail email@example.com, with the author’s name on the subject line.