Portfolios Transform Writing
Assessment at Carleton College
Editor's Note: In September
2004, AAC&U's board of directors released Our
Students' Best Work: A Framework for Accountability
Worthy of Our Mission, a statement that urges all colleges
and universities to focus their educational planning and assessment
efforts on a set of outcomes essential for success in today's
world. The statement noted that there is a strong consensus
emerging among educators as well as business, policy, and
civic leaders that higher education must "embrace a
small number of highly valued and widely affirmed educational
goals, establish high standards for each, and assess their
achievement across the curriculum." Many AAC&U member
institutions have already made progress in organizing their
curriculum and assessment programs around these goals. These
institutions' innovations provide valuable models for
how to effectively advance and comprehensively assess the
outcomes that form the core of a contemporary liberal education.
Over the next several months, AAC&U News will
highlight several of these models, focusing on the key learning
outcomes described in Our Students' Best Work.
|| Carleton College is a private, liberal
arts institution located in Northfield, Minnesota.
A recent study by the National Commission
on Writing has shed light on a disturbing trend: at a time
when writing increasingly is seen as a fundamental skill for
professionals, many employers are reporting that college graduates
have difficulty communicating in clear, grammatical prose.
This finding, drawn from a national survey of business leaders,
has economic as well as educational implications. According
to the study, businesses in the United States may spend as
much as $3.1 billion on remedial writing instruction each
year, with many additional dollars lost as a result of inefficiencies
related to poor writing. To prepare graduates for the contemporary
workplace, the commission concludes, educators need to develop
more comprehensive and sustained ways of teaching and assessing
Carleton College, a private,
liberal arts institution in Northfield, Minnesota, is one
of a growing number of schools developing new strategies to
improve and assess students' writing skills. Carleton
has long emphasized "writing across the curriculum"—distributing
writing instruction in many classes and throughout the disciplines—and
in 2001, the college introduced sophomore writing portfolios
to broaden its assessment of student writing. Required midway
through every student's college career, the writing
portfolios enable the college to ensure that undergraduates
can write competently in a range of styles and contexts. At
the same time, by encouraging students to reflect on—and
revise—their writing, the portfolios themselves constitute
an important educational experience.
A New Writing Requirement
Carol Rutz, the director of the
writing program at Carleton College, has overseen the school's
transition from its old "single course" model
of writing assessment to the use of writing portfolios. By
the time Carleton decided to abandon the old model, she says,
there was a broad consensus among students and faculty that
assessments based on writing from a single class were inadequate.
But when the school decided on portfolios as a new assessment
vehicle, it faced a different set of challenges. Portfolio
assessment requires large amounts of time and money and significant
administrative oversight. "The wealth of data comes
at a price," Rutz says, "but we are finding—even
after a very short period of implementation—that gains
made in faculty development and student self-awareness are
well worth the investment."
This is the
first article in a special series on advancing and assessing
key liberal education outcomes.
To meet the school's current
portfolio requirement, students at the end of their sophomore
year must submit three to five papers demonstrating their
ability to write effectively in different rhetorical and disciplinary
contexts. The papers must be "authenticated" by instructors,
who certify that the papers were written for their classes
and indicate if they have since been revised. Finally, students
write reflective essays about their writing to introduce the
Requiring such a breadth of written
work—as opposed to the narrower sample of student writing
produced in single course writing assessments—tends
to foster healthier attitudes toward writing. Clara Shaw Hardy,
a professor in the Department of Classical Languages who is
active in the portfolio program, notes that the breadth requirement
"eliminates the perception that writing is something
to be checked off after one course rather than a skill that
underlies most of an undergraduate's education." Moreover,
she says, "it honors the fact that different disciplines
value different kinds of writing skills."
Professor Hardy stresses that her
own approach to teaching—like that of many of her colleagues—has
also been "radically affected" by participation
in the writing program. "I now regularly require drafts,
use peer feedback, and incorporate a wide range of formal
and informal writing assignments in my courses," she
says. Faculty development programs, including grants for course
development, workshops, and speaker series, contribute greatly
to such successes. Indeed, in the 2003-04 academic year, 41.5
percent of tenured and tenure-track faculty participated in
grant-funded activities related to the writing program.
Evaluating the portfolios,
a task Carleton assigns to a volunteer group of "faculty
readers," provides another opportunity for faculty development.
Associate Dean Elizabeth Ciner, who has been involved in the
writing program at Carleton for almost three decades, points
out that faculty readers gain a broad perspective on the state
of student writing on campus. And these faculty members inevitably
learn from what they find: "To see what others are doing
in their writing assignments, to see how well students can
write, to see what gives students general difficulty . . .
all of this is possible reading portfolios."
|Carleton's writing program encourages
students to become more self-conscious about their writing.
Assessing the Portfolios
During assessment, faculty
readers assign one of three scores to every writing portfolio:
"pass," "exemplary," or "needs work." These scores are determined
according to locally developed standards, says Jacqulyn Lauer-Glebov,
Carleton's coordinator of educational assessment. Such locally
developed assessments have several advantages over national
standards-based assessments. Unlike high-stakes, "snapshot"
assessments such as the new SAT writing test, Lauer-Glebov
says, portfolios offer a flexible assessment model that can
document a range of ability.
Carleton's evaluation criteria
emphasize the demonstration of this range. Each portfolio
must represent at least two of the college's four curricular
divisions (Arts and Literature, Humanities, Social Sciences,
and Mathematics/Natural Sciences) and must include at least
one paper from the student's "Writing Requirement"
course. Together, the papers must also demonstrate their author's
mastery of each of several key writing skills—the ability
to report on observation, to analyze complex information,
to provide interpretation, to use and document sources, and
to articulate and support a thesis-driven argument.
In addition to assessing the student's
success in these areas, faculty readers also provide feedback
on the quality of writing. Each portfolio is rated on whether
it "rarely," "usually," or "consistently"
demonstrates attention to audience and purpose, clarity of
prose, clear organization, effective use of evidence, distinctive
voice, appropriate diction, and control of error.
At the end of the assessment process,
about 8 percent of portfolios typically are rated "needs
work" (an average of 78 percent receive a passing grade,
with 14 percent earning "exemplary"). Students
who do not pass the writing assessment resubmit their portfolios
after receiving written suggestions and, if needed, personalized
As more information becomes available about the strengths
and weaknesses of portfolio-based writing assessments, Carleton's
writing program will continue to evolve. Some aspects of the
program, such as the treatment of failing portfolios, have
already come under scrutiny. As Carol Rutz points out, assessing
student writing midway through college affords Carleton "a
chance to intervene with students who need help," but
students whose portfolios do fail still receive help only
on a case-by-case basis. Adopting a more formalized approach
to working with these students could strengthen the program.
On the other hand, early data already
is indicating some positive trends. For example, an analysis
of portfolio contents suggests that Carleton's faculty development
efforts have paid off: a statistically significant correlation
has been found between faculty participation in development
activities and the presence of papers in portfolios, with
students more likely to include papers from courses taught
by faculty who have been active in the writing program. This
finding, Jacqulyn Lauer-Glabov explains, suggests that "as
faculty become more conscious of the writing they ask of their
students, students respond by selecting that work to represent
their writing mastery."
Ciner hopes that the effects
of such successes will extend beyond student writing. The
writing program itself, she says, is "becoming a model
for all sorts of curricular initiatives—and our experience
in portfolios is guiding us as we consider other aspects of
the curriculum." These initiatives—which target
other key learning outcomes such as information literacy,
interdisciplinary learning, quantitative reasoning, and critical
thinking—promise to interact with the writing program
in fruitful ways and continue the transformation of undergraduate
learning at Carleton.
For further information about Carleton
portfolios, visit the writing program's Web pages.
The results of the survey
from the National Commission on Writing are also online.
AAC&U's recent accountability statement, Our
Students Best Work, offers a blueprint for assessment
efforts at American colleges and universities. For more information
about innovative writing programs in colleges and universities
today, see the special
issue of Peer Review on that topic.