AAC&U News | March 2005

IUPUI Uses Electronic Portfolios to Assess Diversity Learning Gains

Indiana University-Purdue University Indianapolis (IUPUI) has faced unique challenges in its efforts to build a coherent educational plan for all of its students. An urban, public university formed in 1969 through a confederation of professional schools, IUPUI lacked at its founding the kind of shared curriculum that would typically serve as a basis for general education.

To address the problems posed by its decentralized structure, in 1998 IUPUI formally adopted the Principles of Undergraduate Learning (PULs)--a set of six higher-order abilities that the university expects all students to develop. More recently, faculty and administrators have turned to the equally challenging question of how to ensure student mastery of the PULs: how can learning in these areas be effectively assessed across the institution, in the many schools and majors? IUPUI, like a growing number of colleges across the country, is now experimenting with electronic portfolios as a possible component of its assessment strategy.

One PUL that has received particular attention in IUPUI's recent assessment efforts is "Understanding Society and Culture." Defined as "the ability of students to recognize their own cultural traditions and to understand and appreciate the diversity of human experience," this PUL focuses on global diversity as well as diversity within the United States. By defining a set of expectations for this principle at every stage of undergraduate education and using electronic portfolios to assess the achievement of these expectations, IUPUI hopes not just to evaluate what students are learning about diversity, but also to gather information that will help the institution improve how it teaches about diversity.

Diversity as a Principle of Undergraduate Learning

IUPUI's Principles of Undergraduate Learning are infused throughout the majors and professional programs rather than being confined to a core curriculum. Developed with the input of hundreds of faculty members, students, and staff, these principles currently include core communication and quantitative skills; critical thinking; integration and application of knowledge; intellectual depth, breadth, and adaptiveness; understanding society and culture; and values and ethics.

Interdisciplinary communities of practice support campus work on each of the principles. These communities establish expectations for the PULs and provide rubrics to guide their integration into academic departments, syllabi, and course assignments. Associate Dean Sharon Hamilton, who has been involved in the development of the PULs as well as the electronic portfolios, sees the continuing growth of the communities as a key to the success of IUPUI's educational plan. The goal, she says, is "to have as many faculty as possible view their role in the university not just as a member of a department, but also as a member of one of these communities."

The community of practice for "Understanding Society and Culture," which is headed by Construction Technology Professor Daphene Cyr Koch, has been active over the past year in defining expectations for student learning. Diversity has long been recognized as a core element of "Understanding Society and Culture," Koch says, and "as the world is growing smaller," it is becoming increasingly apparent how the diversity of the U.S. "mirrors the world." A proposed revision of the PUL's name to "Understanding the Diversity of Society and Culture," up for consideration by IUPUI's faculty council later this year, is expected to formalize this principle's focus on exploring different cultures and understanding the implications of diversity within cultures--both in the U.S. and around the world.

The ePort Program

IUPUI's electronic portfolio--or "ePort"--program is designed to assess student competency in "Understanding Society and Culture" and the other PULs. First tested in the classroom a year and a half ago, the ePort program is now in an advanced pilot phase, with about 150 first-year students using the portfolios. This fall, program administrators anticipate that nearly 1,500 entering students will be using portfolios, and in the fall of 2006, they plan a full launch of the program for all 4,000 first-year students.

The core assessment component of the electronic portfolios is the "Learning Matrix," conceptualized as a table with the six PULs on one side and, on the other, four levels of achievement: introductory (expectations of first-year students), intermediate (second-year expectations), advanced (junior/senior-level expectations within the major), and experiential (cocurricular expectations). As students progress through their studies, they upload artifacts--papers, projects, videos, and any other relevant evidence--to document their progress and mastery of each PUL at each level. To complete any cell in the matrix, a student must also write a short reflective essay that explains how the evidence provided satisfies the expectations for the given level of the PUL. The evidence in each cell is then sent to an external reviewer, who assigns students scores of 1, 2, or 3 depending on whether they have exceeded, met, or failed to meet the expectations.

IUPUI's communities of practice specify the expectations and provide examples of assignments that might promote learning in the different PULs. In the "Understanding Society and Culture" community, faculty have recently defined expectations for the introductory level and tentative expectations for the intermediate level. Students at the introductory level are expected, for example, to

have identified and explored some aspects of the range of diversity and universality in human history, society, and culture;
have recognized some aspects of interconnectedness of local and global concerns;
have experienced working toward a shared goal with mutual respect within a safe environment at the university;
have interacted, in person, in literature or film, or through academic reading, with people and ideas in a different culture.

These expectations are to be demonstrated in the context of the major. An anthropology major, the community of practice suggests, could thus demonstrate interaction with a different culture through an ethnographic interview essay, while a construction technology major might use role playing to demonstrate working toward a shared goal with mutual respect. At the intermediate level, more sophisticated work is required--for instance, students in a health care course might study the health care needs and health beliefs of the Hispanic community and then participate in service-learning at a clinic that serves the Hispanic population.

Moving Forward

As IUPUI moves toward full implementation of the ePort program, much work remains to be done. Expectations at the intermediate and advanced levels still need to be finalized, and staff training needs to be expanded to meet new demand. Reviewer workload and staffing pose perhaps the most significant challenge: in the pilot phases, the portfolios have been reviewed by volunteer retired faculty--the "Senior Academy"--but as the number of students using the portfolios increases, many more reviewers will be needed. IUPUI hopes that, by offering reimbursement and course-load adjustment as incentives, enough faculty and community of practice members can be drawn into the reviewing process. Other possibilities under consideration include using some senior students to review first-year students' work and training graduate students in relevant programs to serve as reviewers.

Despite the costs of fully implementing ePort, administrators are optimistic about the benefits of electronic portfolios. On the one hand, says Sharon Hamilton, "portfolios provide authentic evidence of learning in ways that most other assessments can only approximate." At the same time, Hamilton expects that the portfolios themselves will enrich learning: the structure of the learning matrix, with its emphasis on documenting progressive expectations and reflecting on learning, will allow students to "see" their growth over time.

Hamilton also anticipates benefits on the institutional level. Portfolios enable institutions "to take a second look at student learning, one that goes beyond individual faculty assessing the learning of individual students in their classes," she says. Moreover, electronic portfolios allow faculty and administrators to examine connections between student performance, demographic variables, and individual courses, departments, or schools. Together, these data promise to help professors, department chairs, and administrators make informed decisions and work more effectively toward their ultimate educational goals--to ensure achievement and improve learning across the institution.

To learn more about the Principles of Undergraduate Learning and the ePort program, visit IUPUI's Web site.

Sharon Hamilton, who has played a key role in developing the ePort program, is a participant in "Greater Expectations for 21st-century Learning: Implications of Technology for General Education," a series of Webcasts presented by the Teaching, Learning, and Technology Group and AAC&U. She will speak in the final Webcast in the series, "Integrative Learning and Electronic Portfolios," which will take place at 1 pm EST on May 12.

AAC&U also has many assessment resources available online. The Association's recently released board statement, Our Students' Best Work: A Framework for Accountability Worthy of Our Mission, offers a blueprint for assessment efforts at American colleges and universities.