Peer Review

Electronic Portfolios a Decade into the Twenty-first Century: What We Know, What We Need to Know

During the last decade, campuses designing electronic portfolios have used them both in curricular and assessment contexts. And in many campus e-portfolio projects, diverse stakeholders—faculty, staff, students, potential employers, and members of the public—have participated in the design and review of e-portfolios. Such electronic portfolios have included a range of exhibits, from multimedia artifacts and reflective commentary to artifacts-as-evidence linking to institutionally sanctioned programmatic outcomes and to more personalized self-identified outcomes. In sum, these e-portfolios have provided a new, continuing mechanism both for documenting specific practices and student accomplishments and the effects of that these activities have on learning outcomes.

What We Know

At the heart of this work in electronic portfolios is what was first a hope and then an assumption, and now a research-based claim: that creating, evidencing, connecting, and reflecting involved in electronic portfolios engage students in new and beneficial ways—especially when the portfolio provides a space for student-informed participation

The literature on e-portfolios suggests that student engagement is a critical element of portfolio development (Barrett 2000; Batson 2002; Yancey 2001). The inability to get students engaged or excited about their e-portfolios will result in a flawed implementation. From the students’ perspective the ability to personalize their e-portfolio contributes to their motivation to “work” on it throughout the year as well as their engagement in the process (Ring, Weaver, and Jones 2008).

In other words, when the e-portfolio is designed by the student as much as by the institution, implementation efforts are more likely to succeed. As important, where programs are successful in motivating students to be engaged at this level, they see higher rates on key educational metrics when comparing students creating e-portfolios with students who have not done so. Such metrics include higher rates of student engagement on a local measure of engagement (Kirkpatrick et al 2009) as well as on the nationally normed Community College Survey of Student Engagement; higher rates of course completion; and higher rates of retention (Eynon 2009). In these terms, e-portfolios work to increase student engagement.

More recent research conducted at Seton Hall University has focused on the ability of e-portfolios to foster the development of noncognitive traits as well, a topic of increasing interest in higher education. Typically such traits are defined as behaviors and attitudes, such as the ability to work with others, that correlate with success in school and employment. As reported in Inside Higher Ed, for example (Jaschik 2007). Oregon State University has implemented an admissions activity called the Insight Resume (IR)—a set of six questions that all students must answer—that address such factors as “Leadership/Group Contributions” and “Dealing with Adversity.” The IR is rated by faculty as part of the admissions process, producing a score that counts for 30 percent of the overall admission score. The benefits of the IR are many. In addition to providing a fuller picture of the student, the IR has resulted in the admission of more students of color, and as significant, these students have higher retention rates. In a similar effort, ETS will in July begin marketing a graduate admissions procedure called the Personal Professional Index, which is a similar measure of noncognitive factors keyed to success in graduate school. Seton Hall’s contribution to this line of research is to focus on the use of e-portfolios as a site for students’ recording and reflecting upon noncognitive traits, specifically five such traits, including familial support for success in college and social integration during students’ first year. The intent in this e-portfolio project, then, is to foster the development of these noncognitive factors so that students stay in school. Initial data from this project show two important outcomes: (1) that scoring guides keyed to these traits can be developed and applied to e-portfolios, and (2) that students who score well on such traits are in fact more likely to stay in school. In these terms, electronic portfolios are also working.

Four Critical Interacting Factors: E-Portfolio Models, Technologies, Programs, and Context

At this stage of their development, we are not certain about why e-portfolios produce effects like increased levels of engagement and retention, and we can’t yet account for how e-portfolio design or structure contributes to fostering learning, increasing engagement, and increasing retention. Most e-portfolio projects are still at beginning stages and we are still learning about the critical relationships that define them—particularly the relationship of any given model for e-portfolios (be it focused on learning, outcomes, or career preparation) to a given curricular program (general education, departmental, accreditation, graduate) and to a given technology. As they develop, e-portfolio models create various relationships among these three dimensions, and for those engaged in developing e-portfolios, this set of dimensions raises new questions. Unlike print portfolios, which were largely course-based and which played a limited assessment role, typically substituting for a final exam in a course, electronic portfolios tend to operate in a larger frame of reference, across courses and often across experiences: at Georgia State University, for example, they operate inside a first-year writing program; at Alverno College; they provide an evidentiary base for advising across the college years (Rickards and Guibault 2009); and at Thomas College, they operate across a major, culminate at graduation, and provide a link to employment (Edwards and Burnham 2009). In other words, unlike their print cousins, these e-portfolio models are designed to document learning not just inside a course but across courses and across experiences in college and beyond. More research is clearly needed on the role of multiple contexts for e-portfolios and their relationship to fostering intellectual development.

When initiating e-portfolio projects, campuses often begin by deciding on a specific technology to support e-portfolios. Common criteria for such technologies include cost and ease of use, but as recent research demonstrates, another criterion is equally important: the ways the technology is programmatically formative. Although e-portfolios are not themselves about technology, any technology—be it the common tool, the open source software, the homegrown system, the commercially available e-portfolio tool, or the Web 2.0 social network—is a “structured system” (Johnson 2009) and will permit or support certain kinds of activities and preclude others. Penn State University’s research on electronic portfolios provides an excellent example of how this works. The Penn State team initially hoped for a single e-portfolio “enterprise solution,” but increasingly found a disconnect between their interest in institutional program assessment and their equally important commitment to fostering student dialogue and participation. As the research team explains:

Throughout our participation in coalition research on e-Portfolios at Penn State our research question has remained focused around cocurricular learning and the role that structured systems play in facilitating student engagement in specific learning outcomes. What has challenged our research endeavors has been changing technology within which we have had to conduct this activity. For various reasons we have moved from open web space and common web publishing tools to ANGEL e-Portfolio to PebblePad and now to MovableType (Johnson 2009).

In other words, the Penn State original plan for e-portfolios consisted of finding an enterprise system solution that would support learning for all students while at the same time providing an administrative ‘back door’ through which an aggregation of rich assessment data related to learning could be harvested. Such a hypothetical system to satisfy all these needs is untenable (Johnson 2009).

Given their experience, the Penn State project has identified three complementary approaches and technologies to provide for differentiated purposes: (1) program-specific learning outcome templates for MovableType, which supports student e-portfolio activity and dialogue; (2) backtrack to e-portfolios from student resume samples, which supports internal student reflection on artifacts seen in multiple contexts (course, program, and employment) that can prompt new engagement and learning; and (3) an assessment management system, which provides faculty the opportunity to identify and tag key learning artifacts. In this more differentiated approach, the selection of technologies is more than rhetorical. Each tool is selected precisely because it supports a given purpose and audience. Moreover, as the Penn State experience illustrates, research on how different e-portfolio outcomes are supported or discouraged by specific electronic portfolio technologies is necessary to determine how these technologies can or cannot support learning more generally.

The e-Portfolio “Translation Effect”: The Role of Multiple Contexts in Fostering Learning

As suggested in the Penn State example, the role of context(s) and its impact on learning has been the focus of additional research, in some projects, a key role is played by students-as-native informants. Because students create the e-portfolios under investigation, researchers can turn to them for insight into the effects of creating an e-portfolio, including the role e-portfolios play in teacher education candidates’ understanding of assessment (e.g. at the University of Nebraska–Omaha [Topp and Goeman]), or the reasons for the connections among artifacts (e.g. by Clemson psychology students [Stephens 2009]). Put simply, students’ explanations, whether through reflective commentary or interviews, provide a window into the e-portfolio experience. Using this methodology, researchers have also inquired into the impact on learning as students move from a reflection on learning inside a single context—that of the course—to a reflection on learning in a larger context, across courses. For example, Clemson engineering student Josh Reynolds shared such a perspective.

Josh had created several portfolios, working in software like Mozilla and Dreamweaver, and two of his e-portfolios had been awarded prizes by Clemson’s Pearce Center for Professional Communication. Because the university was interested in how different software might affect the creation of excellent e-portfolios like the ones Josh had composed, Clemson went straight to the source, to Josh. They asked Josh first, to:

recreate his award-winning e-Portfolio within Blackboard and then create a gen ed e-portfolio that makes use of some of his original e-Portfolio, [t]he goal . . . to see how a student might best demonstrate general education competencies without letting [a] structured . . . portfolio template or tool interfere with his creativity and learning (Weaver 2005).

As a consequence of this retrofitting, Josh reported that “the ease of the . . . portfolio template,” for him, made the portfolio-creation process feel more list-like and that it removed a good deal of the freedom he associated with creativity. He also reported a loss of multiple contexts in the templated approach. Without the multiple contexts, Josh claimed to have learned less.

“The whole point of the portfolio that I made [in the nontemplated model] was to help me realize the connections that I made across the curriculum, and to make these connections obvious to the people who view my portfolio, to show that I indeed did learn something, and not just how to regurgitate the assignments of the past semester” (Weaver 2005).

Josh noted, as well, that for him the value of e-portfolios was creating connections from classes to larger contexts. Students, he notes,

“have already proven that they have the knowledge to answer specific questions by passing their classes, but it is just as important for them to demonstrate that they can make connections among those things they have learned. This is where I believe the value of the e-portfolio lies” (Weaver 2005)

In making such observations, Josh is not alone. Students at Florida State University (FSU) working in a very different model—a career portfolio—speak similarly of the shift from discrete courses to a larger frame of reference. The FSU career e-portfolio, like several e-portfolio models, isn’t limited to academic courses. Rather, it provides space for learning to occur in three areas: (1) curricular situations, which are largely course-based; (2) cocurricular situations, which are often linked to the curriculum (e.g., service learning opportunities, internships, peer tutoring, and leadership experiences); and (3) extracurricular situations (e.g., jobs, sports activities, etc.). The matrix structure FSU uses to foster this multicontextual thinking—what FSU calls a Skills Matrix—resembles the general education matrix created at Indiana University–Purdue University Indianapolis (IUPUI). Researchers at IUPUI suggest that this model promotes “matrix thinking”(Hamilton and Kahn). The logic of matrix thinking is that a matrix prompts students to place any single artifact in multiple cells, in the process seeing the same artifact in multiple ways and thus drawing different conclusions about it and yet also synthesizing its value. At FSU, the Skills Matrix is comprehensive, as documented in this report:

In the Skills Matrix, students document the experiences in which they have developed various transferable skills. The skills that are integrated in the portfolio are: communication, creativity, critical thinking, leadership, life management, research/project development, social responsibility, teamwork, and technical/scientific. Students have the ability to add their own skills, which may include skills that are more directly related to their academic major or career goals. The experiences students can use to reflect on the skills they have gained include: jobs/internships, courses, service/volunteer work, memberships/activities, and interests/life experiences.

In using this matrix to organize work from multiple domains, students “translate” their experience from one context into a larger context. As Barbara Cambridge explains, “Movement between [contexts] is the site of invention,” a point made by one FSU student in articulating her experience:

“The portfolio has been so useful in helping me realize what skills I’ve learned through the experiences I’ve had and classes I’ve taken. Having my classes and jobs organized according to the skills I’ve gained from them allows me to see what I’ve actually accomplished through my education…The portfolio really has proven to be a powerful tool that forced me for the first time to consider what I’ve done with my college career. It brings a whole new way of thinking about classes; instead of just evaluating success through test scores and completed requirements I’m seeing what valuable skills I’ve gained that will help me in the future” (Cambridge 2009).

This translation effect, occurring as a function of students seeing their work in multiple contexts, is occurring in other e-portfolio programs as well. In the Virginia Tech English Education model, for instance, students locate evidence drawn from courses, which provide one context, and internship experiences, which provides a second context. They also relate their work to the Interstate New Teacher Assessment and Support Consortium (INTASC) standards, which provides a third context. In addition, some students create their own set of outcomes, which provides a fourth context. These contexts are layered into the e-portfolio, and the movement between layers provides new opportunities to learn and to invent. Likewise, research on e-portfolios in the accounting major at the University of Waterloo shows that of all sites of learning, the most productive for multicontextual invention happens as students move to a co-op experience where, they apply and modify what they have learned in classroom settings and use multiple contexts in dialogue to frame their learning.

Evidence and Reflection

At the heart of e-portfolio practice research is a claim about the significance of evidence-based learning. Whether outcomes are programmatically identified or student-designed, the process of connecting artifacts to outcomes rests on the assumption that the selection of, and reflection on, a body of evidence offers another opportunity to learn and a valid means of assessment. At the same time, research has only recently focused on the process of selection and on what counts as evidence. As the e-portfolio research team at George Mason University observes:

Despite the central role of evidence in e-portfolio practice, the dynamics of its use by portfolio authors is under-examined. The role of evidence is often assumed [to be] uniform: artifacts produced by the author (or assertions about them) are connected to a competency the author claims they possess, and the evidence is either sufficient or insufficient. In fact, our research suggests that the actual use of evidence in e-portfolios is much more complex (Blank-Godlove et al 2009).

During the last three years, as part of their participation in the Inter/National Coalition for Electronic Portfolio Research, the George Mason team has developed what they call an emergent typology on use of evidence in portfolios. Based on content analysis of multiple portfolios, the researchers hypothesize that students’ use of evidence “varies along three dimensions”: “(1) the characteristics of the item used as evidence, (2) the explicit or deduced purpose of the portfolio creator in incorporating the selected evidence, and (3) the characteristics of the learning activity reflected in the use of evidence.” Excellent e-portfolios “align” evidence with context and with audience, and “there is a match between the content of the evidence and the way it is framed in the reflective narrative of the e-portfolio.”

The role that reflection plays in student learning and how it can be supported is also focusing research efforts. One set of research questions has to do with the contribution that reflection makes to learning and assessment: is reflection a contextualizing device, evidence itself, or both? This question is, again, one of interest to higher education more generally. For example, Bob Gonyea , who works on the National Student Engagement Survey project, has suggested that reflection, which we once thought of as a proxy for learning, may itself be evidence. E-portfolio reviewers’ observations make the same point. When Florida State University asked prospective employers, for example, “whether it was the experience itself or the way in which students described their experiences [that was most influential], three of the four employers believed that a student’s ability to effectively describe his or her experience outweighs the experience itself.” Many colleges and universities—including Sheffield Hallam University, the University of Waterloo, and Alverno College—have also found that helping students develop a “capacity to reflect” is a critical educational outcome, in and of itself. The researchers at the University of Waterloo summarize the issue this way:

Reflection is a learned skill. Students do not necessarily “know” how to reflect effectively on their learning and use those reflections to make connections between the learning that occurs in different contexts (academic, workplace, community). Indeed, we found that the majority of students in our study groups did not begin to make connections, despite being encouraged to do so, until they moved from one context (academic) to another (workplace). This indicates that we need to carefully scaffold opportunities for reflection into academic programs for students so that they have time to develop this ability. Providing feedback upon which they can act, and providing it in a timely manner is critical to the development of the capacity to reflect. With the expectation that learning is a life-long endeavor, students must become more aware of how they learn if they are to continue their personal growth and development after they graduate. Developing the capacity to reflect is key to this outcome (Penny-Light et al. 2009).

Given that many e-portfolio practitioners and researchers understand reflection as the connective tissue for the intellectual work and exhibits we see in electronic portfolios, the next generation of electronic portfolio research is likely to focus on questions around reflection.

Conclusion

Research on electronic portfolios has developed in a unique way, involving faculty, of course, but also students, staff, potential employers, and many others in the process. Because electronic portfolios are worldwide, our knowledge base is both wide and culturally complex. Because many e-portfolio practitioners want to know if e-portfolios “work”—if they make a difference in students’ lives and if they can contribute to student success of many kinds—research from the beginning has linked individual efforts to larger bodies of research. And because e-portfolios link curriculum and assessment in ways that acknowledge and build on students’ experiences, they provide new sites for learning about how we assess, about how we teach, and perhaps most importantly, about how we all learn.

References

Blank-Godlove, J., D. Cambridge, K. Danner, K. Eby, H. Hare, J. Owen, and L. Smith. 2009. George Mason final report for the Inter/National Coalition for Electronic Portfolio Research. Feb. 28.

Cambridge, B. 2009. Stimulating and tracking learning: The benefits of electronic portfolios. University of Maryland Writing Project.

Cambridge, D., B. Cambridge, and K. B.Yancey, eds. 2009. Electronic portfolios 2.0: Emergent findings about implementation and impact. San Francisco: Stylus Publishing.

Edwards, T., and C. Burnham. 2009. The promise of e-portfolios for institutional assessment. In Electronic portfolios 2.0: Emergent findings about implementation and impact, eds. D. Cambridge, B. Cambridge, and K. B. Yancey, 87–91. Sterling, VA: Stylus Publishing.

Eynon, B. 2009. Making connections: The LaGuardia ePortfolio. In Electronic portfolios 2.0: Emergent findings about implementation and impact, eds. D. Cambridge, B. Cambridge, and K. B. Yancey, 59–69. Sterling, VA: Stylus Publishing.

Hamilton, S., and S. Kahn,. 2009. Demonstrating intellectual growth and development: The IUPUI ePort. In Electronic portfolios 2.0: Emergent findings about implementation and impact, eds. D. Cambridge, B. Cambridge, and K. B. Yancey, 91–97. Sterling, VA: Stylus Publishing.

Jaschik, S. 2007. Making holistic admissions work. Inside Higher Ed. March 2. www.insidehighered.com/news/2007/03/02/holistic

Johnson. G. 2009. Penn State final report for the International Coalition for Electronic Portfolio Research. Feb. 28.

Kirkpatrick, J., T. Renner, L. Kanae, and K. Goya. 2009. A values-driven ePortfolio journey: Nā Wa’a. In Electronic portfolios 2.0: Emergent findings about implementation and impact, eds. D. Cambridge, B. Cambridge, and K. B. Yancey, 97–103. Sterling, VA: Stylus Publishing

Penny-Light, T., B. Sproule, K. Lithgow, and P. Charbonnneau. 2009. University of Waterloo final report for the I/NCEPR Cohort III. Feb. 28.

Rickards, W. and L. Guibault. 2009. Studying student reflection in an electronic portfolio environment: An inquiry in the context of practice. In Electronic portfolios 2.0: Emergent findings about implementation and impact, eds. D. Cambridge, B. Cambridge, and K. B. Yancey, 17-29. Sterling, VA: Stylus Publishing.

Ring, G., Weaver, B., and Jones, J. 2008. Electronic portfolios: Engaged students create multimedia-rich artifacts. Journal of the Research Centre for Educational Technology 4 (2).

Stephens, B. 2009. E-portfolios in an undergraduate research experiences program. In Electronic portfolios 2.0: Emergent findings about implementation and impact, eds. D. Cambridge, B. Cambridge, and K. B. Yancey, 103-09. Sterling, VA: Stylus Publishing.

Topp, N. and B. Goeman. 2009. Perceptions of teacher candidates on ePortfolio use. In Electronic portfolios 2.0: Emergent findings about implementation and impact, eds. D. Cambridge, B. Cambridge, and K. B. Yancey, 109-19. Sterling, VA: Stylus Publishing.

Weaver, B. 2005. General education portfolio required Faculty directions. www.clemson.edu/ccit/about/publications/facultydirections/
archives/fall_2005/eports_in_gen_ed.html

Previous Issues