Peer Review

Eportfolios, Assessment, and General Education Transformation

Salt Lake Community College (SLCC) is an urban, multi-campus institution that has been on a twelve-year journey to improve its general education program and assess essential learning outcomes. Over the course of our journey, we have learned much about how students are actually experiencing general education, and about the strengths and limitations of using eportfolios to assess student learning.

A 2004 visit from our regional accrediting body made us acutely aware that our general education program lacked learning outcomes and was only being assessed indirectly using surveys that asked graduating students whether they were satisfied with their experience. Clearly, we had much to do. We gathered a committee of faculty, academic administrators, and student affairs staff that worked diligently in summer 2005 to write a set of joint institutional and general education learning outcomes that passed the faculty senate in the fall of that year. Our curriculum committees then worked for two years to ensure that those high-level learning outcomes were translated in increasing detail into program- and course-level learning outcomes, because we wanted to be able to collect data only once—in courses, using actual student work—and use that data to assess academic programs and general education.

Eportfolio Implementation

At the same time that we embedded learning outcomes at all levels of our institution, a handful of faculty in a variety of disciplines used a Utah Higher Education Technology Initiative grant to begin piloting eportfolios in their courses. They experimented with different eportfolio platforms, talked with each other about their relative merits, and played around with how best to use eportfolios in the classroom. After four years of valuable experience piloting eportfolios, learning from other institutions such as La Guardia Community College, and reading the small but growing literature on the potential for eportfolios, we decided in 2009 to put forward a proposal to implement eportfolios as a requirement in all general education courses. After considerable discussion in curriculum and faculty governance bodies, the proposal passed and was implemented in summer 2010.

The basic principles of our eportfolio implementation are simple but have had a powerful impact on the institution and on student learning. Electronic portfolios are not a graduation requirement, but they are a required component of every general education course. Students maintain one integrative eportfolio, and faculty in each general education course ask students to archive at least one signature assignment and reflection from the course. A signature assignment is a realistic application of knowledge—such as a paper, presentation, or project—that requires students to demonstrate work relevant to two or more of the general education learning outcomes. We encourage but do not require faculty to identify more than one signature assignment in the course so students can make curatorial decisions about the work they want to showcase. Faculty design their own reflection prompts, but on our faculty support site (www.facultyeportfolioresource.weebly.com), we provide them generic ideas such as the following:

  • ƒ Make connections between this general education course and others you have taken.
  • ƒ Make connections between your academic work and your life or the broader world.
  • ƒ Explain how your performance on this signature assignment demonstrates your progress toward general education learning outcomes.
  • ƒ Reflect on your thought processes or actions as you faced the challenges of completing the assignment itself.

Learning outcomes are further reinforced on the “goals and outcomes” page of a student’s portfolio. In addition to listing their personal and professional goals on that page, students also list SLCC’s general education learning outcomes. We encourage students to link from each outcome to evidence in the portfolio indicating that they are attaining it.

Portfolio Culture Developing Slowly but Surely

Our eportfolio requirement in general education has been in place for six years now. We have discovered that it takes a long time to develop an institutional eportfolio culture and what Penny Light, Chen, and Ittelson (2012) call “folio thinking” at such a large institution, but we are making progress. We find that student understanding of and appreciation for eportfolio is highly dependent on how individual faculty treat it in their courses. If a critical mass of a student’s instructors incorporate eportfolio effectively in their courses—instead of making it an add-on to an otherwise unchanged course—students begin to understand that eportfolios are a means of documenting and reflecting upon their learning.

In a spring 2015 faculty survey that had a surprisingly good 17 percent response rate for an emailed instrument, 62 percent of the respondents thought the eportfolio requirement had a “very” or “mostly” positive impact on the general education program, and 61 percent said that it had a “very” or “mostly” positive impact on students’ ability to demonstrate their progress toward general education learning outcomes. In both cases, only 8 percent of the respondents said that it had a “very” or “mostly” negative impact. Similarly, 56 percent of the respondents thought that eportfolio had a “very” or “mostly” positive impact on students developing a sense of ownership of their education, with only 11 percent saying that it had not.

We have evidence that eportfolios are helping students better understand and intentionally work toward general education learning outcomes. In spring 2010—the semester before eportfolio implementation—I added a new question about learning outcomes to a student survey conducted in random on-campus (not online) general education courses. At that time, 27 percent of the students said that they had been introduced to SLCC’s general education learning outcomes in the course in which they took the survey. In spring 2015—five years after implementation of the eportfolio requirement—I repeated the question in another random sample of general education courses. Now 62 percent of the students said that they had been introduced to the general education learning outcomes in the course.

Our faculty-led general education committee plays a large role in the development of portfolio culture. Each general education course comes up for review every five years. We revamped the review process in several important ways. Each course is assigned two mentors from the general education committee who work with the faculty bringing the course for review. They provide important pre-review advice and feedback that greatly increase the chances that the committee will approve the course. Once the course reaches the committee, members scrutinize its signature assignments, eportfolio integration, and reflective practices. Our intent is for the general education course review process to be a form of collaborative faculty learning, because faculty on the committee get to see a variety of positive ways of integrating eportfolio into courses.

Assessing General Education with Eportfolios

We have come a long way on general education assessment. After our ten-year accreditation visit in 2014, SLCC received a commendation from its regional accreditor for the way we use eportfolios to assess general education learning outcomes. Every May our institutional research office pulls a random sample of 100 students who have just graduated with an Associate’s degree and who have taken all of their general education courses from SLCC. Our eportfolio coordinator organizes teams of faculty who apply modified VALUE and homegrown rubrics to those students’ eportfolios.

Our assessment teams look for data that answer two kinds of questions, the first of which might seem a bit rudimentary but is nonetheless critical: Do our graduates get enough experience in doing the kinds of assignments and reflection that would give them a reasonable chance of attaining general education learning outcomes? We have found that eportfolio assessment often helps us answer this question.

Basic information literacy is one of our general education learning outcomes where eportfolios have given us great insight. Do our students get enough experience conducting research using outside-of-class materials and credible sources, and properly citing those sources in their work? Data from our most recent assessment report illustrate a pattern we’ve seen every time we conduct this kind of assessment. About half of our students had considerable evidence (four or more signature assignments) of conducting outside-of-class research, while only 42 percent had considerable evidence of using credible outside sources, and only a third had considerable evidence that they can adequately cite their sources. This data inform our efforts to better target time and resources toward improving information literacy instruction.

Our assessment teams also use components of the VALUE rubrics to determine how well our Associate’s degree graduates are doing as they move from sophomore to junior status. In particular, they apply pieces of the VALUE rubrics dealing with written communication, quantitative literacy, and critical thinking. For example, we are interested in knowing whether students are using genre conventions and effectively developing content in their writing, so one assessment team used those elements of the VALUE rubric for written communication to assess student writing in three courses: their second-semester writing composition course, the course they use to satisfy their Humanities distribution requirement, and the course they use to satisfy their American Institutions core requirement. We modified the VALUE rubrics to accommodate multiple assignments and produce an average score for each student’s body of written work.

Tables 1 and 2 illustrate the kinds of data we are getting when we apply VALUE rubrics for written communication to student work in their general education eportfolios. It is very useful indeed to be able to get this high-level look at student writing, because it allows other academic programs to benchmark their more detailed assessments of student writing. It also allows department chairs and our Faculty Development Center to use the data to structure professional development opportunities for full-time and adjunct faculty. Finally, we are somewhat emboldened to know that when the “no evidence” student portfolios are set aside, our data looks quite similar to the Multi-State Collaborative pilot-year data for community colleges.

Table 1. Percentage of Students Whose Mean Scores for Effectively using Genre Conventions Fell into the VALUE Rubric Ranges
(Overall mean, excluding the “no evidence” students = 2.37)

0
No Evidence
1 2 3 4
15% 21% 35% 20% 9%
Student had no writing assignments in this portfolio from ENGL 2010, Al, and HU courses.
Attempts to use a consistent system for basic organization and presentation for a specific writing task.
Follows expectations appropriate to a specific writing task for basic organization, content, and presentation.
Demonstrates consistent use of important conventions particular to a specific writing task, including organization, content, presentation, and stylistic choices.
Demonstrates detailed attention to and successful execution of conventions particular to a specific writing task, including organization, content, presentation, formatting, and stylistic choices.

 

Table 2. Percentage of Students Whose Mean Scores for Content Development Fell into the VALUE Rubric Ranges
(Overall mean, excluding the “no evidence” students = 2.45)

0
No Evidence
1 2 3 4
15% 16% 36% 24% 9%
Student had no writing assignments in this portfolio from ENGL 2010, Al, and HU courses.
Uses appropriate and relevant content to develop simple ideas in some parts of the work.
Uses appropriate and relevant content to develop and explore ideas through most of the work.
Uses appropriate, relevant, and compelling content to explore ideas within the context of the discipline and shape the whole work.
Uses appropriate, relevant, and compelling content to illustrate mastery of the subject, convey the writer’s understanding, and shape the whole work.

The Challenges and Rewards of Eportfolio Assessment of General Education

Using eportfolios to assess general education at a large community college is not without its difficulties, two of which I would like to highlight here. A portfolio contains a large volume of assignments and reflections, and the reviewers usually do not know the details of the assignments or the reflection prompts the faculty gave to the students. Necessarily then, assessment of something as large and diverse as a general education program has to be cast at a certain level of generality, but not so general that it does not provide useful data. Thankfully we have found that many components of the VALUE rubrics are well suited to eportfolio assessment of general education, and we have concentrated on using them effectively and then closing the loop on the assessment.

The other difficulty we’ve encountered, albeit one that has lessened in the past six years, has to do with faculty and student compliance with the eportfolio requirement. We will feel more comfortable with the validity of our assessment data when eportfolio culture and folio thinking become universal at SLCC and the percentage of portfolios with “no evidence” in key areas drops closer to zero. We have made tremendous progress in this regard—primarily through professional development and the course review process of the general education committee—but know that we still have work to do.

As useful as our annual assessment reports have been, and as gratifying as the commendation from our accreditor has been, even more important to us is the way that eportfolios have opened up general education at SLCC. We have a larger shared sense of responsibility for general education, and we know so much more about how what Yancey (1998) calls the “delivered curriculum” is actually experienced by students. It is one thing to know about the inputs—the courses that students take—and quite another to be able to see the outputs of general education in the form of signature assignments and student reflections. Our eportfolio requirement has given us a great tool with which to knit together a menu-driven general education into a more integrative experience for students, and it has provided students with a roadmap that helps them understand and more intentionally achieve essential learning outcomes. 

 

References

Penny Light, Tracy, Helen Chen, and John Ittelson. 2012. Documenting Learning with ePortfolios: A Guide for College Instructors. San Francisco: Jossey-Bass.

Yancey, Kathleen. 1998. Reflection in the Writing Classroom. Logan: Utah State University Press.


 David Hubert, assistant provost for learning advancement, Salt Lake Community College

Previous Issues