Peer Review

Portraits of Learning: Comprehensive Assessment through E-Portfolios in the Metro Academies Project

Why not establish an intimate connection between knowledge considered basic to any school curriculum and knowledge that is the fruit of the lived experience of these students as individuals?

—Paulo Freire, “Pedagogy of Freedom” (1998)

Candace Masaquel, one of five siblings raised by a single mother, had dreams of attending university but was concerned her family finances would not afford her the opportunity. Grateful to gain entry to a four-year program at San Francisco State University (SF State), Candace still felt trepidation about how she would manage her courses, fit in with the large student body, and ultimately forge her personal and professional path toward a future in health education. Candace’s fears were calmed, however, when she found a supportive home in the Metro Academies community. Metro is an academic program that redesigns the first two years of college with the inclusion of several high-impact educational practices that have been spotlighted by the Association of American Colleges and Universities (AAC&U). These practices include a learning community of two classes per semester for two years; a first-year experience course; common intellectual experiences; repeated practice of core academic skills in writing, critical thinking, oral communication, quantitative reasoning, and information literacy; collaborative assignments and projects, and exposure to social justice, cultural diversity, and global learning.

As Candace progressed through her first years of college with her Metro cohort, the electronic portfolio (http://candacemasaquel.myefolio.com/) she created prompted her to collect and reflect upon her academic and personal experiences. This process proved to be a grounding and transformative component of her Metro Academy experience: “I knew that I wanted to be a part of public health, but I didn’t know what that meant. My portfolio helped me to think, to really organize my thoughts and my work. Now I’m able to say, this is who I am and this is who I want to be in the future.”

SF State places a high priority on social justice and student success, especially for first-generation and underserved populations. This case study illustrates how a committed partnership between an academic program, Metro Academies, and a central service unit, Academic Technology, worked to provide and scale the transformative practice of e-portfolio assessment for this important group of students.

Metro Academies at a Glance

In 2007, a longstanding partnership between City College of San Francisco (CCSF) and SF State developed a curricular innovation aimed at increasing persistence and graduation for low-income and underrepresented students. After reviewing the higher education literature and studying student patterns at these two institutions, it was clear that the first two years were the leakiest part of the higher education pipeline. Metro is a multi-semester cohort-style learning community program, in which first-generation, low-income, underrepresented first-year students study together in two general education linked classes over three to four semesters. The curriculum is designed to engage students early in the big questions in their field of interest, such as health or science. Students are encouraged to support each other academically and socially, and to form strong relationships with faculty. In addition, Metro provides a series of wrap-around services, including tutoring, academic counseling, and financial aid support. With faculty from both segments working together on curriculum and assessment tools, this design fosters a very close alignment between the community college and university. Faculty also are required to participate in forty-five hours of professional development to enhance their teaching practices and build a community dedicated to student success.

The majority of Metro students place at two to three grade levels below college-ready English, with most testing as “double remedial” in both English and math. Despite such placement, data from Metro students completing the program show them strongly outperforming their peers at the same institution. The Metro students at SF State persist into their junior year at a 20 percent higher rate than their fellow students on campus, while the CCSF students are nine times more likely to be transfer prepared than a matched comparison group. Students in Metro are 10 percentage points more likely to graduate in four years and the anticipated five-year graduation rate is expected to outstrip the campus norm by 30 percentage points. Finally, a cost study of the Metro model finds that by reducing attrition and decreasing time to graduation, tremendous cost savings are realized on both campuses. At SF State an investment of $944 per Metro student leads to a cost reduction per graduate of $17,879. At CCSF, the investment of $1,484 per Metro student results in a cost reduction per transfer of $22,714. For details, visit metroacademies.org/news/coststudy.

The Culture of Assessment and Its Impact on Students

In their summary of e-portfolio practices, Clark and Eynon (2009) noted that a tension exists between educators who view the goal of assessment as producing evidence for accountability and accreditation purposes, and those who believe it should be used to create powerful personalized learning experiences. Metro embraces both aspects, taking the view that e-portfolios present the opportunity to collect and review an array of academic evidence and relevant “lived” expressions of learning in addition to its use as a tool for program assessment and change.

AAC&U has advocated for well-planned electronic portfolios that provide opportunities to collect data from “multiple assessments across a broad range of learning outcomes while guiding student learning and building self-assessment capabilities.” Furthermore, “assessment of work in (portfolios) can inform programs and institutions on progress in achieving expected goals” (AACU 2009).

Like other historically underrepresented students, Candace benefitted from engaging in e-portfolio assessment practices, rather than traditional testing. Candace’s e-portfolio gave her a place to explore and find her academic identity, and gave her instructors a way to authentically assess her work. Her collection of coursework and reflections demonstrated how she had integrated her learning across courses and lived experiences.

Planting E-Portfolios in the Metro Academies Program

Metro Academies presented a unique opportunity to use e-portfolios in a two-year learning community for first-time, first-year students from historically underrepresented communities. E-portfolios presented opportunities for a rich assessment that could help students see themselves as learners and recognize their growth over time.

Metro’s culture supported the integration of e-portfolios since faculty were interested in collecting multiple examples of authentic, reflective, and integrative student work in a variety of formats. They wanted to observe academic development over time, and have students collect evidence of context-rich integrative learning connected to their lived experiences. Faculty also wanted to track competencies in key foundation areas—writing, quantitative thinking, public speaking, and critical thinking. As part of the Metro/academic technology e-portfolio implementation project, a faculty workgroup and curriculum team adapted AAC&U Valid Assessment of Learning in Undergraduate Education (VALUE) rubrics and mapped signature assignments to the program-level student learning outcomes. E-portfolios in Metro represented an opportunity to ensure that the culture of assessment was “edu-diverse” from the start.

Obstacles to Overcome

Lacking dedicated resources and adequate staffing, this project initially faced several challenges. First, it was difficult to pool and promote educational best practices due to the fact that many of the sixty faculty members were part time and had different comfort levels with technology. Faced with crowded courses, some faculty felt pressure to “get through” the curriculum and found it difficult to integrate e-portfolio assignments. Many preferred to use e-portfolios to showcase completed, polished work, instead of works in progress. Second, faculty and student support needs varied over time, since a unique aspect of e-portfolio development is the simultaneous process of developing both technical and cognitive skills required for digital literacy development. While some students and faculty initially struggled with the technical skills, most students required more guidance in the process of posting reflections and content that appropriately showcased their work, and faculty in the process of nurturing and evaluating these contributions. Although this is a technology-enabled project, the reliance on academic technology to provide the technical solution has resulted in few technology obstacles, except for the logistical challenge of provisioning and transferring e-portfolio accounts as students pass through the program.

Successfully Building an E-Porticulture

E-portfolios have become an important avenue to collect, select, and reflect on Metro student work throughout their two years in the program. It gives students the opportunity to reflect on their learning both within and between courses. It also gives them a platform to bring their whole selves to their efforts to build identities as young scholars, embracing and integrating their ethnic heritage through the use of video, multimedia, photos, and stories. They even make good use of the language translator to share what they do with those in their lives who speak a language other than English, as many of our students speak a language other than English at home.

Students are seeing the value of the e-portfolios in their own lives. In an end-of-year survey compiled by the Connect to Learning network, one student said that using an e-portfolio “helped me see my assets” and another commented, “I was able to reflect back on the importance of the classes I have taken. I learned that each class and each activity contributes to the kind of person I am today.” Others commented on how using an e-portfolio allowed them to see their growth since their first year and realize how much they have accomplished. They are finding that building an e-portfolio is not only engaging and personally meaningful, but can help them put forward their best academic identities.

The Role of Academic Technology

The changing demographic of faculty and students in higher education has increased the expectations for academic technology services in ways that better conform to their experiences outside of the campus. Non-traditional students, who may attend part-time, work full-time, have dependents, or be single parents, require increased flexibility to complete their studies. To support the expanding role that technology now plays in the lives of all students and faculty, academic technology provides support wherever teaching and learning touches technology, including enhanced classrooms, video and media services, technology applications, and training and support. Academic technology helps faculty develop curricula in an expanding continuum of instructional modes that use technology, thereby helping ensure universally accessible, flexible, and meaningful learning experiences.

Now that academic technologies occupy a mission-critical status on campuses, exemplary implementations of technology-enabled initiatives such as e-portfolios need to focus on three intersecting elements:

  • Educational best practices, to enable best teaching and learning practices from an educational planning and assessment perspective;
  • Student and faculty support systems, to offer comprehensive and tailored pedagogical and technical support for faculty and students to design and engage in media-rich learning activities;
  • Technology infrastructure, to provide a robust, reliable, and scalable technological environment in which students and faculty can build, archive, and share their work.

Since 2005, academic technology at SF State has developed the organizational capacity and expertise to support roughly 30 percent of academic departments using e-portfolios. In addition to providing the educational best practices, student and faculty support systems, and the technology infrastructure that the departments rely upon, academic technology has also facilitated a cultural shift of assessment within programs. Drawing inspiration from “poly or permaculture” practices in agriculture, academic technology began to describe its work metaphorically as planting an “e-porticulture” across the campus. E-porticulture was defined as “the act or custom of learning, developing intellectually and professionally, and transmitting knowledge through the creation, review, and assessment of authentic, reflective, and integrative student work that is shared over time via electronic portfolios” (Shada et. al 2009).

Some of these cultural transitions have been direct, such as moving from required paper-based portfolios to similar online versions. Others have involved more groundwork since they involved shifting from “monoculture” practices, such as high-stakes summative testing, towards a range of more flexible assessment strategies.

Figure 1. Cohort-Style, Four-Semester Learning Community based on a GE Pathway

malikFig1.jpg

Faculty Learning Communities

One of the keys to Metro’s success has been its faculty learning community. Metro instructors work collaboratively to focus on student success. They learn how to use evidence-based educational practices to provide interactive, engaging instruction. This process fills a collegial vacuum felt acutely by part-time instructors who mainly teach lower-division courses. This community spans the general education curriculum and multiple disciplines, and represents an innovative approach to professional development. It gives instructors the opportunity to meet monthly to develop engaging pedagogy and share best practices. They discuss educational equity through the larger context in which our students attend college. Instructors of linked courses also have time to identify common themes and readings. These meetings lend themselves to conversations about student work, naturally integrating the e-portfolio into faculty development.

From Planting and Growing to Harvesting and Expanding

To address the success of academically underprepared students who are disproportionately of low-income and underserved backgrounds, colleges and universities must stop tinkering at the margins of institutional life, stop the tendency to take an “add-on” approach to institutional innovation, and adopt efforts that restructure the learning environments in which we ask students to learn (Engstrom and Tinto 2009).

The Metro project has created an opportunity to cultivate and break new ground with e-portfolios to support curriculum and assessment development. Partnering with academic technology has provided an incubator-learning environment to refine emergent educational best practices, faculty and student support structures, and the technological infrastructure needed to help a diverse group of students develop twenty-first-century digital literacy skills. This foundational work will become increasingly important as Metro scales up to serve 25 percent of all first-time freshmen at SF State. The use of e-portfolios for general education assessment is also under consideration. Metro Academies recently won the 2013 Most Visible Progress Degree Completion Award bestowed by the Association of Public and Land-Grant Universities.

Candace is now successfully on her path to global citizenship and will be graduating with a master’s degree in public health in 2015. Her undergraduate e-portfolio work gave her the opportunity to make an intimate connection between knowledge of her school curriculum and the fruit of her lived experience. Ultimately, e-portfolios are helping to restructure the environment in which Metro students like Candace are learning, and in the process are promoting a culture of direct evidence, continuous quality improvement, and campus-wide scholarship.

 

REFERENCES

Association of American Colleges and Universities. 2009. VALUE: Valid assessment of learning in undergraduate education. Accessed February 7. http://www.aacu.org/value/.

Clark, E., and B. Eynon. 2009. “E-Portfolios at 2.0, Surveying the Field.” Peer Review 11 (1).

Engstrom, C., and V. Tinto. 2008. “Access without Support Is Not Opportunity.” Change: The Magazine of Higher Learning 40 (1): 46–50.

Freire, P. 1998. Pedagogy of Freedom: Ethics, Democracy and Civic Courage. Lanham, MD: Rowman & Littlefield Publishers.

Malik, S. K. 2012. Engaging in the Beautiful Struggle: Influence of Faculty Learning Communities on Teaching. (Order No. 3549477, San Francisco State University). ProQuest Dissertations and Theses, 154. Accessed February 7. http://search.proquest.com/docview/1284158649?accountid=13802. (1284158649).

Shada, A., K. Kelly, R. Cox, and S. Malik. 2011. “Growing a New Culture of Assessment: Planting ePortfolios in the Metro Academies Program.” International Journal of ePortfolio Research 1 (1): 71–83.

Shada, A. 2011. Electronic Portfolio Implementation in the Metro Academies program at San Francisco State University and City College of San Francisco. Unpublished master’s thesis. Mills College, Oakland, California.


Savita Malik is the director of curriculum and faculty development; Alycia Shada is the manager of data and communications at Metro Academies; Ruth Cox is a faculty liaison to academic technology and lecturer in health education; Maggie Beers is the director of academic technology; Mary Beth Love is a professor and chair of health education and the executive director—all of Metro Academies, San Francisco State University.

Previous Issues