Liberal Education

Connecting the Dots: A Methodological Approach for Assessing Students’ Civic Engagement and Psychosocial Well-Being

As the saying goes, there are more ways than one to skin a cat. The same goes for assessment. Choosing one of the myriad instruments by which to measure civic engagement and psychosocial well-being is relatively easy. The hard part is choosing the one—or two, or three—that will actually connect the most salient elements of civic engagement and psychosocial well-being to the goals of a particular course, project, or institution. The important thing to remember in any discussion of the connection between civic engagement and psychosocial well-being is that neither of these concepts is unidimensional. Each is a composite. One’s civic engagement, like one’s psychosocial well-being, is an amalgam of multiple strands of measureable outcomes related to individual thought, perception, and action. How, then, can the linkages between students’ civic development and their psycho­social well-being be meaningfully defined and assessed at the campus level?

Get a definition

Given that the multidimensionality of civic engagement and psychosocial well-being provide opportunities for measuring many different things, it is essential that the process of assessment begin by interrogating the concepts themselves. For institutional or programmatic assessment, it is helpful to understand both how these concepts are used on campus and the ways in which they have been defined and contextualized within existing documents (e.g., mission statements, strategic plans, and syllabi). The following three questions offer starting points for interrogation.

1. What does it mean to be civically engaged? Civic engagement can mean many things on a campus. The term may be nonexistent, consistently conflated with service learning or community-based learning, enmeshed with the language of social or political action, or distinctly articulated. In order to identify a valid measure of students’ civic engagement, it is necessary first to understand what the term means within particular contexts. Is the presence of civic engagement equated with the performance of service learning or volunteer work on campus? If a student is in a service-learning course, does this mean he or she is civically engaged? What are the boundaries of civic action? Is it enough to know about students’ engagement in the context of their civic obligations (e.g., voting behavior) or is engagement also—or instead—an exercise of civic opinion (e.g., organizing a protest, writing a petition, convening a meeting)?

2. What counts as a definable community for civic action? Webster’s dictionary defines a community as a “unified body of individuals.” Therefore, it is important to assess, or at least to consider, the proximal range of civic involvement that defines students’ engagement. For example, the disaggregation of attitudes and behaviors across a range of community contexts—campus, local community, national, or international—may be helpful for understanding the breadth and depth of students’ civic engagement.

3. What is psychosocial well-being? As Lynne Friedli notes, “there is widespread agreement that mental health is more than the absence of clinically defined mental illness” (2009, 10). In other words, we cannot fully assess the dimensions of psychosocial well-being by, for example, deploying a standardized scale to determine whether students are depressed. Psychosocial well-being, like civic engagement, is defined by many interlocking components drawn from elements of an individual’s short-term and long-term affect, outlook, and social functioning. Keyes (2002), for example, defines psychosocial well-being, or positive mental health, through the lens of “flourishing,” a combination of an individual’s hedonic tendencies (positive feelings) with their eudemonic behaviors (positive functioning).

Get operational

After defining what the “thing” is, the next step is to use those defining qualities to choose an appropriate form of measurement. To “operationalize” a concept is to untangle dimensions until there is clarity on what can actually be measured or assessed. For example, the notion of “time spent” is often a central measure by which to assess degrees of civic engagement. But is it enough to ask how frequently one engages in community life, either by asking how often or by counting the number of hours? Or is it also important to know what happened during that time? There is no right answer, but the substantive difference between these questions changes the nature of the measure from one that equates engagement with time to one that equates it with characteristics of time.

Is civic engagement a behavior or an attitude? To be civically engaged is to be what exactly? The concept connotes ways of doing, knowing, perceiving, and ways of planning for future action. It is not possible to measure every aspect of civic engagement, but measurement should, at a minimum, account for both behavioral and attitudinal expressions of engagement. Responding to the following questions can help account for behavioral expressions of engagement:

  • In addition to time spent, what are the types and/or amount of civically oriented or focused activities students are engaged in?
  • What plans do students have for future engagement?
  • How and in what ways have students’ engagement in civic affairs or community life changed over time?

Similarly, responses to these questions can help account for attitudinal expressions of engagement:

  • How have these experiences affected students’ views or perceptions of social issues, processes, or the role of individuals within society?
  • How have students’ civic experiences affected their moral or ethical development?
  • In what ways have student attitudes toward diverse groups, or people unlike themselves, changed as a result of the experiences?

Next, what qualities of students’ psychosocial well-being are relevant for assessment? The answer is likely “all of them.” The real puzzle for researchers, faculty, and institutions is more specific: which of the qualities of positive mental health and social functioning are most connected to the civic engagement experiences students will have? On campuses, where discussions of mental health are often left to student affairs, defining psychosocial well-being means ascertaining the applicability of this concept within a larger discussion of students’ civic engagement and, more broadly, in relationship to student learning. With regard to students’ civic engagement, there are several important questions to ask:

  • What impact does this experience have on students’ sense of hope?
  • How does engagement in the community affect students’ feelings of self-efficacy or resilience?
  • In what ways do these experiences broaden students’ sense of diversity and comfort working with people unlike themselves?
  • How have these experiences changed, deepened, or developed students’ connections with peers, faculty, and community members?

Get a map

Any good assessment plan starts with a map showing where you are and where you want to go. An evaluation of the relationship between civic engagement and psychosocial well-being should also be mapped in order to better understand the direction and conceptualization of these terms in relation to each other. One could imagine a causal map, but establishing causality is not—and really should not be—the goal. Instead, the idea is to confront assumptions about how the mechanisms of civic engagement and psychosocial well-being work, and realistically to evaluate where assessment of each should occur.

Is civic engagement a program or an outcome? When assessing civic engagement, it is critical for campuses to distinguish the capacity in which civic engagement will be expressed. Is civic engagement assumed as a quality of a program (e.g., a service-learning course or community-based-learning project) in order to assess other outcomes (specifically psycho­social well-being, but also other outcomes like retention, persistence, and depth of learning)? Or is civic engagement the outcome of a program or group of experiences after or during which qualities of civic engagement will be assessed (e.g., civic-mindedness, moral development, openness to diversity, and attitudes toward social or political action)? The differences among these assumptions are paramount to how instruments should be chosen and how the evaluation should be conducted.

How can psychosocial well-being be fully captured, given the expanse of student life and interaction on campus? Far more than civic engagement will impact students’ psychosocial well-being while in college. Moreover, it is impossible to control, or perhaps even to account, for all the potential influences on psychosocial outcomes within the noisy social and emotional realm of college life. A well-developed programmatic map can, however, detail the ways in which students are being engaged civically and are engaging with others around civic issues. With greater knowledge of the multiple areas in which civic participation, action, or collaboration is being borne out in students’ lives, we can more discernibly connect the degree and dimensionality of these occurrences with psycho­social outcomes. Engaging the knowledge base and resources of those in student affairs can be a vital part of the outcomes mapping process.

Should outcomes related to civic engagement and psychosocial well-being be assessed before, during, or after the program? Both civic engagement and psychosocial well-being are closely connected with processes of student development. The manifestation of an outcome associated with either concept will not occur neatly or linearly upon completion of an experience, course, or academic year. Ideally, assessment should be attentive to student development by carefully mapping, over time, a protocol for formative evaluation that incorporates multiple outcome dimensions for exploration in the context of developmental learning and growth. At a minimum, quantitative instruments should be aligned with qualitative reflection throughout the experience(s) and gathered in a way that enables students to self-evaluate outcomes. One way to build both formative and summative assessment around critical reflection and other products of student work (e.g., papers and group assignments) is to enlist rubrics as part of the assessment plan. Rubrics on civic engagement, ethical reasoning, and dimensions of social development (e.g., teamwork and intercultural competency) provide opportunities for direct assessment of student learning, and are also an accessible means by which to engage students in their own self-assessment of these outcomes.

Whatever approach is taken with regard to the assessment of civic engagement and psycho­social well-being, the goal should be to tell a coherent story about the ways in which these experiences shape students’ lives and learning. The aforementioned suggestions are simply guides for developing the story in ways that attempt to capture both the richness of the experiences and the attention of the campus audiences who will be privy to the storytelling. And we must also remember that despite the reliability of our scales and the impressiveness of our data, the best storytellers are often the students themselves. Any really good assessment plan should leave sufficient space for the inclusion of student voice on these matters.

Selected Resources

Bringing Theory to Practice (BTtoP)
In partnership with the Association of American Colleges and Universities, the BTtoP project supports campus-based initiatives that demonstrate how uses of engaged forms of learning—actively involving students both within and beyond the classroom—directly contribute to students’ cognitive, emotional, and civic development.
http://www.aacu.org/bringing_theory

Campus Compact
Campus Compact is a national coalition of college and university presidents dedicated to promoting community service, civic engagement, and service learning in higher education.
http://www.compact.org

Center for Information and Research on Civic Learning and Engagement (CIRCLE)
Based at the Jonathan M. Tisch College of Citizenship and Public Service at Tufts University, CIRCLE conducts research on the civic and political engagement of young Americans.
http://www.civicyouth.org

Center for Social Concerns
The service and community-based learning center of the University of Notre Dame, the Center for Social Concerns provides educational experiences in social concerns inspired by Gospel values and the Catholic social tradition.
http://socialconcerns.nd.edu

Core Commitments: Educating Students for
Personal and Social Responsibility
An initiative of the Association of American Colleges and Universities, Core Commitments helps campuses create learning environments where all students reach for excellence in the use of their talents, take responsibility for the integrity and quality of their work, and engage in meaningful practices that prepare them to fulfill their obligations in an academic community and as responsible global and local citizens.
http://www.aacu.org/core_commitments

Deliberative Democracy Consortium
The Deliberative Democracy Consortium is a network of practitioners and researchers—representing more than fifty organizations and universities—collaborating to strengthen the field of deliberative democracy. The consortium seeks to support research activities and to advance practice at all levels of government, in North America and around the world.
http://www.deliberative-democracy.net
 
New England Resource Center
for Higher Education (NERCHE)
NERCHE is committed to collaborative change processes in higher education to address social justice in a diverse democracy. As a center for inquiry, research, and policy, NERCHE supports administrators, faculty, and staff in becoming more effective practitioners and leaders as they navigate the complexities of institutional innovation and change.
http://www.nerche.org

Positive Psychology Center
Located at the University of Pennsylvania, the Positive Psychology Center promotes research, training, education, and the dissemination of “positive psychology,” the scientific study of the strengths and virtues that enable individuals and communities to thrive.
http://www.ppc.sas.upenn.edu

Valid Assessment of Learning in Undergraduate Education (VALUE)
An initiative of the Association of American Colleges and Universities, VALUE helps campuses define, document, assess, and strengthen student achievement of essential learning outcomes as well as develops ways for students and institutions to collect convincing evidence of student learning. http://www.aacu.org/value

References

Friedli, L. 2009. Mental Health, Resilience and Inequalities. Copenhagen: World Health Organization.

Keyes, C. 2002. “The Mental Health Continuum: From Flourishing to Languishing in Life.” Journal of Health and Social Research. 43: 207–22.


Ashley Finley is director of assessment and research at the Association of American Colleges and Universities and national evaluator for the Bringing Theory to Practice project.


To respond to this article, e-mail liberaled@aacu.org, with the author’s name on the subject line.

Previous Issues