Diversity and Democracy

Diversity Assessment, Accountability, and Action: Going Beyond the Numbers

When campuses begin assessing diversity, most focus on the numbers of men and women and the numbers of individuals from historically underrepresented groups among students, staff, and faculty across departments and units. Institutions often circulate annual reports on "the numbers," occasionally broken down across units, to prompt discussion. Some campuses use slightly more sophisticated metrics to focus on relative representation (or underrepresentation) of individuals from particular groups: for example, comparing the demographics of admitted students to those of high school graduates eligible for college, or juxtaposing the demographics of recently hired faculty and staff with those of the hiring pool. But at the national level, relatively little is known about whether campuses take significant action in light of these annual reports, or how campuses hold individual units accountable for progress on diversity goals.

Institutional efforts to monitor compositional diversity are important, because such diversity not only reflects equity within institutions, but also affects how individuals perceive and experience the work and learning environment (Hurtado et al. 2012)—that is, the psychological and behavioral dimensions of campus climate. In advancing the review of compositional diversity, some campuses have used an Equity Scorecard developed by the Center for Urban Education at the University of Southern California to evaluate such diversity and prompt discussions related to diversity goals in a variety of functional areas (Bauman et al. 2005; Williams, Berger, and McClendon 2005). The Equity Scorecard prompts campus leaders and practitioners to evaluate their institutional performance using disaggregated data and to initiate action across the institution in response to their findings.

While it is essential to examine representational equity, it is also important to document other dimensions of institutional diversity that contribute to campus climate, including perceptions and behaviors among individuals and groups, and organizational practices across units. Using new methods to capture and disseminate data about these dimensions, campus leaders and practitioners can enhance learning and accountability across the organization. The purpose of this article is to highlight approaches to and uses of diversity assessments with implications for organizational learning. Such assessments can prompt changes in behavior and make achieving diversity goals a responsibility that is shared widely across the institution.

Linking Quantitative and Qualitative Data

In addition to accounting for compositional diversity across the institution, many campuses have opted to assess the climate for undergraduate students (and occasionally for faculty and staff), often in reaction to an incident or a series of incidents. It is important to conduct climate surveys not merely in reaction to challenging events, but also as part of proactive, ongoing assessments of the environment experienced by students and those who work at the institution. Climate surveys can signal to all that the campus is concerned about improving the work and learning environment.

Campus leaders can take steps to optimize the usefulness of climate surveys. Those charged with leading assessment efforts can proactively address two dilemmas that often arise with regard to climate surveys: the need to determine a priori the survey questions and response categories, which may or may not reflect the experiences of various groups on campus; and the question of whether to include small numbers of specific groups in reports (since on the one hand, it is difficult to determine statistically significant differences with small sample sizes; but on the other, continuing exclusion of these subgroups prevents campuses from identifying factors that affect their underrepresentation).

One campus faced these challenges when it collected student survey data on inclusion and engagement and reported findings for all groups except those with the smallest sample sizes (e.g., Native American students and specific Asian American subgroups). To provide a more complete assessment that allowed for better understanding of the quantitative survey data, especially findings for small groups, the campus sought to collect qualitative data during the subsequent term by conducting focus groups with students from several diverse communities on campus (e.g., underrepresented racial and ethnic groups, LGBTQ students, and low-income students). These focus group meetings provided an occasion for students to voice feelings and detail interactions that revealed "invisibility" as a major theme across several groups. This theme was not captured in the design of the climate survey nor reflected in the responses of specific groups. Had campus leaders relied only on the campus climate survey report, they might have further contributed to students' feelings of invisibility.

The focus group report documented and affirmed that individuals were not alone in feeling invisible, presented common stories of discrimination and bias that faculty could use for teaching about diversity on campus, and provided additional insight into how to improve the campus climate survey on its next administration. Importantly, the institution made both the survey report and the focus group report publicly available on the college's website, where visitors could use a dashboard to view survey responses by group and could read powerful stories and quotes from students and staff. Both reports were widely shared across the institution, providing useful information for planning and ongoing initiatives across units.

Connecting Assessment Strategies

Campus climate surveys are commonly administered as stand-alone initiatives unrelated to other data collection efforts on campus. Yet research has shown that the climate for diversity is not an isolated phenomenon, but affects students' transitions to college and educational outcomes (Hurtado et al. 2012), as well as faculty and staff members' satisfaction with the quality of work-life. Such connections are not widely known on individual campuses unless those campuses make an effort to link climate survey data with data from other sources (e.g., enrollment data, outcomes data, or data on faculty and staff attrition).

By making connections between these different data sets, campus leaders can tie campus climate assessments to outcomes that affect student success and important work-life issues in departments and campus administrative units. Campuses should integrate and link climate assessments with other ongoing data collection initiatives, and should use all these data sources to inform practice. For example, linking data on students collected through the Cooperative Institutional Research Program's (CIRP's) Diverse Learning Environments Survey with student enrollment data can help institutions identify key factors in the climate and learning environment (e.g., students' validation and sense of belonging) that could be enhanced by implementing institutional practices to mediate reductions in student attrition.

Campus review and reward processes represent another area where regular assessment takes place in the form of annual reports on staff and faculty performance and teaching evaluations at the end of each term. Because these review and reward processes shape expectations for behavior and can convey diversity as a core value at all levels of the institution, several campuses (e.g., the University of Michigan) have incorporated questions about inclusive pedagogy and diversity experiences into course teaching evaluations. At the beginning of each term, faculty can choose from a set of optional questions addressing diversity and inclusion to be included in their students' end-of-term course evaluations, usually in addition to required items about the quality of instruction.

With the addition of these questions, the student evaluations provide information about classroom climate and faculty behaviors, and can serve to advance conversations within departments about inclusive pedagogical techniques. The practice of including these questions also embeds support for diversity in the classroom as a core value in the campus reward system, since teaching evaluations play a role in tenure and promotion processes. Some campuses (e.g., the University of California–Los Angeles) now require reporting of individual faculty members' diversity efforts in their promotion and tenure materials and include activities that advance diversity as an area of performance in staff evaluations. By incorporating diversity in review and reward processes, campus leaders can influence behavior and promote conversation across campus about shared responsibility for diversity goals.

Mapping Institutional Diversity Efforts

Until recently, there was no systematic way to collect information about the organizational dimension of diversity (e.g., curriculum, policies, and budget allocations), even though this dimension has been hypothesized to shape the climate for diversity on campus (Milem, Chang, and Antonio 2005). Several institutions are now attending to the organizational dimension of diversity by documenting initiatives and institutional change related to structures, policies, and practices. While many campuses have strategic plans focused on diversity, additional efforts are needed to tie accountability and metrics to diversity planning goals (Williams, Berger, and McClendon 2005). A campus-wide diversity framework can help institutions ground and present an integrated approach, although institutional performance related to diversity can vary across units and depends on many actors working together to achieve progress.

To address the challenge of developing an overview of campus activities that advance diversity goals across units, Rona Halualani and her research team at Halualani and Associates developed "diversity mapping" as a form of inquiry and a research methodology with new metrics for benchmarking institutional change related to diversity. Diversity mapping is a reflexive practice focused on identifying where a college or university is—in terms of values, principles, objectives, goals, outcomes, and resource allocations—with regard to establishing a deeply embedded campus structure grounded in diversity (Halualani, Haiker, and Lancaster 2010). This process, which was created in 2007 when Halualani served as the diversity leader at San Jose State University, involves a team of researchers from Halualani and Associates evaluating an institution's extant diversity efforts and curricula over a period of ten months to gauge that institution's commitment to and investment in diversity and inclusive excellence.

More specifically, diversity mapping entails "taking stock of current diversity efforts and then analyzing such mappings to identify the current status of inclusive excellence at that institution" (Halualani, Haiker, and Lancaster 2010, 127). The process helps higher education institutions locate their actual (not projected) engagement with and implementation of diversity efforts by producing a visual map that reflects all diversity initiatives, programs, events, and even curricula across the institution, at all levels and within all units. Such diversity mapping represents a valuable process of inquiry for any campus beginning to form an institution-wide diversity strategy or master plan. This practice can provide a sense of where the institution has been, where it currently is, and how it has operationalized diversity and inclusive excellence, in both intentional and unintentional ways. This mapping involves more than just taking an inventory or engaging in a diagrammatic exercise; instead, it is a meaningful practice of inquiry through which singular pieces of information that are typically isolated within campus silos are organized and framed in relation to one another. The resulting holistic portrait provides a comprehensive overview of diversity on campus at both structural and thematic levels (see figure 1 for an example map). The diversity map allows practitioners to locate duplication in efforts, empty zones or areas of neglect, and practices that are more nominal than functional.

Figure 1: Segment of a Campus Diversity Map

Click here to expand figure 1

To enhance the mapping process and categorize stages of development, Halualani and Associates have developed a set of diversity analytics. After gleaning information from campus websites, documents, and discussions with staff and faculty, several areas are coded by Halualani and Associates researchers to produce additional layers of organizational analysis. These analytical layers include (but are not limited to)

  • year of effort;
  • level of focus (primary or partial);
  • division/departmental location;
  • level of integration (connections and links among divisions);
  • type/theme of diversity effort;
  • change order (first-, second-, third-, and fourth-order change);
  • number of times a diversity effort is highlighted in campus news and media;
  • innovation score (the extent to which an institution pursues new practices);
  • target population (all campus constituents, leadership, faculty, staff, students, or community members);
  • initiation point (university-wide or program-driven);
  • target focus (mainstream or focused on a specific group);
  • Diversity Engagement/Learning Taxonomy Assessment (DELTA) (based on seven levels of diversity engagement and learning);
  • student stage (class level of targeted student population);
  • definitions of diversity used in efforts;
  • type of discourse around diversity (e.g., equal but separate, cultural pluralism, colorblind equality, social justice, identity inequalities);
  • specific questions about diversity that the campus is exploring;
  • prospective reach level (how many people are likely affected);
  • enduring factor level (time frame/sustainability of effort);
  • number of direct and indirect measures of impact (actual data on outcomes).

To date, twenty campuses (of varying sizes and types, public and private, two-year and four-year, faith-based and secular) have conducted diversity mapping as part of their strategic planning efforts and can now use the analytics to measure their progress toward diversity goals. Many of these campuses have been large and organizationally complex, and the mapping exercise has demonstrated connections that any one unit or administrator would otherwise have difficulty seeing. Some campuses have used the diversity mapping exercise as a baseline measure before engaging in strategic efforts and actions related to diversity. The maps and embedded analyses highlight the extant leverage points, current resources, empty zones, and areas of untapped potential, helping institutions understand what they need to do in order to fully embody a commitment to diversity and inclusive excellence.

Promoting Organizational Learning for Institutional Change

When collected and disseminated extensively, diversity assessment data can provide new knowledge about the campus climate; the experiences of students, faculty, and staff; campus practices; and equity in outcomes. It can also create greater awareness about what it means to foster inclusive excellence on campus. However, changes in individual mindsets and behaviors will not result in institutional change without unwavering support from institutional leaders, both those in formal leadership positions and those working at the grassroots level (Kezar 2014). By incorporating diversity assessment into regular campus processes (e.g., assessments, rewards, and budgeting), positional leaders in particular can send the message that diversity is everyone's responsibility.

Institution-wide change requires organizational learning and authentic forms of professional development that empower faculty (both full- and part-time) and all levels of staff to implement transformative practices that advance diversity and student success. Campus leaders cannot expect new behaviors to emerge without providing guidance, which can take the form of information sessions and opportunities to learn what pedagogy looks like in an inclusive classroom. Learning communities for faculty and staff provide useful opportunities to share knowledge, information, and problem-solving strategies. Some institutions encourage individuals to learn from peers on other campuses as they seek ways to advance diversity goals and student success. Campus leaders need to empower educators to become change agents. As one student affairs professional stated, "Institutions do not make things happen, people do" (Hurtado, González, and Calderón Galdeano, forthcoming).

Sylvia Hurtado and Rona Halualani offer insights on diversity research and assessment at the Diversity Research Institute sponsored by the Higher Education Research Institute at the University of California–Los Angeles each summer. For information, see http://heri.ucla.edu.

References

Bauman, Georgia L., Leticia Tomas Bustillos, Estela Mara Bensimon, M. Christopher Brown, II, and RoSusan D. Bartee. 2005. Achieving Equitable Educational Outcomes with All Students: The Institution's Roles and Responsibilities. Washington, DC: Association of American Colleges and Universities.

Halualani, Rona T., Hugh L. Haiker, and Christopher M. Lancaster. 2010. "Mapping Diversity Efforts as Inquiry." Journal of Higher Education Policy and Management 32 (2): 127–36.

Hurtado, Sylvia, Cynthia L. Álvarez, Chelsea Guillermo-Wann, Marcello Cuellar, and Lucy Arellano. 2012. "A Model for Diverse Learning Environments: The Scholarship on Creating and Assessing Conditions for Student Success." In Higher Education: Handbook of Theory and Research, edited by John C. Smart and Michael B. Paulsen, vol. 27, 41–122. New York: Springer.

Hurtado, Sylvia, Rene González, and Emily Calderón Galdeano. Forthcoming. "Organizational Learning for Student Success: Cross-Institutional Mentoring, Transformative Practice, and Collaboration among Hispanic-Serving Institutions." In Hispanic-Serving Institutions: Advancing Research and Transformative Practice, edited by Anne-Marie Núñez, Sylvia Hurtado, and Emily Calderón Galdeano. New York: Routledge.

Kezar, Adrianna. 2014. How Colleges Change: Understanding, Leading, and Enacting Change. New York: Routledge.

Milem, Jeffrey F., Mitchell J. Chang, and Anthony L. Antonio. 2005. Making Diversity Work on Campus: A Research-Based Perspective. Washington, DC: Association of American Colleges and Universities.

Williams, Damon A., Joseph B. Berger, and Shederick A. McClendon. 2005. Toward a Model of Inclusive Excellence and Change in Postsecondary Institutions. Washington, DC: Association of American Colleges and Universities.


Sylvia Hurtado is a professor and director of the Higher Education Research Institute at the University of California–Los Angeles; Rona Halualani is a professor of language, culture, and intercultural communication at San Jose State University and the founder of Halualani and Associates.

Previous Issues