Select any filter and click on Apply to see results
Table of Contents
From Application to Action Plan: How the Language of Gen Ed Reform Changes over Time
For over twenty-five years, campuses have been sending teams to the Association of American Colleges and Universities (AAC&U) Institute on General Education and Assessment (www.aacu.org/summerinstitutes/igea/2018). Every summer, teams comprising of at least five people, typically including faculty and senior administrators, come ready to dig in and “fix general education.” The fix is largely about making general education more relevant for students, better understood by faculty, and better aligned with institutional missions and priorities. Teams arrive hopeful and ambitious. They are not seeking ways to simply save general education. They are seeking the ways to make general education flourish.
The process for selecting teams for the institute begins with an application. The application is designed to get a sense of what the team’s priorities are, how they are approaching the work ahead, and what they see as primary areas of strength heading into the reform process. The application specifically prompts teams to consider areas of need or interest pertaining to the General Education Maps and Markers (GEMs) design principles for general education: proficiency, agency and self-direction, integrative learning and problem-based inquiry, equity, and transparency and assessment (www.aacu.org/gems). Teams selected for the institute may be at any stage of general education reform—planning, developing, implementing, or refining. Because of space constraints, not all applications submitted can be accepted. The primary criteria for selecting teams for the institute are elicited by two questions: “Does the team have a sense of what they want to accomplish while at the institute” and “Why does this summer’s institute matter for advancing the work?” The goal of proposal reviewers is to determine which teams might be helped the most by the allotted team time, faculty expertise, and session attendance.
Though the applications offer insights into a team’s goals for the institute, it is expected that their foci will shift over the course of the institute as participants reflect, collaborate, and work on professional development. All teams must produce and present an action plan at the completion of the institute. Action plans encapsulate the steps and priorities for carrying the reform work forward once team members have returned to campus. Plans typically include (re)articulations of vision statements or imperatives for undertaking general education reform, timelines for engaging stakeholders (particularly faculty), and strategies for implementation (typically assessment and/or high-impact practices). Table 1 provides the prompts given to teams to complete institute applications and for creating action plans. It is important to note that while the institute application explicitly guides teams to consider AAC&U’s GEMs principles of general education reform, the action plan prompts are more broadly conceived so as to enable teams to authentically articulate their own pathway forward.
In this article, we present an analysis to understand how team priorities and objectives change from the time they submit their institute applications to when they deliver their action plans. The goal of this examination is to identify the thematic, sometimes subtle, shifts in language and areas of reprioritization that underlie general education reform efforts. Though the time from application submission to action plan completion is relatively short—a span of about five months—a good deal can change with the completion of an academic year, professional development at the institute, and time for reflection. Because of this, the following findings are offered as insights for any campus undertaking general education reform to increase their odds for success by illuminating the path from starting point to actionable steps.
Table 1: Application and Action Plan Prompts
|Application Short-Answer Questions||Action Plan Questions|
“[P]lease tell us briefly about your expectations for the institute in terms of topics, issues, or interests related to the components of AAC&U’s General Education Maps and Markers Framework”:
Table 2. Institutional Demographic Data for Forty-Eight Teams Attending IGEA 2018
|Two- or Four-Year||Two-Year||8|
|AAC&U Member Status||Member||42|
For this analysis, we used the applications and action plans submitted by the forty-eight teams who attended the June 2018 Institute on General Education and Assessment. Table 2 shows descriptive information about the teams and their institutions. We analyzed applications and action plans using NVivo 12 Pro qualitative data analysis software to identify common word counts within and between the two different documents. The NVivo software helped us identify trends in the use of language and thematic areas of focus across teams from their applications to their action plans.
When looking at simple word clouds (see figs. 1 and 2) of words used in team applications and action plans, the words that were used most often in the texts appear in the largest font size. The five most frequently used words in both applications and action plans were “faculty,” “students,” “educational,” “assessment,” and “general.” However, a closer look at the use of words and topic areas yielded noticeable differences in the frequency and use of language between the two documents. Findings indicate rather substantive shifts in the language used to indicate priorities around assessment, high-impact practices, equity, and timelines for action steps.
It is no surprise that teams undertaking general education reform are interested in assessment. The word “assessment” appeared 764 times in the applications of campuses attending the institute in 2018, far more than any other term analyzed. However, in the analysis of action plans, the term appears only 434 times, a decrease of 43 percent. A thematic review of action plans suggests this decrease is not because teams are less interested in assessment. Rather, it appears to be due to teams becoming more focused on the role and execution of particular elements of the assessment process, namely rubrics and assignment design. The institute provides multiple sessions on the background, national use, and campus application of the AAC&U VALUE rubrics as part of a comprehensive strategy to support direct assessment in general education (www.aacu.org/value/rubrics). This strategy requires the careful construction of assignments to reflect the broad learning outcomes of general education and associated rubrics for assessment. Thus, while mention of rubrics tended to increase from applications to action plans, the increased mention of “assignments” (by 129 percent) was particularly notable. The following quotes from action plans help to illustrate this point:
Workshops for faculty on developing ‘signature assignments’ that directly assess CapCore learning outcomes and/or how to translate current assignments to directly assess CapCore learning outcomes.
To assist with assessment best practices, we will create common ‘signature [general education] assignments’ and rubrics across the university.
—Clark Atlanta University
In August, the CSB/SJU faculty come together for an all-faculty workshop. We created a plan for this workshop. After a very brief overview, we will facilitate small-group discussion and an activity centered on creating signature assignments that would go into student ePortfolios, based on our themed courses that faculty from all disciplines will contribute to. This workshop will introduce faculty to the implementation timeline and processes, and we want excitement to continue to develop around the Integrations Curriculum.
—College of Saint Benedict and Saint John’s University
Our previous assessment cycle indicated that the assignments that are being used to assess our outcomes are not providing the data that we need. Recognizing this need and then benefiting from some excellent instruction at the conference has given us many ideas for the development of signature assignments with a more reflective pedagogy. Professional development that focuses on creating signature assignments that clearly link back to our general education outcomes will be a key piece to accomplishing this. . . . In addition, we will plan on developing clear curriculum maps that map assignments back to general education outcomes. This will hopefully identify gaps in our assignment structure that will help inform the professional development that we can offer in these areas.
—College of Southern Idaho
Year 2—Faculty professional development. Signature assignment training in addition to VALUE rubrics. (VALUE Institute in 2019—certified to be a scorer).
—Our Lady of the Lake University
These ideas [for our action plan] centered around helping faculty better design assignments, changing the course approval/review process, improving the supporting materials the college offers faculty, and improving how we assess information literacy in student ePortfolios.
—Salt Lake Community College
Though it is hard to know exactly what this shift in language means, given the emphases of the institute curriculum we posit that teams are better able to specify their assessment plans over time. The “need” for assessment that is broadly and vaguely articulated in institute applications becomes honed with the development of action plans. Campus teams replace the language of assessment itself with more specific language around the tools for execution—rubrics and assignments.
Figure 1. Word Cloud of Team Applications
Figure 2. Word Cloud of Team Action Plans
Connecting the Dots from Hips to ePortfolios
Similar to the trajectory of assessment design, the NVivo analysis also revealed a compelling trend in how teams articulated the role of high-impact practices in general education reform. As with assessment, global references to “high-impact practices” or “HIPs” decreased from applications to action plans by 67 percent. References to specific high-impact practices, such as learning communities, service learning, undergraduate research, writing-intensive courses, and internships also, collectively, decreased by 54 percent. This is unlikely to be because teams thought less about the value of these practices, particularly given the extent to which these pedagogies were promoted as powerful tools during the institute. There is plenty of research to suggest HIPs are beneficial for student learning, which is widely cited across the institute curriculum. A more likely explanation for teams’ use of the term “HIPs” occurring less often may have something to do with the increased use of the term “ePortfolios.” This term increased by 60 percent from application to action plans as teams likely considered the role of this high-impact practice among the breadth of HIPs already employed on their campuses.
Though the institute offers multiple sessions on ePortfolio pedagogy and national ePortfolio experts are part of the institute faculty, these factors alone are not reason enough for this trend. The visible placement of ePortfolios within the institute is intended to emphasize the utility of this tool for learning and assessment. In short, ePortfolios are touted as a powerful medium for transparently and coherently connecting the dots between courses, academic learning, cocurricular learning, and institutional outcomes. In many ways, ePortfolios are the workhorse of general education reform because they can be used to address multiple issues (e.g., transparency, engaged learning, reflection, direct assessment). But, as the action plans also reflect, that comes only with careful planning, messaging, and implementation. For example, the following action plan excerpt details how ePortfolios will be used across all four years of a student’s college career:
This fall (August 2018) we will introduce our plan for the ePortfolio along with the current literature and best practices related to Folio Thinking to our first-year seminar and first-year writing course faculty. . . . Workshops focused on ePortfolio course and assignment design will be offered multiple times during [the year]. These courses will also include signature assignments and reflections. During the sophomore experience, students will take a required sophomore seminar. During this seminar, students will explore off-campus problem-based projects. The work of this project might be a mini-portfolio in itself. This course will provide a guided opportunity to do signature work and reflection, resulting in material ready to be used in the larger ePortfolio. Our juniors will also have a focused portfolio reflective piece that must be completed by the end of the junior year. . . . Advisors will discuss the creation of this reflective piece with students during . . . registration, as a discussion about the curation of the student’s story up to this point and the future curation of their story. . . . The feedback [on the portfolio from students’ advisor or designated staff] will be provided before the start of their senior year. The senior ePortfolio experience will document capstone work in programs and majors.
Another institute team detailed how their institution would develop and implement the ePortfolio alongside a robust plan for direct assessment:
Our proposed timeline is as follows:
This Summer 2018
- small work groups to further develop ideas (core pathways, ePortfolios, structure, etc.)
- small work groups expand to engage full faculty and other stakeholders (staff, students)
- ePortfolio pilot with selected faculty/courses
- identify and begin collecting baseline data for later comparison
- New Course and Core Curriculum Committee (NC4) to establish process for course/core approval
- expand ePortfolio pilot to include more faculty/courses
- continue collecting baseline data
Spring and Summer 2019
- faculty development for course revisions/alignment
- implement new core for first-year students
- full ePortfolio implementation for first-year students
- begin collecting comparison data
Leveraging Faculty Development
Equity is a core component of the GEMs framework and, therefore, an important area of emphasis for the institute curriculum. Teams are encouraged to consider not only how they will reform general education, but how students, particularly those from historically underrepresented groups, will be differentially affected and supported by the resulting curriculum and outcomes. These are not easy discussions to have, at the institute or back on campuses. Thus, along with strategies for disaggregating data to examine equity gaps, teams receive guidance on how to structure faculty development opportunities and to meaningfully engage colleagues in discussions around data, equity, and the reform process.
Perhaps because of this, action plans reflect far greater intent to engage faculty than referenced in applications. For example, though the word “communicate” was used roughly the same number of times between applications and action plans (fifteen instances and twelve instances, respectively), use of words like “faculty development,” “retreat,” and “workshop” collectively increased by 63 percent. Notably, the term “buy-in” was used 72 percent less often in action plans. Institute faculty tend to advise teams against aiming for “buy-in” because it can imply a kind of blind adoption or selling of others’ ideas (usually the administration’s) without fully engaging stakeholders (usually faculty) in owning or even understanding those ideas. Additionally, the term “stakeholder” also increased by 97 percent from applications to action plans, a reflection perhaps of the institute’s response to the growing interest among teams to engage colleagues from across campus, particularly from student affairs, in general education reform. The following quotes from action plans exemplify how campus teams describe their intent to engage a spectrum of stakeholders through a combination of intentional communication strategies and professional development:
Develop a communication strategy to engage and inspire relevant stakeholders, including faculty, students, administrators, institutional support areas (Student Affairs, Student Advising, Institutional Research, and Information Technologies).
To encourage others to join the conversation we need the support of the coordinators and course managers. To help get there we will use the faculty who are currently doing the work, the assessment fellows, and the general education committee. Students are the stakeholders and we need to be inclusive of their voices, including as consultants for the General Education Committee. We do have the necessary resources to accomplish our work.
—Hostos Community College
Stakeholders: Students, provost, deans, associate deans, faculty, advisors, recruiters, librarians, registrar, Chancellor's cabinet
—University of Wisconsin–Milwaukee
Long Term: Explore and develop support resources such as a mentor program, faculty and staff retreats, faculty development workshops, conference attendance funding, and additional funding (stipends, etc.) to support campus constituencies in engaging in general education assessment.
—Adams State University
We suggest more faculty development workshops and “hands-on” lab time to alleviate technology-related fears and anxiety
—Clark Atlanta University
Engage faculty leaders in teaching development experiences.
—University of Oregon
Establishment of a certificate program for general education advisors. How to use learning goals to drive assessment? How to use an ePortfolio during advising?
—College of Saint Benedict and Saint John’s University
Making a Move
It is of little surprise that action plans contain a good deal more practical planning than the applications. The word “timeline” increased by 223 percent from applications to action plans. However, it is the specificity of those timelines that is noteworthy. Higher education is often pilloried for its “glacial pace,” a stereotype that does not bode well for teams undertaking general education reform. Institute teams must not only refine or adjust their thinking to new paradigms for organizing curricula, but they must also shake the persistent notion that anything they undertake will take years to accomplish. In fact, the timelines presented in action plans reflect an earnest desire for change to happen sooner rather than later.
We've developed an aggressive but realistic timeline for our reform agenda. We are not under the impression that these changes will not be met with some skepticism or even resistance; however, we feel confident that we will be able to effect enough important changes to our general education program to make a difference in students’ lives and to engage faculty with a renewed excitement in teaching and learning.
—The University of New Orleans
We have identified a number of actions to be undertaken when we return to campus, and we’ve organized those actions into three major areas: process/assessment, communication, and faculty development. Each action has been placed on a timeline for that area which includes immediate, ongoing, and future action. For example, actions labeled “Now” are items we can initiate as soon as possible post-institute; actions labeled “Ongoing” are items we can start developing and will continue to develop over a longer period; and actions labeled “Long Term” are items we may not start right away but will become priorities in the future. . . . We have identified a number of actions to be undertaken when we return to campus, and we’ve organized those actions into three major areas: process/assessment, communication, and faculty development.”
—Adams State University
We have developed a timeline of tasks that aligns with our program review process. In 2018–19, we will do a pilot, revising the seminar learning outcomes and taking the social, cultural, and historical revision proposal through the Senate process for approval to test the revision process, communication with the campus, and implementation of changes. Then in 2019–20, review the [general education] learning outcomes. In 2020–21 we implement changes and return to standard new course designations and renewals.
—Saint Mary’s College of California
The first year of our timeline will be comprised of a self-study in which we will conduct surveys of faculty (and instructional academic staff) and alumni and focus groups of faculty/staff, students, and alumni. . . . The second year of our timeline will be a “sandbox year” of design and faculty development. We will present the results of the self-study to the campus community and brainstorm as a community possible models and revisions. . . . At the end of the second year, we expect there to be a vote of some sort on moving forward with the revisions. Following that vote, our expectation is that the third year of the process will involve piloting of some possible revisions. . . . Finally, our immediate action plan for the rest of the summer is to revise (and formalize) this timeline and present it to several stakeholders: administrators (including our new Director of Assessment) and our General Education Committee.
—University of Wisconsin–Platteville
Our immediate action steps upon return to campus include a multiphase, multimedia communication plan; development of a set of faculty surveys based on proposals received by our June 15 deadline; and finalization of the format and advance preparation for the Fall Faculty Workshop.
Campus teams participating in the institute are expected to complete a brief six-month follow-up survey. Because six months have not yet passed, this follow-up has not been completed for the 2018 institute cohort. However, a supplemental analysis of postinstitute reports from previous years indicates teams make significant advancements even within just six months. This is perhaps the most illustrative quality of the five case studies contained in this issue of Peer Review. Each campus that has attended the institute within the last three years, and even a campus just a year out (Longwood University), has made significant strides. This does not mean every action a campus takes is successful or that every proposal on learning outcomes or a new general education model will pass a faculty vote. But as Franklin D. Roosevelt said in his May 1932 address at Oglethorpe University, “It is common sense to take a method and try it: If it fails, admit it frankly and try another. But above all, try something.”
Language is powerful, but it is not absolute. The qualitative findings presented above are intended to provide a snapshot of how language adjusts over time as campus teams embark upon and further general education reform efforts. It is expected that applications are often the first time campus leaders have reflected upon particular processes of what will ultimately become a new general education curriculum. Action plans are not the intended end point. These documents are only an initial indicator of the potential changes to come. Actually, action plans are not just likely to be altered, they are expected to be. Perhaps our biggest hope is that teams that attend the institute remain nimble and resilient as they move forward. More than anything, the space between applications and action plans shows that the path of reform is not linear. Plans change. Paradigms shift. Timelines get adjusted. The institute itself provides perhaps the two most critical ingredients of any reform process—collaboration and time. These are the words that underlie much of the preceding analysis and much of the potential for success when teams return home.
Ashley Finley, Senior Advisor to the President and Secretary to the Board, and Erin Horan, Postdoctoral Research Analyst—both of AAC&U