Membership Programs Meetings Publications LEAP Press Room About AAC&U
Association of American Colleges and Universities
Search Web Site
AAC&U
Resources on:
Liberal Education
General Education
Curriculum
Faculty
Student Success
Institutional Change
Assessment
Diversity
Civic Engagement
Women
Global Learning
Science & Health
PKAL
Connect with AACU:
Join Our Email List
RSS Feed
Facebook
Follow us on Twitter
LEAP Blog
LEAP Toolkit
YouTube
Podcasts
Support AACU
Online Giving Form
 
Liberal Education Fall 2012 cover

 

 

Fall 2012, Vol. 98, No. 4

"Tuning" the Disciplines


By Norman Jones


Since March of 2009, the Utah System of Higher Education has been a partner with the Lumina Foundation for Education in the Tuning USA project, Lumina’s first experiment in introducing the European concept of degree “tuning” to American academia. Developed in the European Union as a way to create common degree standards across multiple nations, “tuning” is a methodology whereby subject-area teams develop criterion-referenced learning outcomes and competencies for particular degrees. It is a faculty-led approach that involves seeking input from students, recent graduates, and employers in order to create a common understanding of what students should know, understand, and be able to do when they finish each level of a disciplinary degree program.

Tuning arrived in Utah at an opportune time for us. We had already been engaged in system-wide faculty dialogues that we call “majors’ meetings” and in an annual conference, known for the past fourteen years as “What is an Educated Person?,” that concentrates on general education outcomes and assessment. Moreover, having worked closely with the Association of American Colleges and Universities through its Liberal Education and America’s Promise (LEAP) initiative, Utah was becoming a “LEAP State.” All of Utah’s colleges had adopted the LEAP Essential Learning Outcomes, and the Utah System of Higher Education had been actively promoting access and authentic assessment. Consequently, when Lumina invited us to participate in a faculty-driven process of defining degree outcomes, it made perfect sense to us. We already had developed many of the mechanisms needed for tuning, and the general education task force was an established voice on all our campuses—so we knew who to call to get things done. We were ready to work toward tuned degree profiles.

At the core of successful degree tuning are three considerations that make every experience unique. I think of them as visions, trust, and mechanics. If these three are not attended to, you cannot successfully tune. And although tuning may go a long way in one or two areas, unless the three are brought into tune with one another, it will not be possible to complete the process.

Visions

What do I mean by “visions”? I mean the metaphysics of the process—the “why” of tuning. For thirty years, I have been engaged in conversations about higher education reform and assessment, and I have seen most attempts fail because they impose a “why” that means nothing to the faculty in the classroom. They do not invite disciplinary faculty into a conversation that allows them to voice their own values. Faculty in all disciplines have professional values, and they have ways of teaching and assessing those values that they can articulate. But too often, externally imposed reforms and assessments fail to link to the “why” of faculty efforts. This is why tuning has such promise. It takes seriously the professional values of disciplines and disciplinary communities.

By professional values of disciplines, I mean those parts of professional self-understanding created by indoctrination (used in its classical sense) into the academic field. Adherence to these values marks one as a member of the discipline and guides one’s practices. After years of doctoral training, one literally become a disciple, one who follows the discipline. For instance, as historians, my colleagues and I do not believe in tests that do not include written analyses of primary documents. We are fierce in our commitment to careful documentation, despising presentists, antiquarians, politicians, and movie makers who distort and abuse historical evidence. We are generally suspicious of theories, and we do not trust a priori hypothesizing. We believe in the power of jargon-free writing. In short, we know what we value, and we have a centuries-old tradition of evaluation. Any system of assessment that ignores our vision of our professional responsibility will fail because we will dismiss it as untrustworthy and as an enemy of our professional mission.

Tuning asks disciplines to identify their larger communities and find a common language for the outcomes of the discipline. These communities are made up, most obviously, of the people inducted into the discipline by taking a degree in it. We call them alumni. A properly indoctrinated major is a credit to the department that shaped her and an asset to her employer. The vision of her employers is also formed by, and helps form, disciplinary values. The external disciplinary community includes the workplaces in which people prepared in particular disciplines find work, since those employers often share the vision of the discipline held by the faculty and graduates. They seek to hire graduates because of their disciplinary preparation.

Envisioning these disciplinary communities is important, and there is a great danger in too simplistic a notion. Some disciplines have presumed obvious employers. Whole disciplines have been named after employers—accounting trains accountants to be hired by accounting firms—or so the simplistic logic runs. But even in these, the alumni upset the calculation with their behavior. And in many disciplines the uses of the alumni skill set create a very broad, even generic, community of employers. History represents one of these disciplines, with so many employers that imagination is required to identify the disciplinary communities. Of course, there are obvious employers—schools, libraries, museums, publishers, and others—but there are many less obvious ones, such as those that need researchers, analysts, and writers but do not have jobs with “history” in the title. And then there are the others: graduate schools, law schools, medical schools, business schools, and, importantly, the self-employed. Nor should we forget that there are public consumers of history who depend on historians to tell them the truth about the past in all sorts of venues for all sorts of public purposes. This is one of the important lessons of tuning. There are shared visions and values that transcend academic boxes and can inform the way we think about disciplinary degrees.

The tuning process creates the opportunity to identify the members of these disciplinary communities and to express the vision inherent in degree preparation on both the philosophical and the practical levels, expressing these as outcomes that can be measured according to the values of the discipline in ways that are acceptable within the disciplinary community. For those disciplines that do not have obvious constituencies, it is more difficult to determine the values of the disciplinary community. Early attempts at using surveys to learn about employers’ desires and understandings taught us that surveys are very poor instruments. It is much more effective to convene professionally mediated focus groups that can explore the usefulness of the outcomes in settings that allow probing discussions. Focus groups are not cheap, but they are very effective—even to the point of being cost-effective.

If we listen to the community of the discipline, the outcomes of a degree quickly become visible. If the articulated outcomes have these people nodding their heads in agreement, you know that you have captured the visions of a group that wants what the discipline teaches.

Trust

We learn the vision by listening to the faculty and the disciplinary constituencies, but that only generates warm feelings amongst the people who have reconfirmed their self-image. If tuning is to begin and succeed, we have to pay attention to the trust necessary to get it off the ground. What do I mean by “trust”?

I mean that the discussion of outcomes, their formulation, and their enactment must be valued and supported by academic leadership. The disciplinary faculty must trust that their leaders value and will support the effort tuning requires. The faculty must be heard, and their vision must be allowed, even if it seems contrary to institutional habits. In tuning, we have to understand the politics of the process, identifying the people and organizations that have the power to convince the faculty that they will not be harmed if they articulate their values and base their assessments upon them.

Think of the arguments faculty members are likely to make when asked to undertake any assessment process. Experienced academic leaders can name several instantly, many turning around the fear that efforts to identify the outcomes of a degree and assess its success are tools of the evil upper administration/politicians/pundits/lazy students to punish the faculty and increase its work load. If we make obvious what we do, and if we discover we don’t always succeed in doing it, faculty members say, terrible things are likely to happen. The department may be denied funds or have faculty lines taken away, or teaching loads may be increased. Worst of all, external actors who do not understand or value the discipline may interfere with the faculty’s duty to its profession. After all, it is not just laziness that prompts historians to resist lecture sections enrolling hundreds. It is because their professional values tell them that they must make students write, and you cannot require and assess writing in classes that large. If you ask faculty to betray their profession, they demur.

Moreover, most assessment systems do not care about content, yet content is at the heart of professional preparation. Institutions are even more uninterested in content. The American system of higher education runs on credit hours, facilely accepting the credit unit as the measure of achievement. We all can do the sums. Accumulate 120 hours and you should have a degree. If students don’t have a degree in 120 hours, the institution has failed to be efficient. Sixty hours should provide you with an associate’s degree; general education can be met in thirty, or forty, or fifty hours, but should be “completed” so it can transfer whole. A major is much the same, a checklist of courses that add up to a specified number of credits. But if you ask whether the outcomes of the degree are met, it undermines this beautiful, Henry Ford–inspired system. Mr. Ford knew you could use an assembly line to speed up production, with each worker performing a particular task and then passing it to the next worker. Carefully planned, the production line supplied parts when needed, and they were assembled with a clear outcome. At the end of the line, the car was driven away.

We in academia have pretended for the past century that degrees are like Fords. We accept that once a particular part—call it Math 1050 or English 2010—is installed, it will always work well when the next part is installed, even after years of rust. A conversation about degree outcomes has the potential to highlight the untenable nature of this assumption. What if the disciplinary outcomes make changes to the curriculum necessary? A discipline that announces that its outcomes require more hours, a tighter curriculum, increased time to graduation, decreased accessibility, and even more students rejected for graduation, justly fears retribution. Faculty members recognize that too much honesty can endanger their departments, and that the administrations of their institutions are likely to reject their professional values if they propose major changes. They have to have permission, trusted permission, from their academic managers if tuning is to be taken seriously.

But even if they trust their dean to understand their deep commitment to professional outcomes, an individual department can seldom do it alone. There are other academic masters larger than a single institution that must be supportive of the effort. Tuning requires a conversation about outcomes that includes many institutions. In Utah, we focused on two disciplines, history and physics, across the state’s system of higher education, including all history and physics programs—from the research universities with large specialized faculties, to community colleges with one or two people in the discipline. This is essential because it gives a discipline the right to think as a discipline, rather than as a department subject to the local pressures of particular institutions. After all, differences in mission, enrollment demographics, and attitudes can undermine the right of the disciplinary faculty to enact their professional values. When multiple institutions stand together, their degrees articulate better, and their professional outcomes are easier to impose. But there is the danger that the local administration will reject the implementation of professional outcomes if they cost money, or if they introduce complications in the larger curriculum. That is why systems have to stand together; no individual department is exposed to the charge of having capriciously high expectations.

However, this protection from local resistance can still fail if the larger structures do not grant permission to impose the outcomes. The system has to express its support for the effort. The least it can do is to express its expectation that the outcomes developed in tuning will be used in the system.

A discipline standing together and blessed by the system in which it is embedded is powerful, but professional organizations, professional accreditors, and regional accreditors must be recognized as having authority over the future of the outcomes. From the beginning of the tuning process, disciplines need to look at the values of their own professional organizations. There is seldom a difference in their visions, but their ways of measuring outcomes may make tuning difficult. Regional accreditors present a different sort of problem, since their assessment requirements do not necessarily make the use of degree outcomes easy.

Of course, the ultimate expression of trust is reward. To prove that the effort is valued, investment can send a strong message. Tangible rewards can create trust.

There are other benefits of tuning, too. We have learned that one of the largest payoffs can be in faculty morale. A discipline that tunes enjoys an increased sense of identity and purpose. It may also find it easier to wrestle with the monsters of assessment—but only if the mechanics of academic change are understood and incorporated into the assessment process.

Mechanics

Even if disciplines and administrations are willing to tune, there still remains another consideration. The mechanics of the process of curricular development are local and systemic. By “mechanics,” I mean the processes of implementation within institutions and systems. Before a discipline begins to tune, it has to think about what it can achieve within its own institutional structures. In particular, it and its upper administration must be very aware of how a discipline nestles within the larger curriculum. Tuning a discipline has implications across the curriculum.

Built into tuning is the concept of levels, benchmarks of performance at various points in the discipline. Obviously, one level is the bachelor’s degree. Another is the master’s degree, and a third is the doctorate. Tuning expects disciplines to indicate what accomplishments accompany each level. In the American system, however, there is yet another level, the point when students move from the general curriculum into the upper division of a discipline. This is sometimes indicated by the awarding of an associate’s degree, but it always marks the beginning of upper-division work in the major, the point at which a student is truly the responsibility of a disciplinary team in a bachelor’s degree–granting institution. To tune successfully, it is crucial that the discipline knows what students entering the major should know, understand, and be able to do.

In the American world of transferring students, online courses, and concurrent enrollment in high school and college, the major must say, “When you enter this major, you must have accomplished the following.” If it does that, it knows where it starts in preparing students for the next level. If it does not, it sends the message that no preparation is necessary to be an advanced student. But when a discipline builds this starting point into its curriculum, it is also saying to the rest of the university, “This is what makes our major what it is.” In the process, it should be telling the student that there are courses that should be taken outside the discipline in preparation for success in the major.

For instance, the history department at Utah State University created a pre-major as a result of tuning. A student who undertakes the pre-major in history is told that certain lower-division history courses must be taken, that statistics are preferred to pre-calculus algebra, that philosophy and anthropology are subjects with which a history major must be familiar, and that two years of a single language are part of the preparatory education for the major. The history major is making use of courses taught by other departments, shaping the general education of its majors in ways that prepare students for successful achievement of history’s outcomes. An important partner with the history department is the university library. The information literacy specialists there have become part of the larger team working with our students. The history outcomes place heavy emphasis on research and the evaluation of evidence, and we have found that librarians working with particular courses can greatly improve student mastery of research skills.

A tuned discipline also regards its students as partners. It makes its outcomes clear to the students and invites them to track their own progress toward their degree outcomes. At the same time, it gives them a vocabulary for talking about how and why they are expected to know, understand, and be able to do things associated with the major. Important allies in developing intentional student behavior in degree programs are the academic advising and career advising staffs. Tuning gives them a clearer understanding of the discipline, and they can then help the major deliver a unified message to the students. We owe it to students not only to expand their knowledge, thinking, and skills. We must also expand their very vocabulary. What words do students use to describe their own experience in a major, a college, a university? How do they answer the question that comes to them from skeptical parents and even more cynical employers: What did you learn? What practical use does your course work have? What are the skills associated with your degree?

A tuned discipline with clear outcomes, rubrics for measurement, reformed delivery, effective advising, and intentional students can solve many of the mechanical problems. But there are implications for others beyond the particular discipline. Curricula do not exist in vacuums. If four hundred history majors are suddenly clamoring for the same philosophy course, the philosophy department may have a staffing problem and history may have a time-to-degree crisis. Worse, since disciplines are tuned across systems, changed expectations in one discipline may make difficulties for other disciplines. For example, in Utah the physicists decided to add a lab to a course required by the engineers. The physicists were happy, but the engineers were upset because physics had added two credits to their students’ curricula without consultation. That lab might lengthen the time-to-graduation for engineers.

Paying attention to the mechanics of curricular relationships can be extremely beneficial, since as disciplines tune they more or less drag their colleges with them. Making more effective use of the larger curriculum creates greater efficiency, and ties general education more tightly to the major. It also opens up conversations with sister institutions and feeder schools, making transfer and articulation easier.

In the best cases, tuning can be applied to whole colleges. At Utah State, the College of Humanities and Social Sciences is creating a faculty-driven core so that students tracking toward the humanities or social sciences have a much tighter common curriculum. This curriculum is being designed, using tuning methods, to ensure that students in those fields have common literacy and skills, functioning as a springboard into the majors. The majors, of course, get the benefit of knowing what their students should know, understand, and be able to do and can plan the curriculum accordingly. The core delivers a student to the major with appropriate preparation so the major can take him or her efficiently to bachelor’s degree level.

Levels for entry into the major raise another question: the level for college entry. What must a student know, understand, and be able to do in order to succeed in college? This question is usually answered by a vague reference to a standardized test score and a minimum grade point average, but tuning confronts the faculty with the question of whether they are teaching at the level of student preparation. Or, to put it another way, are they at the students’ “level”? Asking this question opens a Pandora’s box of issues about K-12 preparation and the design of freshman courses, but it also allows colleges to explain to high schools what successful preparation looks like and to invite change. The mechanics of change require the disciplinary faculty to identify their “level constituencies” and to at least adapt to the limitations those impose. At best, tuning allows a fruitful negotiation that improves student preparation.

As American K-12 systems adopt the new core standards approach, it is important for colleges to articulate their understanding of levels and to work with the K-12 people to maximize the benefits of new curricula for students. One of the important unintended consequences of tuning is the value of the process to general education. If disciplines can articulate their needs, they can articulate the relationship of the discipline to the broader curriculum. In tuning the bachelor’s degree in history at Utah State, for example, it was possible to decrease the number of upper-division courses required when it was realized that better preparation for entry to the program meant that some things did not have to be taught by the department.

However, institutional choices about general education can also severely limit the attempt to impose a beginning level on a degree. If, for instance, the general education curriculum permits a very limited number of choices, the tuned discipline may have to add courses that in other cases would have done double duty in general education and the discipline. If philosophy is not a general education choice and history thinks it necessary, history students will need to take the accepted humanities course for the institution as well as the philosophy course for the history degree. Consequently, the tuned discipline may discover that it needs to increase the preparation provided in the major if it cannot find help in the broader curriculum.

Once all of this negotiating has been done, there are still three mechanical issues facing a tuned discipline: implementation of the outcomes, measurement of their achievement and impact, and faculty preparation. Implementation is a completely local issue for most disciplines, so suffice it to say that tuning must be aided by people who understand how curricular change is carried out bureaucratically. It also must have political support from the upper administration. It may take no time at all, or it may take a year or two before the tuned curriculum is in place.

Many disciplines have outcomes on paper. The difference between the stated outcomes and real effectiveness in disciplines is the way in which outcomes are measured and the conversations the measurements spark. There is no simple way to assess outcomes in tuned disciplines. In our experience, the most effective method is the use of rubrics, containing criteria and standards for measuring individual performance, tied to key moments in the degree program. Utah State’s history department has a rubric for the capstone course taken by students completing the bachelor’s degree. That rubric has also become a part of initiation into the major, appearing first in the entry course to the major. Upper-division courses frequently refer to it, as a way to tell the students how a particular course is preparing them for the capstone. Papers written in the capstone course can be compared and rated, to discover whether students are achieving the established outcomes. Of course, it takes several years to gather enough data to be sure that the outcomes are truly being met and to know the real effect of tuning.

In the short term, the more effective assessment is qualitative research based on interviews with faculty and students. From those, we know that both teachers and students have benefited from clear outcomes. Courses are easier to plan (or at least they are planned with new insights as to their goals), and students have a much better sense of the purpose of the degree and the things they are expected to understand, know, and do.

Inherent in the tuning process is the larger community of the discipline, so any assessment of its effectiveness should continue to involve the alumni, employers, and other groups invited into the initial conversation. It should also involve university staff who help prepare students to meet the outcomes, such as librarians who teach information literacy.

The creation of the outcomes and the use of the rubrics are the result of faculty discussions, but the energy produced by that burst of disciplinary enthusiasm has to be sustained. There need to be mechanisms for both renewing the discussion about the outcomes and for socializing newcomers into their use. New faculty must be introduced to the outcomes and the rubrics and be expected to use them—but that expectation has to be linked to more discussion. Only if new faculty members catch the vision behind the outcomes will the discipline maintain the coherence established by them. New courses have to address the outcome goals, and they need to make use of the rubrics that create a coherent system of preparation for students.

This intentionality creates problems for the use of adjuncts. Departments that depend heavily on contingent faculty may find the recognition and adoption of outcomes in the classroom a difficult thing. Most professionals will share the values inherent in the outcomes, but adjuncts must be trained to understand and use the outcomes and rubrics in their teaching if the department’s courses are to work together toward the desired end. This is another reason why it is necessary to tune by systems, rather than individual institutions. Then adjuncts can teach the same course in several places and know that there are common goals—if they are taught to understand and use them!

Certainly, because clear expectations of outcomes force educators to think about degrees more coherently, tuning challenges some managerial practices common in American colleges. Like Henry Ford, degree-granting disciplines that have tuned know what the end product should look like when it rolls off the assembly line. But knowing that and making it work require a great deal of discussion by the disciplinary faculties, strong support from administrators, and deep awareness of institutional contexts. This means that faculties engaged in tuning must be accompanied on their journey by leaders and systems that are open to the change tuning engenders. If they are, exciting change can be achieved.


Norman Jones is professor of history and director of general education and curricular integration at Utah State University.


To respond to this article, e-mail liberaled@aacu.org, with the author’s name on the subject line.



spacer