By Doug Ward

A recent meeting at the National Academies of Sciences, Engineering and Medicine achieved little consensus on how best to evaluate teaching, but it certainly showed a widespread desire for a fairer system that better reflects the many components of excellent teaching.

The National Academies co-sponsored the meeting earlier this month in Washington with the Association of American Universities and TEval, a project associated with the Center for Teaching Excellence at KU. The meeting brought together leaders from universities around the country to discuss ways to provide a richer evaluation of faculty teaching and, ultimately, expand the use of practices that have been shown to improve student learning.

A CTE rubric for evaluating teaching helps instructors and departments focus on a series of questions.

My colleague Andrea Greenhoot, professor of psychology and director of CTE, represented KU at the meeting. Members of the TEval team from the University of Massachusetts, Amherst, the University of Colorado, Boulder, and Michigan State University also attended. The TEval project involves more than 60 faculty members at KU, CU and UMass. It received a five-year, $2.8 million grant from the National Science Foundation last year to explore ways to create a fairer, more nuanced approach to evaluating teaching.

The TEval project, which is known as Benchmarks at KU, has helped put KU at the forefront of the discussion about evaluating teaching and adopting more effective pedagogical strategies. Nine departments have been working to adapt a rubric developed at CTE, identify appropriate forms of evidence, and rethink the way they evaluate teaching. Similar conversations are taking place among faculty at CU and UMass. One goal of the project is to provide a framework that other universities can follow.

Universities have long relied on student surveys as the primary – and often sole – means of evaluating teaching. Those surveys can gather important feedback from students, but they provide only one perspective on a complex process that students know little about. The results of the surveys have also come under increasing scrutiny for biases against some instructors and types of classes.

Challenges and questions

The process of creating a better system still faces many challenges, as speakers at the meeting in Washington made clear. Emily Miller, associate vice president for policy at the AAU, said that many universities were having a difficult time integrating a new approach to evaluating teaching into a rewards system that favors research and that often counts teaching-associated work as service.

“We need to think about how we recognize the value of teaching,” Miller said.

She also summarized questions that had arisen during discussions at the meeting:

  • What is good teaching?
  • What elements of teaching do we want to evaluate?
  • Do we want a process that helps instructors improve or one that simply evaluates them annually?
  • What are the useful and appropriate measures?
  • What does it mean to talk about parallels between teaching and research?
  • How can we situate the conversation about the evaluation of teaching in the larger context of institutional change and university missions?

Noah Finkelstein, a University of Colorado physics professor who is a principal investigator on the TEval grant, brought up additional questions:

  • How do we frame teaching excellence within the context of diversity, equity and inclusion?
  • How can we create stronger communities around teaching?
  • How do we balance institutional and individual needs?
  • How do we reward institutions who improve teaching?
  • When will AAU membership be contingent on teaching excellence?

Moving the process forward

Instructors at KU, CU and UMass are already grappling with many of the questions that Miller and Finkelstein raised.

At KU, a group will meet on Friday to talk about the work they have done in such areas as identifying the elements of good teaching; gathering evidence in support of high-quality teaching practices; developing new approaches to peer evaluation for faculty and graduate teaching assistants; providing guidance on instructor reflection and assessment; and making the evaluation process more inclusive. There have also been discussions among administrators and Faculty Senate on ways to integrate a new approach into the KU rewards structure. Considerable work remains, but a shift has been set in motion.

KU faculty and staff share insights on teaching

Several KU faculty members have recently published articles about their inquiry into teaching. Their articles are well worth the time to read. Among them:

Briefly …

  • Writing in EdSurge, Bryan Alexander says that “video is now covering a lot of ground, from faculty-generated instructional content to student-generated works, videoconferencing and the possibility of automated videobots.” The headline goes beyond anything in the article, but it nonetheless raises an interesting thought: “Video assignments are the new term paper.”
  • The Society for Human Resource Management writes about a trend it calls “microinternships,” which mirror the work of freelancers. Microinternships involve projects of 5 to 20 hours that the educational technology company Parker Dewey posts on a website. Students bid on the work, and Parker Dewey takes a percentage of the compensation. The company says it is working with 150 colleges and universities on the microinternship project.
  • Writing in The Chronicle of Higher Education, Aaron Hanlan argues that by relying on a growing number of contingent, “disposable” instructors, “institutions of higher education today operate as if they have no future.” In following this approach, tenured faculty and administrators “are guaranteeing the obsolescence of their own institutions and the eventual erasure of their own careers and legacies,” he argues.
  • EAB writes about the importance of reaching out to students personally, saying that email with a personal, supportive tone can be like a lifeline to struggling students.

  • Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

By Doug Ward

CHARLOTTE, N.C. – Faculty members seem ready for a more substantive approach to evaluating teaching, but …

It’s that “but” that about 30 faculty members from four research universities focused on at a mini-conference here this week. All are part of a project called TEval, which is working to develop a richer model of teaching evaluation by helping departments change their teaching culture. The project, funded by a $2.8 million National Science Foundation grant, involves faculty members from KU, Colorado, Massachusetts, and Michigan State.

Rob Ward, Tracey LaPierre and Chris Fischer discuss strategies during a meeting of TEval, an NSF-grant-funded project for changing the way teaching is evaluated. They joined colleagues from three other universities for meetings this week in Charlotte, N.C.

The evaluation of teaching has long centered on student surveys, which are fraught with biases and emphasize the performance aspects of teaching over student learning. Their ease of administration and ability to produce a number that can be compared to a department average have made them popular with university administrators and instructors alike. Those numbers certainly offer a tidy package that is delivered semester to semester with little or no time required of the instructor. And though the student voice needs to be a part of the evaluation process, only 50 to 60 percent of KU students complete the surveys. More importantly, the surveys fail to capture the intellectual work and complexity involved in high-quality teaching, something that more and more universities have begun to recognize

The TEval project is working with partner departments to revamp that entrenched process. Doing so, though, requires additional time, work and thought. It requires instructors to document the important elements of their teaching – elements that have often been taken for granted — to reflect on that work in meaningful ways, and to produce a plan for improvement. It requires evaluation committees to invest time in learning about instructors, courses and curricula, and to work through portfolios rather than reducing teaching to a single number and a single class visit, a process that tends to clump everyone together into a meaningless above-average heap.

That’s the where the “but …” comes into play. Teaching has long been a second-class citizen in the rewards system of research universities, leading many instructors and administrators to chafe at the idea of spending more time documenting and evaluating teaching. As with so many aspects of university life, though, real change can come about only if we are willing to put in the time and effort to make it happen.

None of this is easy. At all the campuses involved in the TEval project, though, instructors and department leaders have agreed to make the time. The goal is to refine the evaluation process, share trials and experiences, create a palette of best practices, and find pathways that others can follow.

At the meeting here in Charlotte, participants talked about the many challenges that lie ahead:

  • University policies that fail to reward teaching, innovation, or efforts to change culture.
  • An evaluation system based on volume: number of students taught, numbers on student surveys, number of teaching awards.
  • Recalcitrant faculty who resist changing a system that has long rewarded selfishness and who show no interest in reframing teaching as a shared endeavor.
  • Administrators who refuse to give faculty the time they need to engage in a more effective evaluation system.
  • Tension between treating evaluations as formative (a means of improving teaching) and evaluative (a means of determining merit raises and promotions).
  • Agreeing on what constitutes evidence of high-quality teaching.

Finding ways to move forward

By the end of the meeting, though, a hopeful spirit seemed to emerge as cross-campus conversations led to ideas for moving the process forward:

  • Tapping into the desire that most faculty have for seeing their students succeed.
  • Working with small groups to build momentum in many departments.
  • Creating a flexible system that can apply to many circumstances and can accommodate many types of evidence. This is especially important amid rapidly changing demands on and expectations for colleges and universities.
  • Helping faculty members demonstrate the success of evidence-based practices even when students resist.
  • Allowing truly innovative and highly effective instructors to stand out and allowing departments to focus on the types of skills they need instructors to have in different types of classes.
  • Allowing instructors, departments and universities to tell a richer, more compelling story about the value of teaching and learning.

Those involved were realistic, though. They recognized that they have much work ahead as they make small changes they hope will lead to more significant cultural changes. They recognized the value of a network of colleagues willing to share ideas, to offer support and resources, and to share the burden of a daunting task. And they recognized that they are on the forefront of a long-needed revolution in the way teaching is evaluated and valued at research universities.

If we truly value good teaching, it must be rewarded in the same way that research is rewarded. That would go a long way toward the project’s ultimate goal: a university system in which innovative instructors create rich environments where all their students can learn. It’s a goal well worth fighting for, even if the most prevalent response is “but …”

A note about the project

At KU, the project for creating a richer system for evaluating teaching is known as Benchmarks for Teaching Effectiveness. Nine departments are now involved in the project: African and African-American Studies; Biology; Chemical and Petroleum Engineering; French, Francophone and Italian; Linguistics; Philosophy; Physics; Public Affairs and Administration; and Sociology. Representatives from those departments who attended the Charlotte meeting were Chris Fischer, Bruce Hayes, Tracey LaPierre, Ward Lyles, and Rob Ward. The leaders of the KU project, Andrea Greenhoot, Meagan Patterson and Doug Ward, also attended.

Briefly …

Tom Deans, an English professor at the University of Connecticut, challenges faculty to reduce the length of their syllabuses, saying that “the typical syllabus has now become a too-long list of policies, learning outcomes, grading formulas, defensive maneuvers, recommendations, cautions, and referrals.” He says a syllabus should be no more than two pages. … British universities are receiving record numbers of applications from students from China and Hong Kong, The Guardian reports. In the U.S., applications from Chinese students have held steady, but fewer international students are applying to U.S. universities, the Council of Graduate Studies reports. … As the popularity of computer science has grown, students at many universities are having trouble getting the classes they need, The New York Times reports.


Doug Ward is the acting director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

By Doug Ward

The spread of evidence-based teaching practices highlights a growing paradox: Even as instructors work to evaluate student learning in creative, multidimensional ways, they themselves are generally judged only through student evaluations.

Students should have a voice. As Stephen Benton and William Cashin write in a broad review of research, student evaluations can help faculty members improve their courses and help administrators spot potential problems in the classroom.optical illusion box

The drawback is that too many departments use only student evaluations to judge the effectiveness of instructors, even as they submit faculty research through a multilayered evaluation process internally and externally. Student evaluations are the only university-mandated form of gauging instructors’ teaching, and many departments measure faculty members against a department mean. Those above the mean are generally viewed favorably and those below the mean are seen as a problem. That approach fails to account for the weaknesses in evaluations. For instance, Benton and Cashin and others have found:

  • Students tend to give higher scores to instructors in classes they are motivated to take, and in which they do well.
  • Instructors who teach large courses and entry-level courses tend to receive lower evaluations than those who teach smaller numbers of students and upper-level courses.
  • Evaluation scores tend to be higher in some disciplines (especially humanities) than in others (like STEM).
  • Evaluation scores sometimes drop in the first few semesters of a course redesigned for active learning.
  • Students have little experience in judging their own learning. As the Stanford professor Carl Wieman writes: “It is impossible for a student (or anyone else) to judge the effectiveness of an instructional practice except by comparing it with others that they have already experienced.”
  • Overemphasis on student evaluations often generates cynicism among faculty members about administrators’ belief in the importance of high-quality teaching.

Looked at through that lens, we have not only a need but an obligation to move beyond student evaluations in gauging the effectiveness of teaching. We simply must add dimension and nuance to the process, much as we already do with evaluation of research.

So how do we do that?

At CTE, we have developed a rubric to help departments integrate information from faculty members, peers, and students. Student evaluations are a part of the mix, but only a part. Rather, we have tried to help departments draw on the many facets of teaching into a format that provides a richer, fairer evaluation of instructor effectiveness without adding onerous time burdens to evaluators.

For the most part, this approach uses the types of materials that faculty members already submit and that departments gather independently: syllabi and course schedules; teaching statements; readings, worksheets and other course materials; assignments, projects, test results and other evidence of student learning; faculty reflections on student learning; peer evaluations from team teaching and class visits; and formal discussions about the faculty member’s approach to teaching.

Departments then use the rubric to evaluate that body of work, rewarding faculty members who engage in such approaches as:

  • experimenting with innovative teaching techniques
  • aligning course content with learning goals
  • making effective use of class time
  • using research-based teaching practices
  • engaging students in hands-on learning rather than simply delivering information to them
  • revising course content and design based on evidence and reflection
  • mentoring students, and providing evidence of student learning
  • sharing their work through presentations, scholarship, committee work and other venues

Departments can easily adapt the rubric to fit particular disciplinary expectations and to weight areas most meaningful to their discipline. We have already received feedback from many faculty members around the university. We’ve also asked a few departments to test the rubric as they evaluate faculty members for promotion and tenure, third-year review, and post-tenure review, and we plan to test it more broadly in the fall.

We will continue to refine the rubric based on the feedback we receive. Like teaching itself, it will be a constant work in progress. We see it as an important step toward making innovative teaching more visible, though, and toward making teaching a more credible and meaningful part of the promotion and tenure process. If you’d like to be part of that, let us know.

****

This article also appears in Teaching Matters, a publication of the Center for Teaching Excellence.


Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

CTE’s Twitter feed