By Doug Ward

Two vastly different views of assessment whipsawed many of us over the past few days.

The first, a positive and hopeful view, pulsed through a half-day of sessions at KU’s annual Student Learning Symposium on Friday. The message there was that assessment provides an opportunity to understand student learning. Through curiosity and discovery, it yields valuable information and helps improve classes and curricula.

The second view came in the form of what a colleague accurately described as a “screed” in The New York Times. It argued that assessment turns hapless faculty members into tools of administrators and accreditors who seek vapid data on meaningless “learning outcomes” to justify an educational business model.

As I said, it was hard not to feel whipsawed. So let’s look a bit deeper into those two views and try to figure out what’s going on.

Clearly, the term “assessment” has taken on a lot of baggage over the last two decades. Molly Worthen, the North Carolina professor who wrote the Times op-ed article, highlights nearly every piece of that baggage: It is little more than a blunt bureaucratic instrument imposed from outside and upon high. It creates phony data. It lacks nuance. It fails to capture the important aspects of education. It is too expensive. It burdens overtaxed instructors. It generates little useful information. It blames instructors for things they have no control over. It is a political, not an educational, tool. It glosses over institutional problems.

Dawn Shew works on a poster during a session at the Student Learning Symposium. With her are, from left, Ben Wolfe, Steve Werninger and Kim Glover.

“Without thoughtful reconsideration, learning assessment will continue to devour a lot of money for meager results,” Worthen writes. “The movement’s focus on quantifying classroom experience makes it easy to shift blame for student failure wholly onto universities, ignoring deeper socio-economic reasons that cause many students to struggle with college-level work. Worse, when the effort to reduce learning to a list of job-ready skills goes too far, it misses the point of a university education.”

So if assessment is such a burden, why bother? Yes, there are political reasons, but assessment seems a reasonable request. If we profess to educate students, shouldn’t we be able to provide evidence of that? After all, we demand that our students provide evidence to back up arguments. We demand that our colleagues provide evidence in their research. So why should teaching and learning be any different?

I’m not saying that the assessment process is perfect. It certainly takes time and money to gather, analyze and present meaningful evidence, especially at the department, school or university level. At the learning symposium, an instructor pointed out that department-level assessment had essentially become an unfunded mandate, and indeed, if imposed from outside, assessment can seem like an albatross. And yet, it is hardly the evil beast that Worthen imagines.

Yes, in some cases assessment is required, and requirements make academics, who are used to considerable autonomy, chafe. But assessment is something we should do for ourselves, as I’ve written before. Think of it as a compass. Through constant monitoring, it provides valuable information about the direction and effectiveness of our classes and curricula. It allows us to make adjustments large and small that lead to better assignments and better learning for our students. It allows us to create a map of our curricula so that we know where individual classes move students on a journey toward a degree. In short, it helps us keep education relevant and ensures that our degrees mean something.

New data about assessment

That view lacks universal acceptance, but it is gaining ground. Figures released at the learning symposium by Josh Potter, the university’s documenting learning specialist, show that 73 percent of degree programs now report assessment data to the university, up from 59 percent in 2014. More importantly, more than half of those programs have discussed curriculum changes based on the assessment data they have gathered. In other words, those programs learned something important from assessment that encouraged them to take action.

That’s one of the most important aspects of assessment. It’s not just data we send into the ether. It’s data that can lead to valuable discussion and valuable understanding. It’s data that helps us make meaningful revisions.

The data that Potter released pointed to challenges, as well. Less than a third of those involved in program assessment say that their colleagues understand the purpose of assessment, that their department recognizes their work in assessment, or that they see a clear connection between assessment and student learning. Part of the problem, I think, is that many instructors want an easy-to-apply, one-size-fits-all approach. There simply is no single perfect method of assessment, as Potter makes clear in the many conversations he has with faculty members and departments. Another problem is that many people see it as a high-stakes game of gotcha, which it isn’t, or shouldn’t be.

“Assessment isn’t a treasure hunt for deficiencies in your department,” Potter said Friday.

Rather, assessment should start with questions from instructors and should include data that helps instructors see their courses in a broader way. Grades often obscure the nuances of learning and understanding. Assessment can make those nuances clearer. For instance, categories in a rubric add up to a grade for an individual student, but aggregate scores for each of those categories allow us to see where a broad swath of students need work or where we need to improve our instruction, structure assignments better, or revisit topics in a class.

Assessment as a constant process

That’s just one example. Individually, we subconsciously assess our classes day by day and week by week. We look at students’ faces for signs of comprehension. We judge the content of their questions and the sophistication of their arguments. We ask ourselves whether an especially quiet day in class means that students understand course material well or don’t understand at all.

The goal then should be to take the many meaningful observations we make and evidence we gather in our classes and connect them with similar work by our colleagues. By doing that on a department level, we gain a better understanding of curricula. By doing it on a university level, we gain a better understanding of degrees.

I’m not saying that any of this is easy. Someone has to aggregate data from the courses in a curriculum, and someone – actually, many someones – has to analyze that data and share results with colleagues. Universities need to provide the time and resources to make that happen, and they need to reward those who take it on. Assessment can’t live forever as an unfunded mandate. Despite the challenges that assessment brings, though, it needs to be an important part of what we do in higher education. Let me go back to Werther’s op-ed piece, which despite its screed-like tone contained nuggets of sanity. For instance:

“Producing thoughtful, talented graduates is not a matter of focusing on market-ready skills. It’s about giving students an opportunity that most of them will never have again in their lives: the chance for serious exploration of complicated intellectual problems, the gift of time in an institution where curiosity and discovery are the source of meaning.”

I agree wholeheartedly, and I think most of my colleagues would, too. A college education doesn’t happen magically, though. It requires courses to give it shape and curricula to give it meaning. And just as we want our students to embrace curiosity and discovery to guide their journey of intellectual exploration, so must we, their instructors, use curiosity and discovery to guide the constant development and redevelopment of our courses. That isn’t about “quantifying classroom experience,” as Werther argues. It’s about better understanding who we are and where we’re going.


Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

By Doug Ward

Here’s a secret about creating a top-notch assessment plan:

Make sure that it involves cooperation among faculty members, that it integrates assignments into a broader framework of learning, and that it creates avenues for evaluating results and using them to make changes to courses and curricula.

Lorie Vanchena, Nina Vyatkina and Ari Linden of the department of Germanic languages and literatures accepted the Degree-Level Assessment Award from Stuart Day, interim vice provost for academic affairs.

Actually, that’s not really a secret – really, it’s just good assessment practice – but it was the secret to winning a university assessment award this year. Judges for both the Degree-Level Assessment Award and the Christopher H. Haufler KU Core Innovation Award cited the winners’ ability to cooperate, integrate and follow up on their findings as elements that set them apart from other nominees.

The Department of Germanic Languages and Literatures won this year’s degree-level assessment award, and the Department of Curriculum and Teaching won this year’s Haufler award. The awards were announced at last week’s annual Student Learning Symposium. Each comes with $5,000.

The German department focused its plan on two 300-level courses that serve as a gateway to the major, and on its capstone course. Stuart Day, the acting vice provost for academic affairs, said the University Academic Assessment Committee, which oversees the award, found the plan thorough, manageable and meaningful. It is one of the strongest assessment plans in place at the university, he said. It emphasizes substantive learning outcomes, uses a variety of methods for assessment, and includes a plan for making ongoing improvements.

Reva Friedman accepted the Haufler KU Core Innovation Award from DeAngela Burns-Wallace, vice provost for undergraduate studies.

DeAngela Burns-Wallace, vice provost for undergraduate studies, said the plan created by curriculum and teaching had similar characteristics, using a rich approach that integrates active learning, problem solving and critical thinking. The department created a “strong and intentional feedback loop for course improvement,” she said, and created a clear means for sharing results throughout the department.

So there again is that secret that isn’t really a secret: A strong assessment plan needs to include cooperation among colleagues, integration of assignments and pedagogy, and follow-ups that lead to improvements in the curriculum.

That sounds simple, but it’s not. Reva Friedman, associate professor of curriculum and teaching, and Lorie Vanchena, associate professor of Germanic languages and literatures, both spoke about the deep intellectual work that went into crafting their plans. That work involved many discussions among colleagues and some failed attempts that eventually led to strong, substantive plans.

“Everything we’re doing informs everything else we’re doing,” Friedman said.

She also offered a piece of advice that we all need to hear.

“All of us have our little castles with moats around them, and we love what we do,” she said. “But we need to partner in a different way.”

A new resource for teaching media literacy

In a world of “alternative facts,” we all must work harder to help students learn to find reliable information, challenge questionable information, and move beyond their own biases. To help with that, KU Libraries recently added a media literacy resource page to its website. Instructors and students will find a wealth of useful materials, including definitions, evaluation tools, articles and websites.


Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

By Doug Ward

Education changes people.

Those of us who teach know that well. We see students transform during their degrees, and sometimes during a semester. Their skills improve. Their thinking deepens. Their confidence blossoms.

As it changes minds, though, education also changes the relationships students have with family and friends, adding stress to students’ lives from an unexpected source. Students generally learn to cope with those changes, but they often aren’t sure how to broach the subject with family and friends. They don’t want to anger others, or make them feel diminished. But they also don’t want to hold themselves back.

That personal aspect of learning is just one of many issues that students have brought up during focus groups I’ve led on learning, engagement, and student motivation over the past two years. I wrote recently about what students say helps them learn. Students openly admit that they struggle with many aspects of higher education, though. If we are willing to listen, they will tell us what they think we need to know.

What students want faculty to know

Some of students’ best experiences in college come from meaningful discussions with professors who have expertise in a particular field or who simply take time to learn who they are and help them find their way. Students rave about professors who “made me feel comfortable and made time for me.”

celka straughn, phil baringer and andrea greenhoot at the student learning symposium
Celka Straughn, director of academic programs at the Spencer Museum of Art, speaks with Phil Barringer, professor of physics, during a lunch session at the Student Learning Symposium. Andrea Greenhoot, director of CTE (background), led the session.

On the other hand, students say that professors who come across as unapproachable, condescending, and inflexible diminish the value of classes, majors, and degrees. Some of those professors simply won’t make time for students, though students recognize the challenge that large classes bring. Big lectures make students feel lost and anonymous, they say, and professors in those classes often have a hard time connecting to students.

Other obligations. Remember that students are taking more than your class. They also have obligations at work, family and organizations. Also remind students about office hours. Yes, they know about office hours, they say, but a reminder now and then helps.

Pointless classes. Far too many classes seem pointless, students say. They give a variety of reasons for that:

  • Some classes lack any relation to students’ goals or interests, and faculty members fail to explain the point of the class. All too often, the same professors who challenge colleagues to explain the “so what?” of research don’t ask the same question about the courses they teach. “I’m at point where I know what I want to do,” one student said. “I value the time and money I put into a class.”
  • Some courses within a major are too basic, students say. They want to be challenged, and some assignments seem like little more than busy work.
  • Some faculty members come across as pretentious and diminish the subject matter of their classes. They may think their class covers the most important topic in the world, students say, but simply saying that doesn’t make it so.

Professors also need to be clearer about the expectations of classes, students say. Some classes require more independent learning; others require students to follow specific guidelines. Students can work with either system, they say. They just need clear expectations.

What students want administrators to know

College costs too much. This is both a political and institutional issue, especially at a state university. Students understand the politics of state reductions in university financing and the corresponding rise in tuition. They also want a strong, vibrant university. Yet most have jobs to help pay college costs, they say, and some work 20 to 35 hours a week.

Advising is uneven. Most students rave about their advisors, especially the ones in their majors. They have harsh words for the general advising system, though. “For every good comment I’ve heard about general advisors, I’ve heard three bad ones,” one student said. Friends had been told to take classes they didn’t need, delaying graduation. One mentioned a friend who was told at freshman orientation that he had five minutes to figure out his first schedule. Most certainly, this isn’t a typical experience. Stories like these stand out in students’ minds, though, and cast a shadow on the advising system. Advisors and faculty members also need to keep an open mind, students say. Students change their minds about majors and classes as they learn more about themselves and their interests, and just learn more in general.

Smaller classes make a difference. I’ve heard this over and over from students, and pedagogical research backs it up. Those least equipped to handle large lecture classes are beginning students, yet those are the very students required to enroll in those classes. Many instructors have worked to overcome the weakness of large classes with in-class group work and two-stage exams, techniques that help shrink the class and improve student learning. Students at the most recent learning symposium spoke about the many benefits of that. Unfortunately, active learning in large classes is still rare.

Students generally rave about first-year seminars, saying that those types of courses help them make connections and help them acclimate to university learning. Some suggest a broader requirement for an orientation-type course that focuses on how to learn and how to structure your life in college.

Make classes more transparent. Students say they want to be able to make more informed decisions about the classes they take. Some want evaluations of courses and professors made public. They would take the evaluation system more seriously, they said, if they saw the results and knew their comments could help future students. They see the current system as opaque and don’t see that their evaluations matter much. That’s one reason they turn to sources like RateMyProfessor.com. They understand the downside of that sort of ratings system – that students who have had really good or really bad experiences are most likely to post – but they say the university has offered them no other options.

What students want families and friends to know

Students are often surprised by how education changes their relationships with friends and family members, they said. That’s especially true with first-generation college students, whose burgeoning independence, broadening views of the world, and distance from home make them wonder what to talk about with family and old friends. Those who had once anchored their lives seem to drift away as their lives change, they form new friendships, and they begin to understand better who they are.

As you can imagine, that unmooring generates stress. Students still care about family and friends back home, but they are torn between a new life and an old life. Family members and longtime friends, they say, sometimes don’t seem to value the changes the students are undergoing. Those changing relationships add to the stress students feel, and that stress can be enormous. A recent study of American freshmen found that a third felt overwhelmed by all the things they had to do in college.

None of this is particularly surprising, but it serves as a good reminder about the complexities of higher education. Students some to college to learn, and yet learning involves a tangle of interrelated components. Students at one focus group were especially vocal about something they wanted faculty members to keep in mind:

We are people, too.


Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

By Doug Ward

Asked to describe the things that help them learn, students provide a remarkably consistent list:

  • Engagement
  • Interaction
  • Clarity
  • Openness
  • Accessibility
  • A sense of belonging

That’s hardly a complete list, but those ideas came up again and again during a focus group at KU’s recent Student Learning Symposium. Not surprisingly, those same components come up again and again in research on learning.

sara rosen, holly storkel and stuart day at the Student Learning Symposium
Holly Storkel accepted the university’s Degree-Level Assessment award on behalf of the Speech-Language Pathology program. She was joined at the Student Learning Symposium by Sara Rosen, acting provost, and Stuart Day, acting senior vice provost for academic affairs.

The focus group at the learning symposium consisted of three undergraduate and three graduate students who had agreed to share their experiences at a round-table discussion. They were open and candid in their assessment of their own learning and of the learning they see in classes they help with. These categories overlap, but they provide a good sense of what students say is important.

Engagement

One of the undergraduates, who also works as a TA for a biology class, described students’ mindset about their classes like this: “Is there a reason for being there?” That is, will attending class help them do better on quizzes, exams, papers, and other assignments? Or will the instructor just read from slides that are later posted online? When in-class work engages students and clearly makes a difference in their performance, “now there’s a reason to be there,” students said.

Openness

Many students are anxious and apprehensive at the beginning of a term, and a sullen or seemingly angry professor will push them away with just a look. Professors who do nothing except lecture can seem standoffish, students said, and students don’t know how those professors will react if students try to talk with them.

On the other hand, instructors who are approachable make a big difference in classes, students say. Even in big classes, some instructors walk around the room, talk with students and answer questions. It’s impossible to get to know students well in big classes, but setting a positive tone can make students feel that they belong and want to be involved in a class, students said.

Clarity

Too many instructors teach as if students are experts rather than beginners, students said. Instructors need to think more from students’ perspective and try to remember what it’s like to work with new material for the first time. Professors simply must get better at explaining, students said.

Relatedly, students say are willing to go along with experimentation in format, assignments, and other elements of a class. The key is communication. “As long as instructors explain, students will be up for that,” one undergraduate said.

Both undergraduate and graduate students said that disciplinary programs needed to do a better job of helping students see the bigger picture of a program or a degree. Students need to “understand that there’s a future to all this information,” one student said. Graduate students said departments also needed to do a better job of integrating coursework with research.

Sense of belonging

The first semester at a new university can be painful, and successfully negotiating the freshman jitters makes a huge difference in whether students will stay the year or eventually graduate. Universities have recognized that and use programs like First-Year Experience to reach out to freshmen and help them maneuver through the educational maze.

Graduate students said they experienced much the same pain, loneliness, and disorientation as undergraduates. Unfortunately, they said, the university doesn’t do much to help them get situated. They recommended orientation sessions for graduate students, a better means of acquiring coping and learning strategies from peers, and better communication from departments. Graduate school is “so much more rigorous and so much harder,” one student said, and the university needs to do a better job of helping students cope.

Interaction

Interaction among students, and between instructor and students makes class much more meaningful, students said, lamenting about classes where they are simply “talked at.” One undergraduate said, “Lecture makes me wonder why I am coming to class to have you read slides to me.”

A Ph.D. student in the sciences echoed that sentiment. Too many graduate courses in STEM fields involve little but lecture, he said. He came to graduate school expecting to delve into critical analysis of problems and pursuit of new ideas. Instead, he found a frustrating emphasis on delivery of information. Graduate students in STEM fields are discouraged from discussion, he said, even though “interaction makes or breaks a class.”

Training for TAs

Graduate students at the learning symposium said the university needed to do a better job of helping them learn to teach effectively, and to provide incentives for graduate assistants to teach well. Many teaching assistants, especially those in the sciences, are told to spend as little time as possible on teaching, students said. (We’ve heard the same lament from students at CTE events for years.)

Gradate programs should be designed so that doctoral students learn to be well-rounded faculty members, not just research machines, students said. They called for more teaching workshops and other training for TAs. “A lot of grad students are doing only research,” one student said. “Once they graduate, they will be thrown into teaching with no experience.” He added, “This narrow focus is hurting many grad students.”


Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

CTE’s Twitter feed