Bloom's Sixth

Author Archives: dbward

A national conversation on evaluating teaching starts to take shape

By Doug Ward

A recent meeting at the National Academies of Sciences, Engineering and Medicine achieved little consensus on how best to evaluate teaching, but it certainly showed a widespread desire for a fairer system that better reflects the many components of excellent teaching.

The National Academies co-sponsored the meeting earlier this month in Washington with the Association of American Universities and TEval, a project associated with the Center for Teaching Excellence at KU. The meeting brought together leaders from universities around the country to discuss ways to provide a richer evaluation of faculty teaching and, ultimately, expand the use of practices that have been shown to improve student learning.

A CTE rubric for evaluating teaching helps instructors and departments focus on a series of questions.

My colleague Andrea Greenhoot, professor of psychology and director of CTE, represented KU at the meeting. Members of the TEval team from the University of Massachusetts, Amherst, the University of Colorado, Boulder, and Michigan State University also attended. The TEval project involves more than 60 faculty members at KU, CU and UMass. It received a five-year, $2.8 million grant from the National Science Foundation last year to explore ways to create a fairer, more nuanced approach to evaluating teaching.

The TEval project, which is known as Benchmarks at KU, has helped put KU at the forefront of the discussion about evaluating teaching and adopting more effective pedagogical strategies. Nine departments have been working to adapt a rubric developed at CTE, identify appropriate forms of evidence, and rethink the way they evaluate teaching. Similar conversations are taking place among faculty at CU and UMass. One goal of the project is to provide a framework that other universities can follow.

Universities have long relied on student surveys as the primary – and often sole – means of evaluating teaching. Those surveys can gather important feedback from students, but they provide only one perspective on a complex process that students know little about. The results of the surveys have also come under increasing scrutiny for biases against some instructors and types of classes.

Challenges and questions

The process of creating a better system still faces many challenges, as speakers at the meeting in Washington made clear. Emily Miller, associate vice president for policy at the AAU, said that many universities were having a difficult time integrating a new approach to evaluating teaching into a rewards system that favors research and that often counts teaching-associated work as service.

“We need to think about how we recognize the value of teaching,” Miller said.

She also summarized questions that had arisen during discussions at the meeting:

  • What is good teaching?
  • What elements of teaching do we want to evaluate?
  • Do we want a process that helps instructors improve or one that simply evaluates them annually?
  • What are the useful and appropriate measures?
  • What does it mean to talk about parallels between teaching and research?
  • How can we situate the conversation about the evaluation of teaching in the larger context of institutional change and university missions?

Noah Finkelstein, a University of Colorado physics professor who is a principal investigator on the TEval grant, brought up additional questions:

  • How do we frame teaching excellence within the context of diversity, equity and inclusion?
  • How can we create stronger communities around teaching?
  • How do we balance institutional and individual needs?
  • How do we reward institutions who improve teaching?
  • When will AAU membership be contingent on teaching excellence?

Moving the process forward

Instructors at KU, CU and UMass are already grappling with many of the questions that Miller and Finkelstein raised.

At KU, a group will meet on Friday to talk about the work they have done in such areas as identifying the elements of good teaching; gathering evidence in support of high-quality teaching practices; developing new approaches to peer evaluation for faculty and graduate teaching assistants; providing guidance on instructor reflection and assessment; and making the evaluation process more inclusive. There have also been discussions among administrators and Faculty Senate on ways to integrate a new approach into the KU rewards structure. Considerable work remains, but a shift has been set in motion.

KU faculty and staff share insights on teaching

Several KU faculty members have recently published articles about their inquiry into teaching. Their articles are well worth the time to read. Among them:

Briefly …

  • Writing in EdSurge, Bryan Alexander says that “video is now covering a lot of ground, from faculty-generated instructional content to student-generated works, videoconferencing and the possibility of automated videobots.” The headline goes beyond anything in the article, but it nonetheless raises an interesting thought: “Video assignments are the new term paper.”
  • The Society for Human Resource Management writes about a trend it calls “microinternships,” which mirror the work of freelancers. Microinternships involve projects of 5 to 20 hours that the educational technology company Parker Dewey posts on a website. Students bid on the work, and Parker Dewey takes a percentage of the compensation. The company says it is working with 150 colleges and universities on the microinternship project.
  • Writing in The Chronicle of Higher Education, Aaron Hanlan argues that by relying on a growing number of contingent, “disposable” instructors, “institutions of higher education today operate as if they have no future.” In following this approach, tenured faculty and administrators “are guaranteeing the obsolescence of their own institutions and the eventual erasure of their own careers and legacies,” he argues.
  • EAB writes about the importance of reaching out to students personally, saying that email with a personal, supportive tone can be like a lifeline to struggling students.
  • Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

Peer learning expands as instructors remake courses

By Doug Ward

Watching David Johnson’s class in digital logic design is a bit like watching synchronized swimming.

After a few minutes of announcements, Johnson and half a dozen GTAs and undergraduate teaching fellows fan out across an Eaton Hall auditorium as 60 or so students begin to work on problems that Johnson has assigned.

student with white headphones sit at table and listens to instructor who leans on table and speaks
David Johnson works with a student during Introduction to Digital Logic Design.

A hand goes up on one side of the room. Johnson approaches, and students around him listen intently as he asks questions and quietly offers advice. Across the aisle, a group of four young men confers about the problem, looking things up on laptops, writing down notes by hand, erasing, writing again, and sharing ideas. A few rows ahead, two young women point at the problem on the screen at the front of the room. They confer, take notes, and confer more. Across the room, hands go up and, one by one, the class assistants approach, offer their help and then search for more raised hands.

“We’re always busy helping someone,” Johnson said.

Similar scenes have increasingly played out across the university – and across the country – as a growing number of instructors, primarily in STEM fields, have hired undergraduate teaching assistants to work in their classes. The undergrad TAs are just one example of how colleges and universities have elevated the importance of peer learning as part of their efforts to retain students and to help them move toward graduation.

Many educational roles

At KU, those efforts extend into many areas. First-Year Experience, honors, pharmacy and business are among the programs that use peer mentors. The Undergraduate Advising Office has a team of peer advisors who, among other things, help students navigate choices of classes and majors, and help them find campus resources. As anxiety and depression have increased among students, Counseling and Psychological Services has created a peer educator program to work with students on mental health.

young woman helps a student with a problem in both foreground and background
Student assistants help their peers work through problems in David Johnson’s class.

The university’s Supplemental Instruction program is also growing and now has peer leaders working with two dozen courses. Those peer leaders have successfully completed the course they are helping with. They lead sessions in which students review course material, prepare for exams, work on study skills, and offer support to one another throughout a semester. That approach, which started at the University of Missouri-Kansas City, has been found to improve grades, retention rates and, ultimately, graduation rates.

Peers have always played a role in learning, and they have long been involved in writing programs, tutoring and review sessions. The use of large numbers of undergraduate assistants in classes is relatively new, though, and is tied to a growing use of flipped classes, active learning, and in-class problem-solving. Over the past five years or so, KU instructors in such fields as geology, biology, chemistry, physics, and engineering have hired undergraduate assistants to work in their largest classes. Those assistants do such things as monitor online discussion boards, help with labs, proctor exams, and hold office hours. A pre-semester training program was started two years ago for undergraduate assistants in STEM courses, allowing the assistants and instructors to gain a better understanding of how to work together.

Undergraduate assistants have been crucial in transforming large lecture courses into hubs of active learning. Coordinating with instructors and graduate teaching assistants, they monitor groups or sections of a classroom, answer questions, offer praise, and through their interactions with students, make large courses more personal. Like students who lead Supplemental Instruction sessions, the undergraduate assistants in large classes have recently taken the course, so they understand the flow of the class, the course material, and the areas where students are most likely to struggle.

As they help their peers, they hone their own understanding of course material, improve their communication skills, and gain experience working with groups of people. That deeper understanding helps prepare them for upper-level classes as well as medical school exams, internships and graduate school. Most rely on the money they earn to help pay their college bills. Nearly all report a sense of satisfaction from the experience.

“When you see someone finally get it, it’s really cool,” one undergraduate assistant said at a recent training session.

Zero lecture timestudent assistant stands and leans elbow on table as student points to a problem on a sheet of paper

Johnson has used graduate teaching assistants, Supplemental Instruction assistants, and undergraduate teaching fellows (as the assistants are known in engineering) to make dramatic changes in his classes. With the help of Molly McVey, a post-doctoral teaching fellow in engineering, Johnson flipped a course in which he was lecturing about half of class time. They created online materials that helped students prepare for in-class problem-solving and hired undergraduate fellows to help in the classroom. When the flipped version debuted last year, lecture time had dropped to nearly zero.

“The only time they really hear me speak is if I have an announcement, usually to remind them of a test,” Johnson said.

Johnson first tried the flipped approach during a summer computer science camp for high school students, and he was surprised by how much more students learned. So he began to transform EECS 140, which he described as a gateway course required of all students in electrical engineering. He received a course transformation grant from the Center for Teaching Excellence and worked with course designers and video specialists at the Center for Online and Distance Learning to create online materials. He had the undergraduate assistants create the in-class problems, which he described as “nearly perfect” because they require considerable thought but can still be completed during class time.young man in yellow shirt runs his hand through his hair as a student assistant sits beside him and helps him with a problem

During the first week, students aren’t sure what to make of the hands-on approach, but that hesitancy quickly disappears as they adapt to the in-class problem-solving.

“The first class they were just sitting there waiting,” Johnson said. “I explained to them again that they could start working. By the third or fourth class, they were already asking questions even before the class started.”

The questions continue throughout the class period, and Johnson and the student assistants are constantly on the move. The constant interaction has helped Johnson better connect to the class.

“When I walk around and talk to students, I really understand what they don’t understand,” Johnson said. “That really helps me do a better job.”

Students are free to leave class once they complete the day’s problems, but many stay for the entire period. Some who finish early work on the next online module, knowing they can get help if they have questions. Others like to stay and help their peers.

Improved learning and a sense of satisfaction

Learning has improved significantly in many areas of the class, and the number of students who drop or fail has fallen. The approach isn’t perfect. Johnson said there was a dip in some areas on the last of the four exams he gives. By the end of the semester, though, many students know that their grades won’t change much regardless of how they do on the last exam, so they don’t approach that exam with the same seriousness they do earlier work, Johnson said.

During class, though, the students are focused and engaged. A hand goes up at the back. An undergraduate teaching fellow kneels, listens and offers advice. A hand goes up in front, in the center, on the far side of the room. Johnson moves from table to table, student to student.

“Students always think it’s easy for an instructor to do that,” Johnson said. “For me, it’s a lot harder to go around and explain something to someone who doesn’t understand some things than it is to just stand up there and flap away and hope they understand. It takes a lot more energy out of me, but I feel much better at the end of class when I think, ‘Wow, I really did teach somebody something.’ ”

Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

Moving higher education from storied past to innovative future

Ann Austin stands on stairs in a lecture hall, raising her hand and looking across the room
Ann Austin calls for a show of hands during her keynote address at the Teaching Summit.

By Doug Ward

We know the story well. We helped write it, after all.

As instructors and students and administrators, we have lived the story of modern higher education. And yet, despite the familiarity of that story – or perhaps because of it – we continue to struggle with its meaning and direction.

Ann Austin, an education professor and administrator at Michigan State, told participants at KU’s annual Teaching Summit last week that that struggle is not only natural; it is also crucial as colleges and universities adapt to a landscape that has changed dramatically over the past 20 years and is poised to change even more dramatically in the next 20.

In her Summit keynote address, Austin moved among the past, present and future as she highlighted the challenges and opportunities that rapid societal changes are posing to colleges and universities. She also challenged faculty members and administrators to think philosophically and creatively about the way they teach, interact and plan.

“What kind of vision do we have in the back of our minds as we go about our day-to-day work?” Austin asked.

“What is our vision for where our learners are going, and what is our vision for the role we play in their lives?”

That vision, after all, guides us in conscious and unconscious ways, and is crucial for the success of the university. We are doing many good things, she said, but we need to be more creative in working with students, curricula and our approach to learning.

‘This noble profession’

Austin maintained an upbeat tone as she made a case that colleges and universities must change to keep pace with society. Universities are exemplars of society, places to share ideas, to advance knowledge and to debate with respect, she said. She evoked the symbolism of KU’s campus on a hill as an indication that it is “involved in something important,” or what she called “this noble profession.”

Ann Austin turns in her seat and speaks with two women behind her
Ann Austin speaks with Lisa Wolf-Wendel and Susan Twombly, professors of education, before the start of the Summit.

Even so, those of us who teach and work and learn and lead at universities must push our institutions to adapt and evolve. We have welcomed an increasingly diverse population of students, Austin said, and we must find better ways to support those students. Right now, she said, there’s a mismatch between social needs and educational practices and outcomes. (There is also a growing political rift over the direction of higher education.) We are doing much good, she said, but we need to do more.

“How do we create environments for the success of all?” Austin asked.

She pointed to large gateway classes as an example of where universities have fallen short. Those courses can guide students toward many types of careers – or prevent them from pursuing those careers. Nationally, half of students in those courses fail, she said, and women and students of color encounter the biggest hurdles. By embracing evidence-based teaching practices and taking a more inclusive approach to teaching and learning, though, we can lower the barriers to success.

“We know that if we change the way we go about our teaching, if we think about what will support this diversity of learners, we can pretty much get rid of that gap,” she said, citing years of research about active and engaged learning.

‘Generosity of thinking’ and other any areas of potential

Gateway courses are just one area where there is a mismatch between social needs and educational practices and outcomes, she said. Another involves soft skills, or what Austin calls “human skills”: things like communicating well; discerning between accurate and inaccurate information; understanding the context of problems and actions; engaging in teamwork and collaboration; and approaching work with integrity and ethical standards.

white man in jeans, left hand in pocket, holds a lavalier microphone as he stands on the stops of Budig lecture hall and speaks
Jeff Hall, professor of communication studies, asks a question during the Summit.

She also singled out something she called “generosity of thinking,” or the ability to work with people different from yourself and to seek out those complementary perspectives on projects at work and in communities.

“We really need to cultivate that even more than perhaps we do,” Austin said.

Austin drew upon her work as co-chair of the Roundtable on Systemic Change in Undergraduate STEM Education for the National Academies of Science, Engineering and Medicine. That group has highlighted the importance of a vibrant educational system and a well-educated citizenry that can join conversations on the challenges facing society. It has also focused on the needs of a changing workforce.

We know that jobs that are common today won’t exist in the future, Austin said. And in 10 or 20 years, “there will be opportunities for work that we can’t even imagine right now.”

“How do we prepare our students for this kind of world?” she asked.

What can we do?

I’ve written before about Austin, who cofounded the Center for Integration of Research, Teaching, and Learning. Her work in organizational change has influenced some of the approaches we take at CTE, and she is a partner on a National Science Foundation grant on creating a more nuanced approach to evaluating teaching. She has worked with many KU faculty members on that project, which is known as Benchmarks for Teaching Effectiveness at KU. The multi-university project is known as TEval.

Austin provides broad insight and thought-provoking questions to everything she does, and the Summit was no exception. She also offered several concrete steps that participants could take to improve their courses, their departments and the learning environments for their students:

white man with mustouche and open collar shirt sits in audience, holding microphone, as he asks question
Robert Hagen, lecturer in environmental studies, asks a question during the Summit.
  • Embrace high-impact practices. These include things like service learning, internships, writing-intensive courses, and learning communities. These and other practices “link the knowing with the doing,” Austin said, and create a more equitable learning environment.
  • “Become more fluent in how learning happens.” Research into learning and higher education continually provides new insights, Austin said, urging participants to consider ways of applying that research in their disciplines. CTE programs and materials can help instructors do that without spending hours combing through journals.
  • Focus on learning, not seat time. Our courses are organized by credit hours, a system that originated in the 19th century and focused on the amount of time instructors delivered information to students. That system is outmoded, especially for online courses, but we can still work within it, Austin said, by emphasizing learning and using effective means of assessing learning.
  • Seek out new ways to reach students. This might involve using technology, taking an innovative approach in face-to-face or online courses or curricula, or using new types of physical classrooms. Austin emphasized the importance of flexibility and creativity in helping students learn. Organize curricula in new ways and look for new pathways that better fit today’s students. She said that included not just degrees but ways for people to move in and out of higher education to refresh skills and share their expertise.
  • Cultivate new partnerships. Communities inside and outside the university help us draw on new perspectives, learn from one another, and create new learning opportunities for our students and our colleagues. These partnerships can also provide opportunities for developing and promoting leadership skills that universities need if they hope to innovate.

Even as she pushed audience members to take action, she urged them to draw on the many good things already happening at universities.

“I’m not in any way suggesting that we just jettison what we’re doing,” Austin said. “We do so much that is so good.”

Rather, she suggested committing to effective practices and ask “what is this changing world suggesting that we might do differently?”

Doing so helps us move from story – a beacon on a hill in a volatile, changing world – to action.

“That’s the story we are part of,” Austin said. “We need to think not only in a philosophical way – that’s part of the story – but in a real practical way. What do we do in our departments, in our programs and in the university to actually let us make the best contributions to our learners and to society?”

A whiteboard with Welcome to KU, new students written in blue
A whiteboard at the School of Engineering

A cloudy day with lots of sunshine

The Summit took place on the same day that hundreds of students moved in to KU’s residence halls. Chancellor Doug Girod, dressed in khaki slacks and a blue KU polo shirt, said at the beginning of the Summit that he always looked forward to helping with the move-in and talking with students and their families.

The day was cloudy, and the sky threatened rain, but school had yet to start and a shiny eagerness and a positive energy permeated the campus.

“This is one of the few days of the year when everybody smiles,” Girod said.

Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

Using data to better understand student ‘grade surprise’

By Doug Ward

A colleague’s daughter recently finished her first year of college. In high school, he said, she had never really had to study to get good grades. In college, though, she had to adjust her study habits and her thinking after her early grades dipped below the A’s and B’s she had routinely – and easily – received.

That sort of dip in grades is common among traditional freshmen as they learn to live away from home for the first time, deal with the liberation and temptations of personal independence, and try to make sense of the academic expectations of college. How they deal with that jolt can affect everything from their choice of majors to their ability to stay in college.

Jennifer Meta Robinson, an anthropology professor at Indiana University-Bloomington, has been studying this phenomenon, which she calls “grade surprise.” Students usually have a good sense of the grades they will receive on assignments or exams, or in a class. When that expectation doesn’t match reality, though, they experience grade surprise.

woman gestures as she stands amid seated in a conference room
Jennifer Meta Robinson explains her work in “grade surprise” to members of the steering committee of the Bay View Alliance.

Robinson explained her research to the steering committee of the Bay View Alliance earlier this month in Bloomington, Indiana. Both Indiana and KU are members of the Bay View Alliance, a consortium of 10 research-intensive universities that are working to elevate teaching and improve learning among students. Robinson and colleagues in chemistry, computer science and informatics recently received a mini-grant from the Association of American Universities to continue their work of surveying students and analyzing university data to try to find questions they have about grade surprise among students:

  • How does grade surprise affect retention in various majors?
  • Does the power of grade surprise grow as students move through additional classes?
  • What approaches help students recover when they encounter grade surprise?

Robinson’s hypothesis is that grade surprise impedes student progress but can be mitigated. When students are overconfident, she said, failure is more painful than when they have low expectations about their grades.

“Surprise creates pain,” Robinson said.

She is also looking at the flip side of that: whether there is positive grade surprise.

“There’s a human tendency to rewrite the past,” she said. “We mitigate our pain by retelling our story in a way that makes it less surprising.”

For instance, students might tell themselves that a low grade was the instructor’s fault or that people like them just don’t do well with this type of material or in these types of classes. That type of thinking can easily push students out of classes or majors.

Interestingly, few students seem to blame instructors when grades come in lower than expected.

“We were surprised at how few students said, ‘The teacher had it in for me,’” Robinson said. “Or, ‘This was out of left field. I studied this other thing and it wasn’t on the test.’ There was very little of that. It really was about more about what I can do, what I practice, where I can spend more time. The locus of control was within.”

Disparities in distribution and reaction

Grade surprise isn’t equally distributed, Robinson said. Underrepresented minority students and first-generation students are more likely to be surprised by their grades. And women feel more disappointment when they receive lower grades.

Robinson and her colleagues have been sharing context about grades to try to ease some of the pain of grade surprise. For instance, in computer science and informatics classes at Indiana, women generally receive higher grades than men. In chemistry, women and men receive similar grades, although all receive lower grades than they did in high school.

“So women may feel that more, that disappointment in themselves, that setback of, ‘Oh, maybe I don’t belong,’” Robinson said. “But that’s where we could say to them that they may be processing this differently but the GPA facts of it are that they are doing the same.”

An analysis of data at Indiana shows that many students bounce back after the shock of an initial grade. They expect an A, receive a C but then eventually get an A in the course. Robinson and her colleagues want to better understand what students do to recover. They are also looking at the mindset of students who think they did poorly on, say, a midterm exam but actually did well. What happens if they enroll for the subsequent semester before they know their grade?

“What is that little detour through the course?” she asked. “How long does that hang in the air that you think you’ve bombed but you get that assignment back and got that A after all?”

A move toward wider use of data

Robinson describes the grade surprise project as one of many that “connect classes to the potential of big data.” Indiana has an ambitious program in helping faculty members combine university demographic data with data about student performance in classes. That combination is often referred to as learning analytics. The Indiana program, known as Learning Analytics Fellows, has led to more than 50 projects since it started in 2015. It is run through a recently created Center for Learning Analytics and Student Success.

We have been working on a similar project at KU, though at a smaller scale. An AAU mini-grant through the Center for Teaching Excellence has allowed several STEM departments to use university data to learn more about their students and about the paths they take through various curricula. The recently created Office of Analytics and Institutional Research (formerly the Office of Institutional Research and Planning) has continued the momentum around wider application of university data. One of its divisions focuses on academic data analytics and is looking at ways of making more data available to faculty members.

These types of data project allow instructors and departments to ask new questions about curricula, with an eye toward improving student retention and graduation rates. As Robinson explained in her talk at Indiana, this use of data is driving culture change as universities find ways to scale learning analytics even as they explore the potential of data in education. Robinson emphasized the importance of providing context for data and of applying “interpretive muscle” to tease out insights.

“These are drivers for change at all of our universities,” she said.

Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

A student view of effective teaching

By Doug Ward

AUSTIN, Texas – How do students view effective teaching?

They offer a partial answer each semester when they fill out end-of-course teaching surveys. Thoughtful comments from students can help instructors adapt assignments and approaches to instruction in their classes. Unfortunately, those surveys emphasize a ratings scale rather than written feedback, squeezing out the nuance.

Speaker holds microphone as she gestures toward screen
Christina Ormsbee and Shane Robinson of Oklahoma State explain results of a qualitative survey of student views of teaching at their university.

To address that, staff members from the Institute for Teaching and Learning Excellence at Oklahoma State spoke with nearly 700 students about the effectiveness of their instructors and their classes. They compiled that qualitative data into suggestions for making teaching more effective. Christina Ormsbee, director of the center at OSU, and Shane Robinson, associate director, shared findings from those surveys last week at the Big 12 Teaching and Learning Conference in Austin.

Here are some of the things students said:

  • Engage us. The students’ favorite instructors vary their approach to class, use interesting and engaging instructional methods, and use relevant examples.
  • Communicate clearly. Students value clear assignments, transparent communication, and timely, useful feedback. They also want lecture notes posted online.
  • Be approachable. Students described their favorite instructors as personable, professional and caring. “Students really want faculty to care about them,” Ormsbee said. They also want instructors to care about student learning. They complained about instructors who were abrasive, sarcastic or demeaning.
  • Align class time with assessments. Students want instructors to respect their time by using class activities and lessons that connect to out-of-class readings and build toward assessments.
  • Be available. Students want instructors to hold office hours at times that are convenient for students and to help them when they ask. They also expect instructors to communicate through the campus learning management system and though email and other types of media.
  • Be organized. Students appreciate organizational tools like detailed class agendas and timelines. They like study sessions before exams, but they also want instructors to go over material they missed on exams.
  • Slow down. Students say instructors often go through course material too quickly.
  • Grade fairly. Students dislike instructors who focus grading too heavily on one aspect of a course, grade too harshly, or deduct points for missing class or for not participating.
  • Don’t give us too much work. (You aren’t surprised, are you?)

Much of this aligns with the research on effective teaching and learning (engagement, alignment, organization, pacing, transparency, clarity). Some of it also aligns with aspects of universal design for learning (see below). Other aspects have as much to do with common courtesy as with good pedagogy. (We all want to feel respected.) Still other parts reflect a consumer mentality that has seeped into many aspects of higher education.

Feedback from students is important, but it is also just one of many things that instructors need to focus on. A class of satisfied students means nothing if none of them is learning. And students know little about the years of accumulated evidence about effective teaching. So we should listen, yes, but we should base decisions about our classes on an array of evidence and thoughtful reflection.

Universal design takes center stage

All too often, instructors, administrators and staff members think about accessibility of course content only when a student requests an accommodation.

The problem with that approach, said Melissa Wong of the University of Illinois at Urbana-Champaign, is that a vast majority of students who need accommodations never seek them out. Sometimes they don’t know about a disability or have never been formally diagnosed. In other cases, students are embarrassed about having to share personal details or assume they can make it through a class without an accommodation.

Wong called the current system of acquiring an accommodation “legalistic.” Students must have health insurance. They must fill out multiple forms and have records transferred. They must maneuver through university bureaucracy and find the right offices, a skill that many students lack. Then they must submit forms in each class they take. In class, they may confront inaccessible course materials, hazy expectations, and daunting assignments.

Each of those barriers adds to students’ burden, ultimately making things harder for instructors and for other students. Instructors can help all their students – even those who don’t need accommodations – by following the principles of universal design for learning, though, Wong said. Wong was among several speakers at the Big 12 conference who emphasized the importance of universal design for learning.

Universal design started with architecture (think curb cuts and self-opening doors), but its importance in education has grown as the diversity of students has grown. In essence, it is a way of thinking about learning in terms of student choices: multiple forms of engagement, multiple means of representation, and multiple forms of action and expression.

Speaker, holding microphone, gestures as he stands in front of a screen with Star Wars characters
Tom Tobin used a Star Wars theme to explain universal design for learning.

Tom Tobin of the University of Wisconsin-Madison suggested thinking of universal design in terms of “plus one.” If you have a written assignment, consider giving students one other option for completing the same work. If you provide a video, make sure it has captions.

“We don’t have to perfect,” Tobin said. “We just have to be good.”

He also suggested reframing the conversation about accessibility to one about access. Good access helps all students learn more effectively and keeps them moving toward graduation.

“The idea of UDL is not to lower the rigor of the material,” Tobin said. “The idea is to lower the barrier of getting into the conversation in the first place.”

Wong offered some additional advice on how to apply universal design in classes:

  • Use a clear organizational structure in your syllabus. Use subheads so that students can find everything easily. And make sure the syllabus has a section on accommodations.
  • Create a list of assignments and due dates. This helps students plan and cuts down on anxiety. Wong said a one-page assignment calendar she creates was one of the most popular things she had done for her classes.
  • Present information in a variety of ways (text, video, audio, multimedia), and provide examples of successful work. Offering choices in assignments can help students feel more in control and allow them to demonstrate learning in ways they are most comfortable with. For instance, you might give students a range of assignment topics to choose from and give them options like video or audio for presenting their work, in addition to writing.
  • Make sure video is close-captioned. If you have audio, make sure students have access to a transcript.
  • Use a microphone routinely, especially in large classrooms.
  • Scaffold assignments so that students can work toward a goal in smaller pieces.
  • Be flexible with deadlines. If you give one student an extension, make sure all students have the same option. If a student is chronically late with assignments or frequently seeks to make up work, try to understand the underlying problems and refer that student to offices on campus that can help.

The best approach is to take accessibility into account from the beginning rather than trying to retrofit things later, Wong said. That not only cuts down on the need for accommodations but creates a smoother route for all students.

Other nuggets from the conference:

Supplemental instruction success. A three-year study at the University of Texas-Austin found that student participation in supplemental instruction sessions improved grades in gateway courses in electrical engineering. Supplemental instruction involves regular student-led study sessions overseen by trained student facilitators. About 40% of students in UT’s Introduction to Electrical Engineering courses participated in supplemental instruction. I’ll be writing more about KU’s supplemental instruction program in the next few weeks.

Practical thinking. Shelley Howell of the University of Texas-San Antonio emphasized the importance of relevance in helping students move toward deeper learning. She drew on a model from Ken Bain’s book What the Best College Students Do, categorizing students into surface learners (who do just enough to get by), strategic learners (who focus on details and stress about grades) and deep learners (who are curious and ask questions, accept failure as a part of the learning process, and apply learning across disciplines). All students need to understand the purpose of individual assignments, and instructors need to make course content relevant, give students choices, and ask questions that take students on a “messy” path to understanding, Howell said.

Red alert. Educators have grown too complacent about student failure, Howell said, and would benefit from a Star Trek approach to student success. Every episode of Star Trek is essentially the same, she said: Something goes wrong. The problem must be fixed right away or the ship will crash. The problem is impossible to fix. The crew finds a way to fix it anyway. What if those of us in higher education had the same attitude? Howell asked, adding: If you knew that every student had to succeed, how would you teach differently?

A final thought. Emily Drabinski, a critical pedagogy librarian at the Graduate Center at City University of New York, offered this bit of wisdom: “For knowledge to be made, it has to be organized.”

Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

A teaching event grows into a celebration

By Doug Ward

When we started an end-of-semester teaching event four years ago, we referred to it simply as a poster session.

The idea was to have instructors who received grants from the Center for Teaching Excellence or who were involved in our various programs create posters and then talk with peers and visitors as they might at a disciplinary conference. In this case, though, the focus was on course transformation and on new ways that instructors had approached student learning.

As the event grew, we decided to call it the Celebration of Teaching, and it has become exactly that.

We didn’t do an official count at the event on Friday, but well more than 100 people attended. There were 54 posters created by instructors involved in Diversity Scholars, the Curriculum Innovation Program, and the Best Practices Institute, and those who received course transformation grants during the year.

Here’s view of the Celebration of Teaching in photographs.

Doug Ward is the acting director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

 

4 ways to increase participation in student surveys of teaching

By Doug Ward

If you plan to use student surveys of teaching for feedback on your classes this semester, consider this: Only about 50% of students fill out the surveys online.

Yes, 50%.

There are several ways that instructors can increase that response rate, though. None are particularly difficult, but they do require you to think about the surveys in slightly different ways. I’ll get to those in a moment.

The low response rate for online student surveys of teaching is not just a problem at KU. Nearly every university that has moved student surveys online has faced the same challenge.

That shouldn’t be surprising. When surveys are conducted on paper, instructors (or proxies) distribute them in class and students have 10 or 15 minutes to fill them out. With the online surveys, students usually fill them out on their own time – or simply ignore them.

——————————————————————————————————————

——————————————————————————————————————

I have no interest in returning to paper surveys, which are cumbersome, wasteful and time-consuming. For example, Ally Smith, an administrative assistant in environmental studies, geology, geography, and atmospheric sciences, estimates that staff time needed to prepare data and distribute results for those four disciplines has declined by 47.5 hours a semester since the surveys were moved online. Staff members now spend about 4 hours gathering and distributing the online data.

That’s an enormous time savings. The online surveys also save reams of paper and allow departments to eliminate the cost of scanning the surveys. That cost is about 8 cents a page. The online system also protects student and faculty privacy. Paper surveys are generally handled by several people, and students in large classes sometimes leave completed surveys in or near the classroom. (I once found a completed survey sitting on a trash can outside a lecture hall.)

So there are solid reasons to move to online surveys. The question is how to improve student responsiveness.

I recently led a university committee that looked into that. Others on the committee were Chris Elles, Heidi Hallman, Ravi Shanmugam, Holly Storkel and Ketty Wong. We found no magic solution, but we did find that many instructors were able to get 80% to 100% of their students to participate in the surveys. Here are four common approaches they use:

Have students complete surveys in class

Completing the surveys outside class was necessary in the first three years of online surveys at KU because students had to use a laptop or desktop computer. A system the university adopted two years ago allows them to use smartphones, tablets or computers. A vast majority of students have smartphones, so it would be easy for them to take the surveys in class. Instructors would need to give notice to students about bringing a device on survey day and find ways to make sure everyone has a device. Those who were absent or were not able to complete the surveys could still do so outside class.

Remind students about the surveys several times

Notices about the online surveys are sent by the Center for Online and Distance Learning, an entity that most students don’t know and never interact with otherwise. Instructors who have had consistently high response rates send out multiple messages to students and speak about the surveys in class. They explain that student feedback is important for improving courses and that a higher response rate provides a broader understanding of students’ experiences in a class.

To some extent, response rates indicate the degree to which students feel a part of a class, and rates are generally higher in smaller classes. Even in classes where students feel engaged, though, a single reminder from an instructor isn’t enough. Rather, instructors should explain why the feedback from the surveys is important and how it is used to improve future classes. An appeal that explains the importance and offers specific examples of how the instructor has used the feedback is more likely to get students to act than one that just reminds them to fill out the surveys. Sending several reminders is even better.

Give extra credit for completing surveys

Instructors in large classes have found this an especially effective means of increasing student participation. Giving students as little as 1 point extra credit (amounting to a fraction of 1% of an overall grade) is enough to spur students to action, although offering a bump of 1% or more is even more effective. In some cases, instructors have gamified the process. The higher the response rate, the more extra credit everyone in the class receives. I’m generally not a fan of extra credit, but instructors who have used this method have been able to get more than 90% of their students to complete the online surveys of teaching.

Add midterm surveys

A midterm survey helps instructors identify problems or frustrations in a class and make changes during the semester. signaling to students that their opinions and experiences matter. This in turn helps motivate students to complete end-of-semester surveys. Many instructors already administer midterm surveys either electronically (via Blackboard or other online tools) or with paper, asking students such things as what is going well in the class, what needs to change, and where they are struggling. This approach is backed up by research from a training-evaluation organization called ALPS Insights, which has found that students are more likely to complete later course surveys if instructors acknowledge and act on earlier feedback they have given. It’s too late to adopt that approach this semester, but it is worth trying in future semesters.

Remember the limitations

Student surveys of teaching can provide valuable feedback that helps instructors make adjustments in future semesters. Instructors we spoke to, though, overwhelmingly said that student comments were the most valuable component of the surveys. Those comments point to specific areas where students have concerns or where a course is working well.

Unfortunately, surveys of teaching have been grossly misused as an objective measure of an instructor’s effectiveness. A growing body of research has found that the surveys do not evaluate the quality of instruction in a class and do not correlate with student learning. They are best used as one component of a much larger array of evidence. The College of Liberal Arts and Sciences has developed a broader framework, and CTE has created an approach we call Benchmarks for Teaching Effectiveness. It uses a rubric to help shape a more thorough, fairer and nuanced evaluation process.

Universities across the country are rethinking their approach to evaluating teaching, and the work of CTE and the College are at the forefront of that. Even those broader approaches require input from students, though. So as you move into your final classes, remind students of the importance of their participation in the process.

(What have you found effective? If you have found other ways of increasing student participation in end-of-semester teaching surveys, let us know so we can share your ideas with colleagues.)

The ‘right’ way to take notes isn’t clear cut

Photo by The Climate Reality Project on Unsplash

A new study on note-taking muddies what many instructors saw as a clear advantage of pen and paper.

The study replicates a 2014 study that has been used as evidence for banning laptop computers in class and having students take notes by hand. The new study found little difference except for what it called a “small (insignificant)” advantage in recall of factual information for those taking handwritten notes.

Daniel Oppenheimer, a Carnegie Mellon professor who is a co-author of the new paper, told The Chronicle of Higher Education:

“The right way to look at these findings, both the original findings and these new findings, is not that longhand is better than laptops for note-taking, but rather that longhand note-taking is different from laptop note-taking.”

A former KU dean worries about perceptions of elitism

Kim Wilcox, a former KU dean of liberal arts and sciences, argues in Edsource that the recent college admissions scandal leaves the inaccurate impression that only elite colleges matter and that the admissions process can’t be trusted.

“Those elite universities do not represent the broad reality in America,” writes Wilcox, who is the chancellor of the University of California, Riverside. He was KU’s dean of liberal arts and sciences from 2002 to 2005.

He speaks from experience. UC Riverside has been a national leader in increasing graduation rates, especially among low-income students and those from underrepresented minority groups. Wilcox himself was a first-generation college student.

He says that the scandal came about in part by “reliance on a set of outdated measures of collegiate quality; measures that focus on institutional wealth and student rejection rates as indicators of educational excellence.”

Wilcox was chair of speech-language-hearing at KU for 10 years and was president and CEO of the Kansas Board of Regents from 1999 to 2002.

Join our Celebration of Teaching

CTE’s annual Celebration of Teaching will take place Friday at 3 p.m. at the Beren Petroleum Center in Slawson Hall. More than 50 posters will be on display from instructors who have transformed their courses through the Curriculum Innovation Program, C21, Diversity Scholars, and Best Practices Institute. It’s a great chance to pick up teaching tips from colleagues and to learn more about the great work being done across campus.

Doug Ward is the acting director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

Drawing inspiration from the stories of innovative teaching

By Doug Ward

Higher education has many stories to tell.

Finding the right story has been difficult, though, as public colleges and universities have struggled with decreased funding, increasing competition for students, criticism about rising tuition, skepticism from employers and politicians about the relevance of courses and degrees, and even claims that the internet has made college irrelevant.

Prajna Dhar works with a student during class.

One top of that, students increasingly see higher education as transactional. Colleges and universities have long lived on a promise that time, effort and learning will propel students to a better life and the nation to a more capable citizenry. Today, though, students and their parents talk about return on investment. They want to know what they are getting for their money, what sort of job awaits and at what salary.

All of this has put higher education on the defensive, searching for a narrative for many different audiences. The Association of American Colleges and Universities has focused its last two annual meetings on telling the story of higher education. I led a workshop and later a webinar on that topic, and participants were eager to learn from each other about strategies for making a case for higher education. At one of this year’s sessions, a workshop leader asked participants whether their universities were telling the story of education well.

No one raised a hand.

All too often, universities use their color brochures and websites to explain how prestigious they are and to sell students on an ivory tower fantasy. Both of those things have a place. By focusing on education as a product, though, they overlook what college is really about: challenges, disappointments, maturity, opportunity, growth, and, above all, learning.

Matt Smith demonstrates how an augmented reality sandbox can create “rain.”

Teaching and learning rarely make their way into the stories that universities tell. If they did, here are some of the things students would learn about:

  • Kim Warren, associate professor of history, who has rethought the language she uses in her classes. In helping students think like historians, she treats everyone as English-language learners so that no one leaves class confused by terminology or expectations.
  • Prajna Dhar, associate professor of engineering, who has made sure that students with disabilities can participate in the active learning at the heart of her classes.
  • Mark Mort, who has transformed once-staid 100-level biology classes into vibrant hubs of activity where students help one another learn.
  • John Kennedy, associate professor of political science, who draws upon his expertise in international relations to help students work through negotiation scenarios that diplomats and secretaries of state struggle with.
  • Matt Smith, a GTA in geography, who has created an interactive sandbox that allows students to create terrain and use virtual rain to explore how water flows, collects and erodes.
  • Genelle Belmas, associate professor of journalism, who has created a “whack-a-judge” game to help students learn about media law and a gamification class that helps them learn about things like audience, interactivity, and creativity.
  • Phil Drake, associate professor of English, who uses peer evaluation to help students improve their writing and to gain practice at giving feedback.
  • M’Balia Thomas, assistant professor of education, who uses Harry Potter books to demonstrate to TESOL and ELL teachers how they can use students’ existing knowledge to motivate them and to learn new material.
  • Ward Lyles, assistant professor of urban planning, who passionately embraces team-based learning and who helps students learn to approach statistics with diversity, equity and inclusion in mind.

    Lisa Sharpe Elles explains her use of artificial intelligence software for grading during CTE teaching demos in November.
  • Lisa Sharpe Elles, assistant teaching professor in chemistry, who has increased the use of open-ended questions in large chemistry classes by using artificial intelligence for grading.
  • Sarah Gross, assistant professor of visual art, who uses self-assessment as a means for students to improve their pottery skills and to learn from peers.

I could go on and on. The stories of innovative techniques and inspirational approaches to teaching and learning at KU seem limitless. All too often, though, they go untold. That’s a shame because they are perhaps the most important stories that students and prospective students need to hear as they make decisions about college and college classes.

The stories we tell remind us of who we are and where we are going. One of our roles at CTE is to tell the stories of the inspiring teachers who form the heart of learning at KU. Another is to bring those teachers together in ways that allow them to inspire and learn from one another.

Great teaching is crucial to the future of higher education. It takes time, creativity, and passion. It is important intellectual work that deserves to be celebrated and rewarded.

That’s a story worth telling.

Doug Ward is the acting director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

Recent Posts

Tags

academic life active learning artificial intelligence assessment classroom design classrooms collaborative learning college costs college enrollment connecting with students course redesign course transformation Covid-19 Education Matters education trends engaged learning evaluating teaching evidence-based teaching flipped courses future of higher education hybrid learning innovation KU Teaching Summit learner-centered teaching learning spaces learning strategies liberal arts online education online learning online teaching reflective teaching remaking higher education scholarship of teaching and learning state funding STEM student engagement student evaluations student learning student skills student surveys teaching teaching and technology teaching conferences technology in the classroom workshops

Archives

Recent Posts

What we write about

academic life active learning artificial intelligence assessment classroom design classrooms collaborative learning college costs college enrollment connecting with students course redesign course transformation Covid-19 Education Matters education trends engaged learning evaluating teaching evidence-based teaching flipped courses future of higher education hybrid learning innovation KU Teaching Summit learner-centered teaching learning spaces learning strategies liberal arts online education online learning online teaching reflective teaching remaking higher education scholarship of teaching and learning state funding STEM student engagement student evaluations student learning student skills student surveys teaching teaching and technology teaching conferences technology in the classroom workshops

Recent Comments

Archives

CTE’s Twitter feed