By Doug Ward

If you plan to use student surveys of teaching for feedback on your classes this semester, consider this: Only about 50% of students fill out the surveys online.

Yes, 50%.

There are several ways that instructors can increase that response rate, though. None are particularly difficult, but they do require you to think about the surveys in slightly different ways. I’ll get to those in a moment.

The low response rate for online student surveys of teaching is not just a problem at KU. Nearly every university that has moved student surveys online has faced the same challenge.

That shouldn’t be surprising. When surveys are conducted on paper, instructors (or proxies) distribute them in class and students have 10 or 15 minutes to fill them out. With the online surveys, students usually fill them out on their own time – or simply ignore them.

——————————————————————————————————————

——————————————————————————————————————

I have no interest in returning to paper surveys, which are cumbersome, wasteful and time-consuming. For example, Ally Smith, an administrative assistant in environmental studies, geology, geography, and atmospheric sciences, estimates that staff time needed to prepare data and distribute results for those four disciplines has declined by 47.5 hours a semester since the surveys were moved online. Staff members now spend about 4 hours gathering and distributing the online data.

That’s an enormous time savings. The online surveys also save reams of paper and allow departments to eliminate the cost of scanning the surveys. That cost is about 8 cents a page. The online system also protects student and faculty privacy. Paper surveys are generally handled by several people, and students in large classes sometimes leave completed surveys in or near the classroom. (I once found a completed survey sitting on a trash can outside a lecture hall.)

So there are solid reasons to move to online surveys. The question is how to improve student responsiveness.

I recently led a university committee that looked into that. Others on the committee were Chris Elles, Heidi Hallman, Ravi Shanmugam, Holly Storkel and Ketty Wong. We found no magic solution, but we did find that many instructors were able to get 80% to 100% of their students to participate in the surveys. Here are four common approaches they use:

Have students complete surveys in class

Completing the surveys outside class was necessary in the first three years of online surveys at KU because students had to use a laptop or desktop computer. A system the university adopted two years ago allows them to use smartphones, tablets or computers. A vast majority of students have smartphones, so it would be easy for them to take the surveys in class. Instructors would need to give notice to students about bringing a device on survey day and find ways to make sure everyone has a device. Those who were absent or were not able to complete the surveys could still do so outside class.

Remind students about the surveys several times

Notices about the online surveys are sent by the Center for Online and Distance Learning, an entity that most students don’t know and never interact with otherwise. Instructors who have had consistently high response rates send out multiple messages to students and speak about the surveys in class. They explain that student feedback is important for improving courses and that a higher response rate provides a broader understanding of students’ experiences in a class.

To some extent, response rates indicate the degree to which students feel a part of a class, and rates are generally higher in smaller classes. Even in classes where students feel engaged, though, a single reminder from an instructor isn’t enough. Rather, instructors should explain why the feedback from the surveys is important and how it is used to improve future classes. An appeal that explains the importance and offers specific examples of how the instructor has used the feedback is more likely to get students to act than one that just reminds them to fill out the surveys. Sending several reminders is even better.

Give extra credit for completing surveys

Instructors in large classes have found this an especially effective means of increasing student participation. Giving students as little as 1 point extra credit (amounting to a fraction of 1% of an overall grade) is enough to spur students to action, although offering a bump of 1% or more is even more effective. In some cases, instructors have gamified the process. The higher the response rate, the more extra credit everyone in the class receives. I’m generally not a fan of extra credit, but instructors who have used this method have been able to get more than 90% of their students to complete the online surveys of teaching.

Add midterm surveys, 

A midterm survey helps instructors identify problems or frustrations in a class and make changes during the semester. signaling to students that their opinions and experiences matter. This in turn helps motivate students to complete end-of-semester surveys. Many instructors already administer midterm surveys either electronically (via Blackboard or other online tools) or with paper, asking students such things as what is going well in the class, what needs to change, and where they are struggling. This approach is backed up by research from a training-evaluation organization called ALPS Insights, which has found that students are more likely to complete later course surveys if instructors acknowledge and act on earlier feedback they have given. It’s too late to adopt that approach this semester, but it is worth trying in future semesters.

Remember the limitations

Student surveys of teaching can provide valuable feedback that helps instructors make adjustments in future semesters. Instructors we spoke to, though, overwhelmingly said that student comments were the most valuable component of the surveys. Those comments point to specific areas where students have concerns or where a course is working well.

Unfortunately, surveys of teaching have been grossly misused as an objective measure of an instructor’s effectiveness. A growing body of research has found that the surveys do not evaluate the quality of instruction in a class and do not correlate with student learning. They are best used as one component of a much larger array of evidence. The College of Liberal Arts and Sciences has developed a broader framework, and CTE has created an approach we call Benchmarks for Teaching Effectiveness. It uses a rubric to help shape a more thorough, fairer and nuanced evaluation process.

Universities across the country are rethinking their approach to evaluating teaching, and the work of CTE and the College are at the forefront of that. Even those broader approaches require input from students, though. So as you move into your final classes, remind students of the importance of their participation in the process.

(What have you found effective? If you have found other ways of increasing student participation in end-of-semester teaching surveys, let us know so we can share your ideas with colleagues.)

The ‘right’ way to take notes isn’t clear cut

Photo by The Climate Reality Project on Unsplash

A new study on note-taking muddies what many instructors saw as a clear advantage of pen and paper.

The study replicates a 2014 study that has been used as evidence for banning laptop computers in class and having students take notes by hand. The new study found little difference except for what it called a “small (insignificant)” advantage in recall of factual information for those taking handwritten notes.

Daniel Oppenheimer, a Carnegie Mellon professor who is a co-author of the new paper, told The Chronicle of Higher Education:

“The right way to look at these findings, both the original findings and these new findings, is not that longhand is better than laptops for note-taking, but rather that longhand note-taking is different from laptop note-taking.”

A former KU dean worries about perceptions of elitism

Kim Wilcox, a former KU dean of liberal arts and sciences, argues in Edsource that the recent college admissions scandal leaves the inaccurate impression that only elite colleges matter and that the admissions process can’t be trusted.

“Those elite universities do not represent the broad reality in America,” writes Wilcox, who is the chancellor of the University of California, Riverside. He was KU’s dean of liberal arts and sciences from 2002 to 2005.

He speaks from experience. UC Riverside has been a national leader in increasing graduation rates, especially among low-income students and those from underrepresented minority groups. Wilcox himself was a first-generation college student.

He says that the scandal came about in part by “reliance on a set of outdated measures of collegiate quality; measures that focus on institutional wealth and student rejection rates as indicators of educational excellence.”

Wilcox was chair of speech-language-hearing at KU for 10 years and was president and CEO of the Kansas Board of Regents from 1999 to 2002.

Join our Celebration of Teaching

CTE’s annual Celebration of Teaching will take place Friday at 3 p.m. at the Beren Petroleum Center in Slawson Hall. More than 50 posters will be on display from instructors who have transformed their courses through the Curriculum Innovation Program, C21, Diversity Scholars, and Best Practices Institute. It’s a great chance to pick up teaching tips from colleagues and to learn more about the great work being done across campus.


Doug Ward is the acting director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

By Doug Ward

Kerry Ann Rockquemore offers excellent advice about what she calls “the teaching trap.”

By that, she means putting so much of yourself into your teaching that you have no time or energy for research, writing or life outside the office. She writes:Education matters logo: Recent news, research, trends and thoughts about education

“If you find yourself coming to campus early and staying late, if you’re spending every weekend grading and preparing for the next week’s classes, if you’re answering student’s text messages into the wee hours of the night, if you’re sacrificing sleep and/or pulling all-nighters in order to get ready for the next day’s class meeting, and – as a result – you haven’t spent any time moving your research agenda forward or investing in your long-term success, then you may have fallen into the teaching trap.”

She provides several reasons this can happen but also offers ways of balancing academic life. Those include setting aside writing time every day, consulting with colleagues about how they handle their workloads, avoiding Ratemyprofessor.com (a perfectionist tendency), and seeking advice from your school’s teaching center.

I’ll add my own plug for her last suggestion. CTE offers many programs, workshops, discussions and publications to help faculty members improve their teaching (which includes finding a healthy balance). We also visit classes and consult with individual faculty members and departments about specific problems.

Teaching can sometimes feel like an all-encompassing, individual profession. It doesn’t have to be. In fact, all teachers are part of a broader community. Recognizing that and then joining the community helps make us all better.


Table showing bachelor's degree outcomes by region, from the National Association of Colleges and Employers survey
From First Destinations for the College Class of 2014, a survey by the National Association of Colleges and Employers: https://www.naceweb.org/surveys/first-destination.aspx?mainindex-banner-first-dest-hom-06042015

Region makes a difference for recent grads

More than 65 percent of 2004 graduates from the Plains states and New England found full-time jobs within six months of leaving college, a new survey says.

Those percentages (Plains, 67.4 percent; New England, 66.1 percent) were the highest among all regions. Overall, 52.5 percent of 2004 graduates reported finding full-time jobs within six months of graduating.

More than 90 percent of students from colleges in those areas, along with those in the Great Lakes states, reported a positive “career outcome,” meaning they had found a job or were continuing their education six months after graduating, the survey said. That is far higher than other regions of the United States. Colleges from the Southeast reported the lowest percentage (67); the national average was 80 percent.

The survey found little difference among institutions in urban, suburban or rural areas. (It also broke down job status based on area of study, but that is too detailed to go into here.)

Graduates from private, nonprofit colleges and universities (58.5 percent) were far more likely to have found jobs than their counterparts in public institutions (48 percent), the survey said, and those with professional degrees were slightly more likely than those with liberal arts degrees to have found full-time jobs (58.7 percent vs. 53.8 percent). The survey’s authors say this is partly the result of student expectations: Those with professional degrees tend to focus on getting jobs; those with liberal arts degrees often go to graduate school.

What are we to make of the data? The report doesn’t really say, except to point out that more than 20 percent of graduates were “adrift” six months after they left college. I found the regional data especially interesting. I have ideas about that, but I won’t speculate without seeing further data.

The survey, from the National Association of Colleges and Employers, contains data from more than 200 colleges and universities. Its creators say it is the first such survey to use a common methodology among all participants, and they told Inside Higher Ed that they hope it will create a baseline for future surveys of graduates.

Briefly …

In an article for Faculty Focus, Berlin Fang warns against becoming a “helicopter professor,” saying it is important to let students struggle with concepts and find answers on their own. … Heather Cox Richardson, a professor at Boston College, writes for Slate on what she calls Gov. Scott Walker’s new Wisconsin Experiment, putting it into the context of “eighty years of maligning universities as hotbeds of socialism in an attempt to undercut workers’ influence in government.” … Jill Barshay of the Hechinger Report says that student debt falls most heavily on three types of students: graduate students, students at for-profit colleges, and dropouts.


Doug Ward is an associate professor of journalism and the associate director of  the Center for Teaching Excellence. You can follow him on Twitter @kuediting.

By Doug Ward

I often roll my eyes at articles that take millennials to task for not measuring up to the standard of the day. All too often, baby boomers and those in generations before seem to wag their fingers at young people and spew out curmudgeonly laments that inevitably start with, “When I was your age …”

question on reading a chart from ETS survey
Sample questions, above and below, from the international survey of millennials’ skills

As I dug into a new report by the Educational Testing Service, though, I began to buy into the concerns it raises about the skills of American millennials when compared with those of their counterparts worldwide. (ETS creates the GRE and TOEFL tests, among others.) The new report, written by Madeline J. Goodman, Anita M. Sands, and Richard J. Coley, is called “America’s Skills Challenge: Millennials and the Future.”

Across the board, data from the study suggest that Americans rank near the bottom in literacy, numeracy, and problem-solving in technology-rich environments (things like using digital maps and calculating the price of tickets from a kiosk). This holds true for high school graduates and for those with bachelor’s and master’s degrees.

“A decade ago, the skill level of American adults was judged ‘mediocre,’” the authors write. “Now it is below even that.”

More education, fewer skills

The 82 million American millennials – those born after 1980 – have received more formal education than any previous generation, the authors write, yet their demonstration of skills in literacy and numeracy are middling at best. The U.S. is the wealthiest among 22 nations where the test was administered, but it is also “the most economically unequal.”

Given the poor performance of American millennials on these tests, the report says, we should “consider critically the value that higher education in the U.S. is contributing to the skills of our young adults,” adding: “For education to be a vehicle to future success, for it to fuel the American Dream, it has to be aligned with an economy that values the skills it imparts, and those skills must be translatable to tangible opportunities.”

One of the most interesting – and ominous – assertions the report makes is that the growing economic inequality in American society today is very likely to compound the disparity in skills. Students whose parents hold college degrees generally have greater skills in literacy and math than those whose parents have only high school diplomas. As the cost of college rises and economic inequality grows, those at the top will have great access to education and the opportunities it brings while those at the bottom will have fewer opportunities.

sample map from ETS survey“A very real danger lies in perpetuating a cycle where low skill levels, less income, and less access to quality education will beget a further entrenchment of deep inequality, with some segments of society more at risk than others,” the authors write.” This is the very opposite of what a meritocratic society purports to offer.”

For instance, scores of blacks and Hispanics were 15 to 24 percent below those of their white counterparts, depending upon the age group. As the report explains, though, whites and Asian-Americans still scored below the average for all those who took the tests internationally.

What can we do?

Tests of all kinds tend to do a much better job of measuring students’ ability to take tests than they do in measuring their ability to apply skills in a realistic setting. I’m also suspicious of “the sky is falling” reports about education. The sky has been falling since at least the 1970s. As I said earlier, though, I’ve had trouble finding fault with this study.

Perhaps the main weakness in the study is its lack of solutions. It makes a strong case that the skills of American millennials rank below those of their international counterparts. It doesn’t explain why or how to fix the problem, though. Here are some obvious ones:

De-emphasize standardized testing. Instead of fixating on standardized testing, give teachers the freedom to help students learn in meaningful ways. Many are doing that already with blended learning, project-based learning, team-based learning, and other methods. Until schools adopt a teacher evaluation system that uses standardized tests as a small part of a much larger portfolio, teachers will focus on test scores.

Make skills matter. Today’s academic culture promotes the idea of education as a product rather than a process of learning. Reversing that will require more meaningful demonstrations of skills over grades and a diploma. And until universities stop basing admissions decisions on standardized test scores, the ability to take those tests will matter far more than than more meaningful skills.

Emphasize active learning. As Maryellen Weimer writes in Faculty Focus: “We can’t seem to disavow ourselves of the notion that teachers should do most of the talking.” Active learning helps students gain crucial skills. Wide-scale of adoption of active learning is unlikely occur, though, until instructors are rewarded for taking risks.

Value teaching. Higher education still lacks a meaningful reward system for high-quality teaching. Until we change that, teaching will play a diminished role that results in diminished skills for students.

The problem is far more complex than any of those solutions, but those areas can at least provide an entry into a much-needed and, as the ETS report emphasizes, increasingly urgent problem.

Briefly …

Jonathan Rees, a professor of history at Colorado State, Pueblo, says that the university tenure system “is on life support” but deserves to be saved because it “is the best form of quality control that higher education has.” … The UK is struggling to recruit and retain teachers, investment in education has dropped, and student skills lag, The Guardian reports, saying, “Education has never mattered more, so why won’t the UK invest in it properly?” … Educause has joined other organizations in expressing concern about the discriminatory aspects of Indiana’s Religious Freedom Information Act. In an email message, the organization’s president and CEO, Diana Oblinger, said Educause had been looking for alternative sites to Indianapolis for its October conference and had written a letter to Indiana’s governor. … Three white papers issued by the U.S. Senate “show that lawmakers are considering significant changes in the ways colleges are evaluated and held accountable for student outcomes,” The Chronicle of Higher Education reports.


Doug Ward is an associate professor of journalism and the associate director of  the Center for Teaching Excellence. You can follow him on Twitter @kuediting.

By Doug Ward

Learning is a partnership, I tell students. As an instructor, I do my best to provide interesting and relevant material, use class time wisely, and grade student work fairly. I also make time in and out of class to help students better understand material they struggle with.

I can do only so much, though, I explain, and I certainly can’t make students learn. Learning takes place only when students engage themselves in their education, complete their work meaningfully, come to class prepared, and participate in discussions and projects.

It sounds simple, but it’s not, as a new study on undergraduate engagement makes abundantly clear.

The study, A Fresh Look at Student Engagement, surveyed freshmen and seniors on such things as higher-order learning, collaborative learning, student-faculty interaction and the supportiveness of their campus. In the foreword to the report, Paul E. Lingenfelter, former president of the State Higher Education Executive Officers Association, sums up the challenges well: “An authentic postsecondary education is more than simply acquiring knowledge; it must produce a sophisticated ability to use knowledge creatively to solve problems and add value.” (5)Pages from NSSE_2013_Annual_Results

That ability starts with engagement, and the report shows that those of us in higher education have lots of work to do on that front. Here are a few of the findings:

Collaborative learning

Only about a third of students reported engaging in collaborative learning, which was defined as working with other students on projects and assignments, explaining course material to others, or preparing for exams together. Those in engineering and biology reported the highest levels of collaborative learning; those in the arts, humanities, social sciences and social services reported the lowest levels.

Study time

Freshmen reported spending an average of 14 hours a week preparing for classes; seniors, 15 hours. Students taking all their courses online reported spending about an hour more studying each week. Seniors in engineering spent the most time studying (19 hours); those in communications, media and public relations, the least (12 hours).

Time spent reading

Students in online courses spent more time reading (eight hours a week for both freshmen and seniors) than students in traditional courses (six hours for freshmen, seven hours for seniors). A third of students reported doing no reading for class.

Amount of writing

Students in online courses did far more writing (59 pages a semester for freshmen, 107 for seniors) than students in traditional classes (45 pages for freshmen, 75 pages for seniors). Not surprisingly, the more reading and writing a course required, the more students perceived that they increased their ability to think in complex and critical ways.

Learning strategies

Students who used such learning strategies as identifying key elements of reading assignments, reviewing notes after class and summarizing what they had learned from a course generally reported higher grades than those who didn’t.

Effective teaching

Only about 40 percent of students at research universities reported that their instructors used effective teaching practices. The study defined those practices as having clear goals and organization, using clear examples to explain difficult concepts, and giving prompt and detailed feedback on assignments. Those in liberal arts reported the highest percentages of effective teaching; those in engineering, science and technology the lowest.

Interaction with faculty

Only about 20 percent of freshmen and seniors at large universities (those with more than 10,000 students) reported having meaningful interactions with faculty members. This included activities such as talking about career plans, meeting with faculty members outside class, and discussing academic performance.

Supportive environment

Older students, veterans and transfer students were less likely to find the campus environment supportive than other students were. Students who lived off-campus were less likely to describe campus as supportive than students who lived on campus were.

Online engagement

Students in online-only courses reported a higher level of interaction with instructors, advisers, student services staff members and administrators than students in traditional classes did. At the same time, they reported low levels of collaborative learning and were less likely to find the class environment supportive than those in face-to-face classes.

Use of technology

Students said courses that used technology for learning and that helped them understand the use of technology improved engagement and higher-order learning.

Adjustments to teaching

Assistant professors and lecturers were far more likely than associate and full professors to use course evaluations to improve their courses and their teaching.

Highimpact practices

This was the umbrella term the survey used to describe such things as learning communities, study abroad, internships, service learning and research with faculty members. Eighty-four percent of seniors reported engaging in at least one of these activities and 60 percent reported two or more. Fifty-eight percent of freshmen reported engaging in at least one, and 12 percent said they had done two. For seniors at research universities, the most common activity was an internship (53 percent), followed by service learning (52 percent). For freshmen, the most common activity was service learning (46 percent).

Student jobs and activities

Seniors at research universities averaged 14 hours a week working for pay, 12 hours relaxing and socializing, 5 hours participating in extracurricular activities, 5 hours commuting (including walking and driving to campus), 4 hours caring for dependents, and 3 hours doing community service.

The study was conducted by the National Survey of Student Engagement, an organization that focuses on undergraduate learning. Dan Bernstein, director of the Center for Teaching Excellence at KU, serves on the organization’s advisory board.

This year’s survey involved more than 1.6 million students at more than 600 colleges and universities. Unfortunately, KU was not among them.

I’d highly recommend looking more closely at the report, which provides a rich snapshot of student and faculty perceptions of higher education. Most certainly, the survey highlights many encouraging aspects of student engagement. As I said, though, we all have a ways to go.


Doug Ward is an associate professor of journalism and a fellow at the Center for Teaching Excellence. You can follow him on Twitter @kuediting.

By Doug Ward

Before you ban cellphones and laptops from your classroom, consider this: Students want to use those devices for learning and are looking to their instructors for guidance.

That’s one of the takeaway points of the latest study by the Educause Center for Analysis and Research on students and information technology. The center, known as ECAR, has conducted an annual survey of undergraduates since 2004, accumulating a wealth of data on students and technology. This year’s survey drew 113,000 responses from students at 251 colleges and universities.

Most of the key findings in this year’s study mesh with my own experiences with students. Among them:

PhotoDune
PhotoDune

Students value the social component of education. More than 40 percent of students have taken an online class, but most still prefer in-person classes that give them the opportunity to interact with instructors and fellow students. Older students who have jobs report a higher preference for online classes.

Students like classes that blend online and in-person components. They say that they not only prefer classes with a flipped, hybrid or blended approach but that they learn more in those classes.

Students want instructors to integrate technology into learning. Nearly three-fourths of respondents would like to have online access to recorded lectures, and slightly smaller percentages would like to see more use of learning management systems like Blackboard. They would also like to see more use of online collaboration tools.

Students’ ownership of technology is growing. Ninety percent of students own at least two Internet-capable devices, and 60 percent own three or more. This includes laptop computers (90 percent of students own one), smartphones (75 percent) and tablets or e-readers (48 percent). Students who own more technology are also more likely to see its potential for learning.

One of the most important points of the survey, I think, is that students want their instructors’ help in understanding how to use new software and new technology, and to use digital tools for learning. Not for texting. Not for watching YouTube videos. For learning.

Saying and doing don’t always go together, of course, and as the study points out, this is an ambiguous and often contentious issue. Students want to learn with technology, but most prefer to separate their social and academic lives. They value their privacy, and they express hesitation about software that tracks their academic work and gives them recommendations for course offerings. They also understand the distractions that technology can bring to the classroom, and many prefer to keep their phones stowed away. Others chafe at bans on technology in class, something that large percentages of instructors do, the survey says.

Despite the distractions that technology can bring, it is an integral part of students’ lives. It’s also an integral part of the learning equation. For instance, 60 percent or more of students say that technology helps them feel more connected to their college or university, and to their instructors. They also see it as important not only for their academic success but for their success in the workplace.

There’s no one size fits all approach to technology in the classroom, but it’s clear that we need to do a better job of harnessing the power of technology for what instructors and students say they want: more opportunities for learning.

Doug Ward is an associate professor of journalism and a fellow at the Center for Teaching Excellence. You can follow him on Twitter @kuediting.