By Doug Ward

Assessment often elicits groans from faculty members.

It doesn’t have to if it’s done right. And by right, I mean using it to measure learning that faculty members see as important, and then using those results to revise courses and curricula to improve student learning.

In a white paper for the organization Jobs for the Future, David T. Conley, a professor at the University of Oregon, points out many flaws that have cast suspicion on the value of assessment. He provides a short but fascinating historical review of assessment methods, followed by an excellent argument for a drastic change in the ways students are assessed in K-12. He also raises important issues for higher education. The report is titled A New Era for Educational Assessment.

Conley says that the United States has long favored consistency in measuring something in education over the ability to measure the right things. Schools, he says, “have treated literacy and numeracy as a collection of distinct, discrete pieces to be mastered, with little attention to students’ ability to put those pieces together or to apply them to other subject areas or real-world problems.”cover of a new era for educational assessment

One reason standardized testing has recently come under scrutiny, he says, is that new research on the brain has challenged assumptions about fixed intelligence. Rather, he says, researchers have come to an “understanding that intellectual capacities are varied and multi-dimensional and can be developed over time, if the brain is stimulated to do so.” Relatedly, they have found that attitudes toward learning are as important as aptitude.

The Common Core has also put pressure on states to find alternatives to the typical standardized test. The Core’s standards for college readiness include such elements as the ability to research and synthesize information, to develop and evaluate claims, and to explain, justify and critique mathematical reasoning – complex abilities that defy measurement with multiple-choice questions. Schools have been experimenting with other means to better measure sophisticated reasoning include, Conley writes. They include these:

  • Performance tasks that require students to parse texts of varying lengths and that may last from 20 minutes to two weeks. (KU’s Center for Education Testing & Evaluation has been working on one such test.)
  • Project-centered assessment, which gives students complex, open-ended problems to solve.
  • Portfolios, which collect a wide range of student work to demonstrate proficiency in a wide range of subjects.
  • Collaborative problem-solving, which sometimes involves students working through a series of online challenges with a digital avatar.
  • Metacognitive learning strategies, which Conley describes as ways “learners demonstrate awareness of their own thinking, then monitor and analyze their thinking and decision-making processes” and make adjustments when they are having trouble. Measuring these strategies often relies on self-reporting, something that has opened them to criticism.

Conley sees opportunities for states to combine several forms of assessment to provide a deeper, more nuanced portrait of learning. He calls this a “profile approach” and says it could be used not only by teachers and administrators but also colleges and potential employers. He asks, though, whether colleges and universities are ready to deal with these more complex measurements. Higher education has long relied on GPAs and test scores for deciding admissions, and more nuanced assessments would require more time to evaluate and compare. He says, though, that “the more innovative campuses and systems are already gearing up to make decisions more strategically and to learn how to use something more like a profile of readiness rather than just a cut score for eligibility.”

Conley raises another important issue for higher education. Over the past decade, high schools have focused on making students “college and career ready,” although definitions of those descriptions have been murky. Because of that, educators have “focused on students’ eligibility for college and not their readiness to succeed there.” Conley and others have identified key elements of college readiness, he says. Those include such things as hypothesizing and strategizing, analyzing and evaluating, linking ideas, organizing concepts, setting goals for learning, motivating oneself to learn, and managing time.

The takeaway? Assessment is moving in a more meaningful direction. That’s good news for both students and wary faculty members.


Doug Ward is an associate professor of journalism and the associate director of  the Center for Teaching Excellence. You can follow him on Twitter @kuediting.

Recent news, research, trends and thoughts about education. Compiled by Doug Ward.

How can we grade participation more effectively?

Maryellen Weimer argues that the way we usually grade participation in our classes doesn’t work. That is, many students still don’t join the conversations for fear of looking dumb. The typical grading approach also rewards quantity over quality. In an article in Faculty Focus, she writes about a colleague’s solution to this: using “extra-credit engagement tickets” students earn by completing assignments on time, joining online discussions, and submitting questions electronically. I agree with the logic, but the solution seems more complicated than I’m willing to attempt. This is definitely something that deserves more attention from most faculty members, though.

Chart for definitions of rigor
This pyramid shows one of several ways that the authors of “Developing a Student Conception of Academic Rigor” describe students’ perception of rigor.

What does ‘rigor’ in a course mean?

Students tend to equate a course’s rigor with workload rather than higher-order thinking, according to a study in the latest issue of Innovative Higher Education. That is, if a course required students to read or write more than they were used to, they generally considered it rigorous. The same held true if a student thought the professor’s grading standards were higher or if the student had little or no previous experience with the course content. This differs from faculty members’ idea of rigor, which the authors described as “learning meaningful content with higher-order thinking at the appropriate level of expectation within a given context.” The authors say that students and faculty need to understand these divergent views so that colleges and universities can better address the importance of academic standards. The study is called “Developing a Student Conception of Academic Rigor.”

 

Study questions value of degrees from for-profit online institutions

Research by the National Bureau of Economic Research strongly suggests that employers value degrees from public institutions over those from for-profit online institutions. The authors of a white paper posted on the bureau’s website found that candidates with bachelor’s degrees from for-profit online institutions were 20 percent less likely to receive a request for an interview or for additional information from employers than those with degrees from public institutions. (A 2012 employer survey conducted by The Chronicle of Higher Education found similar perceptions.) They say that given the high costs, degrees from those for-profit institutions “do not appear to be a sound investment.”

Briefly …

Meris Stansbury of eCampus News writes about the steps involved in a competency-based education program, and about the fluid definition of those types of programs. … An anonymous graduate student, writing in The Guardian, argues that colleges and universities need to more transparent about the time commitments of graduate teaching assistants. “Teaching is often regarded as something of an afterthought to doctoral completion, something fobbed off on those considered just competent enough,” the writer says.

Recent news, research, trends and thoughts about education. Compiled by Doug Ward.

Why a phone book isn’t a good learning tool

Daniel J. Klionsky of the Life Sciences Institute at the University of Michigan asks why so many instructors or programs continue to teach facts that students don’t need to know. In an article in Faculty Focus, he uses the telephone book as an example. No one needs to memorize all the numbers in a phone book. The idea is absurd. And yet, many instructors in science courses insist that students memorize facts they can easily look up, just as they would with a phone book. To help weed out the essential from the nonessential, he says that instructors should approach their courses with these questions:

  • How much of the information in our courses do the students really need to know?
  • How much time do we devote to making sure students know when they need a fact and how to look it up?
  • Do our students know what to do with the facts once they find them?

Dropout rates hit record lows

Pew Research reports that the high school dropout rates have reached a record low, 7 percent, continuing a decline that started in the mid-1990s. The dropout rate among Hispanics has declined by more than half since 1993, and the rate among blacks has been cut in half. Even with the declines, though, the number of high school dropouts is more than 2.2 million.

Those gaps that speak volumes

Matthew E. May writes about the creative power of empty space in attracting attention and intriguing audiences. His piece in the Harvard Business Review is aimed at marketers, but it applies equally as well to educators.

Digital technology for education 

Jane Hart has released her annual list of the top 100 tools for learning. The top of the list offers no surprises – Twitter, Google Drive, YouTube, PowerPoint – but the latter part is a good place to look for new tools you might try. It includes some that I’ve found useful, including Explain Everything and Powtoon.

The Chronicle of Higher Education reports on an app under development at Dartmouth that helps measure students’ mental health.

Recent news, research, trends and thoughts about education. Compiled by Doug Ward.

A focus on efficiency, for learning’s sake

The Evolllution began a series on operational efficiency at colleges and universities with an interview with Cathy Sandeen, vice president for educational attainment and efficiency at the American Council on Education. Sandeen lays out the right goals for cost efficiency, saying the process should aim at ways to help students learn and earn their degrees. “We need to work together to figure out how we can change and do things differently,” Sandeen says. “It’s not saying we’re doing things wrong in the past; it’s just how we can do things better in the future.”

During October, the Evolllution plans to tackle these questions:

  • What impact does operational efficiency have on institutional growth and competitiveness?
  • How can postsecondary leaders effectively communicate the value of efficiency-related changes to institution-wide stakeholders?
  • Do students notice when institutions are efficient?
  • How can institutions leverage technology tools to become more efficient?

Job skills vs. liberal education

Emmanuel Felton of The Hechinger Report writes about how City Colleges of Chicago remade themselves by aligning courses with the skills that were most in demand in the region. The change has helped City Colleges, a system of seven community colleges, improve abysmal graduation rates, at least somewhat. Felton interviewed several business leaders who espoused the benefits of tying education to skills that industry needs. One executive said higher education needed to take the same approach. That might serve some students and it would certainly serve industry in the short term. In the long term, though, we need a wide range of people who possess the critical-thinking and adaptive skills provided by a liberal education.small boy and adult sitting next to each other at desks

Keep an eye out for student mental health issues

Claire Shaw at The Guardian urges instructors to pay attention to students’ mental and physical state. She provides a list of warning signs that may indicate that students need assistance. She also offers several suggestions for instructors to help prepare themselves for struggling students.

A worldwide shortage of teachers

A new UNESCO report says that 93 countries face a chronic shortage of teachers and that many of them are recruiting people who lack any training in education.

An adaptive system that helps personalize learning

Stephen Downes of the National Research Council of Canada writes about beta testing of an adaptive learning system that combines customized materials from dozens of sources with data from learning, social environments and work environments. He calls this a “learning and performance support system.” Despite the stuffy name, it sounds fascinating.

Way cool: A tool for creating animated whiteboard videos

I’ve always been fascinated by animated whiteboard videos that show drawings being created in fast motion right before my eyes. I just came across a tool called VideoScribe that will help create those videos – for a price, of course. The software costs $198 a year. There’s a seven-day free trial, though. I’m going to have to give it a try. I’ll share my results.

By Doug Ward

There’s no shortage of ideas for remaking higher education.

Consider a few recent ones:

Margaret Rhodes at Wired is among the latest to report on ideas for remaking an antiquated educational format that rewards students for taking notes, memorizing facts, and then checking boxes on tests.

“Students don’t need information,” Rhodes writes. “They need to learn how to process and use it.”

Bravo!

Rhodes offers four ways to help higher education become more creative, based on ideas from the Stanford School of Design:

  • Revamp the timeframe. Substitute the four-year degree for a six-year program that allows students to move in an out as their needs change.
  • Eliminate class designations. Rather than designating students as freshmen, sophomores, juniors or seniors, let them range across the curriculum, learning and then applying and then learning something new.
  • Ditch the transcript. Rather than focusing on GPAs, focus on skill building and portfolio development.
  • Forget majors. Rather, have students declare goals or missions and let them take classes that help them meet those goals.
barn-raising
Thinking about education as a barn raising offers many possibilities as we move toward changing teaching and learning.

Cathy Davidson of City University of New York immediately added two other ideas to the list: eliminate tuition, and provide better pay for high-quality instructors.

Davidson is spot-on in her argument that radical changes will have little effect unless we’re willing to change the underlying problems. That is, we say we want high-quality education but still fail to provide the incentives and rewards that would make that happen.

Leonard Cassuto, a professor of English at Forham University, explains perhaps the central difficulty of elevating teaching in the research-driven culture of higher education. “It’s not that teaching doesn’t matter,” Cassuto writes, “but even many community colleges are looking for publication these days. It’s the only credential that crosses institutional boundaries, so it’s the easiest one for institutions to brag on.”

Cassuto is right, but even in higher education’s research-driven culture, we brush aside teaching as impossible to measure. So we send research packets out for external review when a professor goes up for tenure or promotion, but to evaluate teaching, we generally resort to student evaluations and observations of a single class. We shrug our shoulders and move on.

We can change that. The Provost’s Task Force on Course Redesign, of which I was a member, offered several solutions to improving teaching at KU. The solutions apply to every university, though. They aren’t particularly radical, and there’s nothing as eye-popping as the ones from the Stanford School of Design. Rather, the ideas are intended to help change the culture of teaching and the systemic problems that hold good teachers back. They include these:

  • Create community. We need to identify faculty who want to improve or change their courses and provide opportunities for them to network with similar-minded instructors to share ideas that will lead to additional change.
  • Encourage collaboration. This means within departments but also among departments and universities to share ideas and approaches to improving education. It also applies to faculty members who teach different sections of the same course.
  • Provide support. We need to expand programs that provide support for faculty members interested in changing their courses. In our case, that includes the C21 Consortium and the teaching fellows program, both of which help faculty members and departments improve active learning.
  • Recognize and reward effective teaching. Until we truly reward innovative, high-quality teaching in the same way we reward innovative, high-quality research, we have little hope of wide-scale change.
  • Increase funds for remaking classrooms. Classrooms alone won’t change anything, but as I’ve written previously, classroom design can indeed improve student engagement and motivation.
  • Make better use of digital technology for learning. Good teaching starts with sound pedagogy, but digital technology provides the means for reaching students in new ways, making courses more engaging, and time-shifting assignments so we can make better use of class time to address areas where students struggle.

Whether radical or not-so radical, the ideas for improving higher education offer no magic powers. Rather, they provide blueprints we can follow and frameworks on which we can build.

Think of an old-fashioned barn-raising, which provides a lens for looking at many aspects of education. By joining our forces and applying our expertise, we can create something that none of us could accomplish individually. And yet, to effect change we need individuals to step up and join the community.


Doug Ward is an associate professor of journalism and the associate director of  the Center for Teaching Excellence. You can follow him on Twitter @kuediting.

Recent news, research, trends and thoughts about education. Compiled by Doug Ward.

Notes by hand or with laptops? Sorry, wrong question.

Cathy Davidson raises exactly the right question in the debate about whether students should take notes by hand or with laptops in class. The real issue, Davidson writes, is that instructors should be working to avoid lecture and instead engage students in active learning. Even in a large lecture hall, instructors can use active-learning activities that help students learn far more than they would with lecture. Davidson’s suggestion doesn’t involve digital technology. Rather, she says, a simple notecard will do.

Tom Whitby, in Edutopia, reinforces Davidson’s argument by explaining the importance of collaboration in modern pedagogy.

If textbooks are dead, are universities next?

Educause, an orgranization that focuses on information technology in higher education, held its annual conference this week in Orlando, Fla. Most of the discussions were geared more toward IT than educators, but a few interesting nuggets caught my attention:educause logo

Helping students take control of class discussions

In an article in Hybrid Pedagogy, Chris Friend shares some techniques for letting class conversations evolve organically. He writes, “A class discussion where the teacher pre-determines the outcome is just a lecture in disguise, dressed up to feel student-centered while still being instructor-directed.”

Making sure all group members pull their weight

Li-Shih Huang offers tips on making sure all members of a project group share in the workload. Those tips include designing projects so that students complete them in phases, allowing students to choose project topics that match their own interests, and helping students become better problem-solvers. You’ll find the full post at Faculty Focus.

Number of stay-at-home college students hits 20-year high

National Journal reports that the number of college students living at home has reached a 20-year high. Tuition increases and “an economy that still feels like a recession to many families” have played a role, National Journal writes, saying that the combination “may be turning more students into pragmatists.” That shift can make diversifying the student body more challenging.

What to do when a class has a wide range of skills

In an article on differentiated learning, Christina Yu offers suggestions on helping students with wide ranges of skills in the same class. She suggests avoiding a technique that is often recommended: having students who understand course material help those who don’t. I wish she had explored that area more. That approach can certainly help in some scenarios. Yu doesn’t dismiss it outright; rather, she includes it in a list of “what differentiated instruction is not.”

A trend worth watching

eSchool News reports that tablet use is growing increasingly common in grades 4 through 12. School tablet use has reached 66 percent in grades 4-5, 58 percent in grades 6-8, and 42 percent in grades 9-12, the publication reports. Moreover, 81 percent of students say that tablets help personalize learning. These are students who will be in college in the coming years.

Recent news, research, trends and thoughts about education. Compiled by Doug Ward.

That pricey fifth (or sixth) year of college

Jon Marcus of the Hechinger Report writes about the overlooked cost of a fifth or sixth year in calculating the cost of a college education. Ninety percent of freshmen begin college thinking they will graduate in four years, though less than half actually do. … Also, in a disturbing trend, Hechinger reports that the number of homeless students in U.S. public schools has grown 58 percent since 2007-08.

Some trends worth watching

In a report released last week, the consulting firm Michael Cohen Group identified several digital trends in education. The report, which was created in late spring, provides no real surprises yet highlights some of the issues that educators everywhere should pay attention to, including social media, open educational resources, massive open online courses, blended learning, flipped courses, gamification, integration of coding into courses, digital simulations, bring-your-own-device programs, assessment, big data, and adaptive learning.

man standing on dock at foggy lake
Todd Quackenbush, Unsplash

The biggest challenge in educational technology? Managing change.

In its most recent research report, the Center for Digital Education says that technology itself “is never the biggest hurdle” in a changing educational environment. The biggest challenge is managing the changes brought on by technology, including integration into curricula, development of effective personalized learning, and effective training for teachers and staff members who use technology. Above all, the center said, institutions need to help instructors, students and staff member “think about how technology can fundamentally turn old pedagogy on its head.”

A startling statistic from the report: Singapore spends $21,200 per student on education annually, compared with $2,500 in the United States.

The university of the future

U.S. News & World Report recently looked at the challenges that colleges and universities face amid changing demographics, rising costs, a hypercompetitive admissions process, and a growing adoption of online courses, among other things. As part of the report, called College of Tomorrow, U.S. News asked six university leaders to offer their thoughts on the future. They talked about the need for improving transparency, leveraging research, and continuing to challenge students with new opportunities.

The most pessimistic, or perhaps realistic, was Robert Mendenhall, president of Western Governors University. Mendenhall said higher education in the future would probably look a lot like it does today, given the resistance to change on college campuses. He sees a need for change, pointing to areas like accountability and the growth in nontraditional students, and says that colleges that don’t may not survive.

Shirley Ann Jackson, president of Rensselaer Polytechnic, touched on areas I see as among the most important for the future. She wrote, “Students need to acquire new skills for this digitally interconnected environment, including the ability to ‘translate’ between and among disciplines and sectors. They must learn to operate effectively and ethically in virtual communities, immersive environments, and in blended worlds.”

How millennial are you?

Finally, educators need to do a better job of understanding all their students, including Generation Z, but Pew Research republished a four-year-old but still relevant quiz last week about understanding millennials. It’s worth a look.


Doug Ward is an associate professor of journalism and the associate director of  the Center for Teaching Excellence. You can follow him on Twitter @kuediting.

By Doug Ward

At a meeting of the CTE faculty ambassadors last week, Felix Meschke brought up a challenge almost every instructor faces.

Meschke, an assistant professor of finance, explained that he had invited industry professionals to visit his class last semester and was struck by how engaged students were. They asked good questions, soaked up advice from the professionals, and displayed an affinity for sharing ideas with speakers from outside the university.

The interaction was marvelous to watch, Meschke asked, but how could he assess it? He could ask a question on an exam, he said, but that didn’t seem right. The content of the discussions wasn’t as important as the discussions themselves and the opportunities those discussions brought to students.

In a sense, Meschke had answered his own question: His observations were a form of assessment. I suggested that he log those observations so he could provide documentation if he needed it. No, that wouldn’t provide a numerical assessment, but it would provide the kind of assessment he needed to make decisions on whether to do something similar in the future.

Paint pots and letter blocks
WordPerfect

All too often we think of assessment as something we do for someone else: for administrators, for accreditors, for legislators. Assessment is really something we need to do for ourselves, though. Thinking of it that way led to an epiphany for me a few years ago. Like so many educators, I approached assessment with a sense of dread. It was one more thing I didn’t have time for.

When I started thinking of assessment as something that helped me, though, it didn’t seem nearly so onerous. I want to know how students are doing in my classes so I can adapt and help them learn better. I want to know whether to invite back guest speakers. I want to know whether to repeat an assignment or project. I want to know what students report about their own learning. All of those things are natural parts of the teaching process.

That sort of thinking also helped me to realize that assessment doesn’t have to be quantitative. Assessments like quiz and exam grades can indeed point to strengths and weaknesses. If a large majority of students fails an exam, we have to ask why. Was there a problem in the way students learned a particular concept? A flaw in the wording of the exam? A lack of studying by students?

I rarely give exams, though. Rather, I use things like projects, journals and participation.

I use rubrics to grade the projects and journals, but the numbers don’t tell me nearly as much as the substance of the work. Only through a qualitative assessment do I get a sense of what students gained, what they didn’t gain, and what I need to rethink in future semesters.

In the class Meschke described, students applied their learning through active participation. Trying to put a numerical value on that would in some ways cheapen the engagement the students showed and the opportunities they gained in interacting with professionals. Observing those interactions provided excellent feedback to Meschke, though, and by writing a brief summary of his those observations, he could provide documentation for others.

The message was clear: Do it again next semester.

And when it comes to assessment, the message is clear, as well: Do it for yourself.

Additional resources

Portfolio Assessment: An Alternative to Traditional Performance Evaluation Methods in the Area Studies Programs, by Mariya Omelicheva

Assessment Resources for departments and programs at KU

Combining Live Performance and Traditional Assessments to Document Learning, by the School of Pharmacy


Doug Ward is an associate professor of journalism and the associate director of  the Center for Teaching Excellence. You can follow him on Twitter @kuediting.