By Doug Ward

Matthew Ohland talks confidently about the best ways to form student teams.

In a gregarious baritone punctuated by frequent, genuine laughs, he freely shares the wisdom he has gained from leading development of a team creation tool called CATME and from studying the dynamics of teams for more than two decades.

Ohland, a professor of engineering education at Purdue, visited KU recently and spoke with faculty members about the challenges of creating student teams and about the benefits of CATME, which has many devotees at KU. The tool, which launched in 2005, is used at more than 1,300 schools worldwide and has led to a long string of research papers for Ohland and others who have worked on the CATME project.

Matt Ohland, in a blue shirt, kneels at a table as he explains his work
Matt Ohland explains some of the thinking behind the team creation tool CATME

I asked Ohland the question that faculty members often ask me: What are the most important characteristics of a good team? Without hesitation, he offered something that surprised me but that made perfect sense:

“Of all the things you can choose about team formation, schedule is by far the most important,” he said.

That is, if you want students to work together outside class, their schedules must be similar enough that they can find time to meet. If they do all the work in class, the schedule component loses its importance, though.

Before he delves deeper into group characteristics, he offers another nugget of wisdom:

“What you start with in terms of formation is much less important than how you manage the teams once they are formed.”

That is, instructors must monitor a team’s interpersonal dynamics as well as the quality of its work. Is someone feeling excluded or undervalued? Is one person trying to dominate? Are personalities clashing? Are a couple of people doing the bulk of the work? Is a lazy team member irritating others or creating barriers to getting work done?

Whatever the problem, Ohland said, an instructor must act quickly. Sometimes that means pulling a problem team member aside and providing a blunt assessment. Sometimes it means having a conversation with the full team about the best ways to work together.

“Anything – anything – that is going wrong with a team dynamically, the only way to really fix it is face-to-face interaction,” he said.

Delving into team characteristics

In faculty workshops, Ohland delved deeper into the nuances of team formation, asking participants to provide characteristics to consider when creating teams. Among them were these:

  • Demographics
  • Traditional vs. nontraditional student
  • Academic level
  • Gender
  • Ethnicity
  • High performer vs. low performer
  • Interest in the class or the subject
  • Confidence
  • Work styles (work ahead vs. work at last minute)

All of those things can influence team dynamics, he said, and the most consequential are those that lead to a feeling of “otherness.” For instance, putting one woman on a team of men generally makes it difficult for the woman to have her voice heard. Putting a black student on a team in which everyone else is white can have the same effect, as can putting an international student on a team of American students.

“If students have a way of knowing that someone is different, it allows them a way to push them away,” Ohland said. “It’s their otherness that excludes them from certain kinds of team interaction. It’s their otherness that lets people interrupt them.”

Ohland also shared an illustration of an iceberg to represent visible and invisible characteristics of identity. It’s an illustration he uses to help students understand the diverse characteristics of team membership. Gender, race, age, physical attributes and language are among those most noticeable to others. Below the surface are things like thought processes, sexual orientation, life experience, beliefs and perspectives. Awareness of those characteristics helps team members recognize the many facets of diversity and the complexity of individual and team interaction.

Pushback from students

Jennifer Roberts, a professor of geology, uses CATME to form teams in her classes. She said that some students had begun to push back against providing race and gender in the CATME surveys they complete for team formation.

“They went so far as to say that this disenfranchises me because I don’t fit in these categories,” Roberts said.

Ohland said that he understood but that “ignoring race and gender in groups has real consequences.” He suggested explaining the approach to female students this way:

“What I tell my students is that I’m not putting you on a team with another woman so that you will be more successful,” he said. “I am putting on a team with another woman because it changes the way that men behave.”

He cited research that shows that putting more than one woman or more than one person of color on a team improves the performance of everyone by cutting down on feelings of isolation and allowing more views to be heard.

“Men stop interrupting them,” he said. “They start paying attention to their ideas.”

At Purdue, Ohland said, he goes as far as keeping freshmen together on teams in first-year engineering classes, separating them from transfer students who are sophomores or juniors.

“We’ve got to get them by themselves,” Ohland said. “They are at a different phase in life. They’re at a different place academically.”

Preparing students for teams

Ohland said it was important to help prepare students to work effectively in teams. His students go through several steps to do that, including watching a series of videos, engaging in class discussions about how good teams work, reviewing guidelines that team members need to follow, and learning about ways to overcome problems. They also agree to follow a Code of Cooperation, which stresses communication, cooperation, responsibility, efficiency and creativity.

He also explains to students how a student-centered class works, how that approach helps them learn, and what they need to do to make it successful. In a student-centered class, an instructor guides rather than leads the learning process, and students help guide learning, apply concepts rather than just hear about them, reflect on their work and provide feedback to peers.

Students must also understand the system they will use to rate peers, Ohland says, and he spends time going over that system in class. It includes measures on how students are contributing to a team, how they are interacting with teammates, how each member works to keep the team on track, how to evaluate the work quality of teammates, and how to evaluate teammates’ knowledge, skills and abilities.

The ins and outs of teams

It would be impossible to detail all of the advice Ohland offered. I would suggest visiting the CATME informational page, where you will find additional information and research about forming and evaluating groups, and keeping them on track. A few other things from Ohland are worth mentioning, though, largely because they come up in many discussions about using teams in classes.

Don’t force differentiation in evaluations. I have been guilty of this, trying to push students to create more nuance in their evaluations of teammates. Ohland said this creates false differentiations that frustrate students and lead to less-useful evaluations.

Learn what ratings mean. For instance, if team members give one another perfect scores, it could mean they are working well together and want the instructor to leave them alone. It could mean that students didn’t take the time to fill out the evaluations properly or it could mean that students felt uncomfortable ranking their peers. In that last scenario, Ohland sits down with a team and explains why it is important to provide meaningful feedback. If they don’t, individuals and the team as a whole lack opportunities to improve.

“That seems to help get them think about the value of the exercise,” Ohland said. “It gets that discussion going about why are we doing this and why it’s important not to just say everybody’s perfect.”

Keep the same teams (usually). Changing teams during a semester can create problems, he said, because high-functioning teams don’t want to disband and teams that are making progress need more time to work through kinks. Only the dysfunctional teams want to change, he said. The best approach is to find those dysfunctional teams and help them get on track.

The one exception to that guideline, he said, is when learning to form teams effectively is part of a class’s goals. In that case, an instructor should form teams more than once so that students get practice.

Evaluate teams frequently. Ohland recommends having peer evaluations every two weeks. Research shows that evaluations should coincide with a “major deliverable,” he said. That makes students accountable and increases the stakes of evaluations so that students take them seriously.

Create the right team size. In some cases, that may mean three or four. In others, six, eight, 10 or even more.

“Team size depends on what you are asking students to do,” Ohland said. “The critical thing about team size is that you need enough people on a team to get the work done that you are asking them to do – the quantity of work. You also need enough people on a team to have all the skills necessary to do the work represented.”

It also depends on the layout of a room. For instance, a team of three in a lecture hall is ideal because students can have easy conversations. A group of four in the same setting will leave one member of the team excluded from conversations.

A final thought

Research by Ohland and others has helped us better understand many aspects of effective student teams. I asked Ohland whether those components mesh with what students look for in teammates.

Making that connection, he said, “is the holy grail of teamwork research.”

“It’s so difficult to get an absolute measure of performance,” he said. “If our goal is learning, that’s a different goal than a competitive, objectively measured outcome in a project.”

Some data point to a connection between learning and team performance, but proving that is a work in progress.

“We’re getting there,” Ohland said.


Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

By Doug Ward

A new grant-funded initiative at the University of Kansas will promote the use of data to improve teaching, student learning and retention in science, engineering, technology and math programs.

KU is one of 12 universities to receive a $20,000 grant from the Association of American Universities as part of a major AAU project to improve STEM education. The grant will be used to promote faculty-led course and curricular changes that enhance student learning among undergraduates, and to help eliminate long-standing achievement gaps for students from underserved groups.AAU logo

The KU initiative will be led by an interdisciplinary team that includes Andrea Greenhoot, a professor of psychology and the director of the Center for Teaching Excellence; Caroline Bennett, an associate professor of engineering; Mark Mort, an associate professor of ecology and evolutionary biology; DeAngela Burns, vice provost for undergraduate studies; and Doug Ward, associate professor of journalism and associate director of the Center for Teaching Excellence.

“We see this as an important evolution in teaching and learning at KU,” said Greenhoot, who also leads a multi-university course-improvement program called TRESTLE, which is funded with a $2.5 million grant from the National Science Foundation. “Many KU faculty have been actively working to integrate evidence-based practices into their classes. Institutional data will help us gauge whether these efforts are helping students be more successful in later courses, and in completing their degrees.”

The new initiative will build on another project that CTE began last year. That project, part of a multi-university partnership known as the Bay View Alliance, is intended to help faculty members and departments use data to better understand student learning and student success, and to align with university goals of increasing retention and graduation rates.

Both initiatives aim to answer such questions as these:

  • How well are entry-level courses preparing students for later courses in a program sequence?
  • Are redesigns of such courses leading to better preparation and higher rates of success in later courses?
  • Are there inequities in student achievement and success for students from underserved or other groups? How effective are our efforts to reduce such gaps?

The AAU initiative at KU will begin later this semester with a goal of including 10 STEM departments in discussions about how to use institutional data to inform course and curricular improvements that can foster better student learning and improved degree completion. Administrators and deans already have access to this type of data, Burns-Wallace said, but many universities are extending access to faculty as they work to improve student success.

“This is a great opportunity to incorporate faculty into a wider conversation,” Burns-Wallace said. “Student success is a shared responsibility, and this grant will help STEM faculty understand how their course and curricular transformations have an even broader impact on overall student progress at KU.”


Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

By Doug Ward

Consider a few of the changes roiling public higher education.

Technology has created new ways for students to learn and to earn credentials but has also eliminated the need for a physical presence in many courses. Competency-based models have elevated the importance of work and life experiences in learning. Declining state support has pushed tuition costs increasingly higher, leading to growing scrutiny of colleges and universities by families and legislators. Many recent graduates have even expressed doubts about whether college was worth the expense.

Ann Austin, a professor at Michigan State University who recently led the National Science Foundation’s Division of Undergraduate Education, says these types of changes are as significant as those that higher education experienced in the late 19th century. That’s when the land grant acts led to a vast increase in the number of public universities, helping give rise to technical training, science education, social sciences, medical schools, law schools and other professional schools. Also during that time, faculty and curricula began to specialize, but students also gained the right to choose electives. Colleges for blacks and women also took root. By the early 20th century, higher education operated much differently than it did just a few decades earlier.

ann austin photo
Ann Austin

Modern institutions are only beginning to come to terms with the changes that lie ahead, and can really only guess at how those changes might reshape education.

“There’s a big question about what higher education will look like in the coming years,” Austin said.

Austin does a lot of thinking about change, which has been the focus of her research but also her work at NSF and the development of the Center for Integration of Research, Teaching, and Learning, which she co-founded.

Austin spoke recently at the semiannual meeting of the steering committee of the Bay View Alliance, a consortium of nine North American research universities (including KU) working to improve teaching and learning in higher education through cultural change. She drew from her extensive research into cultural and organizational change in higher education, especially in the areas of faculty development and the need to adapt the workplace, as she urged alliance members to think more strategically about the types of changes they are trying to make on their own campuses.

A changing landscape

Her views are especially important during this time of shifting ideas, perceptions and practices in higher education. Societal, legislative and financial forces are bearing down from the outside, providing opportunities for making much-needed changes from the inside, especially in the way we approach and value teaching.

Austin argues, though, that to do that we must not only analyze the problem we are trying to change, but examine it from many different angles and consider the issues that drive or impede change. Many times, she said, we jump into a change process but don’t identify the problem, the issues or the context. Nor do we consider how we would address the problem, even though “this is something we should be coming back to over and over again.”

In essence, she suggested that BVA members engage in change as they would a research project: Clarify a problem that needs to be addressed, gather information about that problem, analyze that information, provide context, and draw conclusions on how best to move forward.

Austin offered many provocative questions to illuminate the process she laid out, drilling down on the many facets of an institution that provide opportunities for or impediments to change:

  • Why is this issue a problem? What elements of the problem need to be addressed? What factors will affect the process of change?
  • Who owns the process of change and has access to data? Who gets recognition? What alliances do we need to form?
  • How do we maintain momentum and energy, especially as leadership changes?
  • How do we establish support mechanisms to aid the process in person and online?
  • Who has informal power, and how do we handle resistance?
  • How do we connect our efforts to institutional priorities?

We must consider these and many other questions if we hope to succeed, Austin said.

“If we want people to change, they have to know what to change and why they should change,” she said. “They also need to know that they won’t be penalized for doing so.”

A multi-university approach to change

BVA has approached change at many levels of university culture as it has worked to improve recognition of innovative teaching at research universities, to promote the use of active learning in large undergraduate classes, and to build community among faculty members so that they can share ideas and experiences that lead to improved student learning. Recent projects include use of embedded teaching experts to improve instruction, use of data analytics to better understand learning, and creation of new processes for evaluating teaching. Ultimately, it hopes to change attitudes toward teaching and the university culture that impedes innovative teaching.

Austin’s presentation came after a morning in which several BVA members raised concerns about the slow pace of change in higher education, saying that members must do a better job of explaining the value of change. Some said teaching centers needed to do more to move change from the grass-roots to the administrative level. Others wondered how they could tie the need for improving teaching to improving university finances. Still others expressed doubt that attitudes toward teaching had changed at all.

Mary Huber, a BVA advisor who is a senior scholar emerita at the Carnegie Foundation for the Advancement of Teaching, challenged that notion. Those of us who value teaching have indeed helped bring about change, she said.

“The conversations we are having today are much different from the ones we had 30 years,” Huber said. “We may have not had the magical transformative powers we had hoped, but what we have done has been hopeful.”

And if Austin is right and we are at the cusp of an enormous wave of change, we must continue to remain hopeful as we work to shape the future.


Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

By Doug Ward

The spread of evidence-based teaching practices highlights a growing paradox: Even as instructors work to evaluate student learning in creative, multidimensional ways, they themselves are generally judged only through student evaluations.

Students should have a voice. As Stephen Benton and William Cashin write in a broad review of research, student evaluations can help faculty members improve their courses and help administrators spot potential problems in the classroom.optical illusion box

The drawback is that too many departments use only student evaluations to judge the effectiveness of instructors, even as they submit faculty research through a multilayered evaluation process internally and externally. Student evaluations are the only university-mandated form of gauging instructors’ teaching, and many departments measure faculty members against a department mean. Those above the mean are generally viewed favorably and those below the mean are seen as a problem. That approach fails to account for the weaknesses in evaluations. For instance, Benton and Cashin and others have found:

  • Students tend to give higher scores to instructors in classes they are motivated to take, and in which they do well.
  • Instructors who teach large courses and entry-level courses tend to receive lower evaluations than those who teach smaller numbers of students and upper-level courses.
  • Evaluation scores tend to be higher in some disciplines (especially humanities) than in others (like STEM).
  • Evaluation scores sometimes drop in the first few semesters of a course redesigned for active learning.
  • Students have little experience in judging their own learning. As the Stanford professor Carl Wieman writes: “It is impossible for a student (or anyone else) to judge the effectiveness of an instructional practice except by comparing it with others that they have already experienced.”
  • Overemphasis on student evaluations often generates cynicism among faculty members about administrators’ belief in the importance of high-quality teaching.

Looked at through that lens, we have not only a need but an obligation to move beyond student evaluations in gauging the effectiveness of teaching. We simply must add dimension and nuance to the process, much as we already do with evaluation of research.

So how do we do that?

At CTE, we have developed a rubric to help departments integrate information from faculty members, peers, and students. Student evaluations are a part of the mix, but only a part. Rather, we have tried to help departments draw on the many facets of teaching into a format that provides a richer, fairer evaluation of instructor effectiveness without adding onerous time burdens to evaluators.

For the most part, this approach uses the types of materials that faculty members already submit and that departments gather independently: syllabi and course schedules; teaching statements; readings, worksheets and other course materials; assignments, projects, test results and other evidence of student learning; faculty reflections on student learning; peer evaluations from team teaching and class visits; and formal discussions about the faculty member’s approach to teaching.

Departments then use the rubric to evaluate that body of work, rewarding faculty members who engage in such approaches as:

  • experimenting with innovative teaching techniques
  • aligning course content with learning goals
  • making effective use of class time
  • using research-based teaching practices
  • engaging students in hands-on learning rather than simply delivering information to them
  • revising course content and design based on evidence and reflection
  • mentoring students, and providing evidence of student learning
  • sharing their work through presentations, scholarship, committee work and other venues

Departments can easily adapt the rubric to fit particular disciplinary expectations and to weight areas most meaningful to their discipline. We have already received feedback from many faculty members around the university. We’ve also asked a few departments to test the rubric as they evaluate faculty members for promotion and tenure, third-year review, and post-tenure review, and we plan to test it more broadly in the fall.

We will continue to refine the rubric based on the feedback we receive. Like teaching itself, it will be a constant work in progress. We see it as an important step toward making innovative teaching more visible, though, and toward making teaching a more credible and meaningful part of the promotion and tenure process. If you’d like to be part of that, let us know.

****

This article also appears in Teaching Matters, a publication of the Center for Teaching Excellence.


Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

By Doug Ward

Two recent education conferences I attended raised similar questions about developing and sustaining high-quality teaching. Things like:

  • How do we measure the success of course transformation?
  • How can we get buy-in from colleagues?
  • How do we gain the support of department chairs and administrators?
  • How do we share ideas among campuses?
  • How do we sustain and grow communities around the idea of improving teaching?

That last question was central to both conferences, one at KU and one at the University of California, Davis.

Participants in the Trestle launch meeting work in a brainstorming session
Participants in the Trestle launch meeting work in a brainstorming session at the KU Alumni Association. From left are Marsha McCartney (psychology), Chris Fischer (physics and astronomy), Dave Benson (chemistry), Natalie Caporale (University of California, Davis), and Sarah Bean (University of British Columbia).

The KU conference in late January helped launch a new CTE-led initiative called Trestle, or Transforming Education, Stimulating Teaching and Learning Excellence. CTE’s director, Andrea Greenhoot, leads the project, which is financed by a $2.5 million grant from the National Science Foundation. The goal is to spread the use of evidence-based teaching methods in science, engineering, technology, and math courses across several participating universities.

The conference at UC Davis was called TEA, or Tools for Evidence-based Action. Its focus was on documenting teaching, learning, and curricula, specifically through digital tools that provide data visualizations of student performance and classroom activities.

At both conferences, faculty members shared successes and failures in teaching, and talked candidly about the challenges we face in bringing more people into the fold. By building a community of engaged teachers, we hope to share our experiences in and out of the classroom, improve our approaches to teaching, emphasize the importance of learning, and shape our classes with evidence-based, learner-centered approaches.

Those are buzzwords, yes, but in essence they mean we need to reflect on our teaching and approach it in measured, meaningful ways. By building community, we can all find ways of doing that.

Submit a proposal for our first Teaching Slam

As a way to expand our community of engaged teachers, we are putting on a Teaching Slam.

A Teaching Slam is a fast-paced session in which speakers from many disciplines around KU share their best teaching tips, assessment ideas, or class activities. Instructors in all disciplines are welcome to submit proposals, and we’ll choose the best for presentation on Friday, March 25 from 2 to 3:30 p.m.

We are looking for proposals for two types of presentations:

  • Six + Six. You will have six minutes and six slides to share a useful class activity, an assessment idea, or a teaching tip.
  • Class Demo. You will have 10 to 15 minutes to lead your colleagues (who will act as your class) through a hands-on classroom activity. These sessions must be interactive.

Both types of presenters must provide a handout to help others use the activity immediately.

Deadline for proposals is noon Friday, March 4. We’ll choose the best and let everyone know the results by early the following week.

Interested? Submit a proposal and join our community.


Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.