| Bloom's Sixth

Cleve Moler meets with students and faculty members after speaking at Eaton Hall.

By Doug Ward

“Look over Spock’s shoulder,” Cleve Moler told the audience at Eaton Hall on Thursday.

He was showing a clip from Star Trek: The Motion Picture, the 1979 movie that revived the classic 1960s science fiction television series. In a scene from the bridge of the Starship Enterprise, colorful graphics blink on a futuristic array of monitors.

Those graphics, Moler explained, came from what is now the Los Alamos National Laboratory, where he was part of team that developed a computational math library and an accompanying library for computer graphics. That work, written in Fortran, was “just a matrix calculator,” Molder said, but it was the basis for Matlab, software that creates an interactive environment for computation, visualization, modeling, and programming.

Moler, a former professor at New Mexico, Michigan and Stanford, is the “chief mathematician” and chairman of MathWorks, a company he co-founded in 1984. He visited KU this week, speaking to an auditorium filled mostly with undergraduates, but also with faculty and administrators, for the Russell Bradt Undergraduate Colloquium.

Graphics from the Los Alamos National Laboratory were used in scenes from “Star Trek: The Motion Picture.”

I’m not a mathematician, so I won’t pretend to understand the intricacies of Matlab. What I do understand is that Matlab plays an important role in researching and teaching mathematics, engineering, and other STEM areas. It is an environment created by mathematicians for mathematicians, and it has attained near iconic status in the academic world, with more than a million users.

In his talk at Eaton Hall, Moler spoke in a gravelly but upbeat voice about his influences in mathematics and computer science and about the steps that led to the development of Matlab. MathWorks, the company, started in 1984 with one employee, doubling every year for the first seven years. It now has about 4,000 employees in 20 offices around the world, Moler said. It has also expanded into such areas as cell biology, image processing, hearing aids, and driverless cars.

About midway through his talk, Moler gleefully described a project called Eigenwalker, which is using Matlab to break the human gait into its mathematical components and using those calculations to create stick and dot animations. As half a dozen panels with stick figures walked in place on the screen, Moler grinned at the audience and said: “I enjoy that demo. Everybody enjoys that demo.”

Then his analytic side came out.

“It’s all very amusing,” he said, “but what do we see here that’s so enjoyable?”

Through those stick figures, he said, we can understand things like mood, gender, and personality just by observing the way they move, and researchers are using the animations to study how people perceive others through their walk.

At 78, Moler doesn’t move nearly as smoothly as he did when he created Matlab, but his status as a math star was apparent at a reception in Snow Hall, where he sat with a cup of coffee and a cranberry oatmeal cookie talking with Professor Marge Bayer and others from the math department. Graduate students ringed the room, seemingly reluctant to approach Moler. They needn’t have worried. Despite his genius, Moler loves interacting with people, telling stories of his family and of the evolution of Matlab.

He shared one of those stories at the end of his talk at Eaton Hall after a student asked about the “why” command in Matlab. The original Matlab used terminal input, he said, and provided answers for commands like “help,” “who,” and “what.” Moler and others decided that Matlab needed a “why” function to go along with the others, so they programmed it to respond with “R.T.F.M.” when someone typed “why.”

That stood for “read the manual,” he said, with an extra word in the middle starting with “f.”

The room erupted in laughter.

Over the years, the “why” function became an inside joke, an Easter egg in Matlab that provided random humorous answers. He gave his audience a sneak peak of 30 or so new responses, including “Some smart kid wanted it,” “To please some system manager,” “To fool a young tall hamster,” “Some mathematician suggested it,” and “How should I know?”

At the end of his talk, Moler made a pitch for his company, which he said was hiring 250 to 300 “good people who know Matlab.” It wasn’t quite the same as in invitation to the bridge of the Enterprise, but for young mathematicians, it was close.

An entrepreneur endorses the liberal arts

The liberal arts got a recent thumbs-up from an unlikely source: Mark Cuban, the entrepreneur, Shark Tank star, and owner of the Dallas Mavericks.

“Unlikely” may not be quite the right description because Cuban encourages high school students to attend college and has a college degree himself (Indiana University business administration, 1981). So he is hardly part of the drop-out-now-and-chase-your-dream crowd of Silicon Valley entrepreneurs like Peter Thiel. Neither is he bullish on higher education, though. He has argued that colleges and universities are constructing unnecessary buildings financed by rising tuition, and that higher education is in the midst of a bubble, much as real estate was before 2008.

“As far as the purpose of college, I am a huge believer that you go to college to learn how to learn,” Cuban wrote on his blog in 2012. “However, if that goal is subverted because traditional universities, public and private, charge so much to make that happen, I believe that system will collapse and there will be better alternatives created.”

In a recent interview with Bloomberg News, Cuban said that students needed to think carefully about jobs, careers, and skills.

“What looked like a great job graduating from college today may not be a great job graduating from college five years or 10 years from now,” he said.

That’s because machine learning and artificial intelligence are changing the nature of work. Companies are hiring fewer employees as technology takes over more jobs, making it crucial for people to understand how to use computers and software, he said.

“Either software works for you or you work for software, and once the software takes over, you’re gone,” Cuban said.

He predicted enormous changes in the workplace in the coming years.

“The amount of change we’re going to see over the next five years, 10 years will dwarf everything that’s happened over the last 30,” Cuban said.

Because of that, Cuban said he expected English, philosophy and foreign language majors and others who are “more of a freer thinker” to have a distinct advantage.

“I personally think there’s going to be a greater demand in 10 years for liberal arts majors than there were for programming majors and maybe even engineering,” Cuban said.

It’s refreshing to hear someone from the business world extoll the virtues of liberal education, especially as higher education – and liberal education in particular – have come under intense criticism from many sides. There is certainly much to criticize, but there is also much to be hopeful about. No matter their career path, students benefit from a broad understanding of the world, an ability to research effectively, communicate clearly and analyze critically, and a desire to keep learning. (I’ll be talking more about those skills in the coming weeks.)

Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

By Doug Ward

BLOOMINGTON, Indiana – We have largely been teaching in the dark.

By that, I mean that we know little about our students. Not really. Yes, we observe things about them and use class surveys to gather details about where they come from and why they take our classes. We get a sense of personality and interests. We may even glean a bit about their backgrounds.

That information, while helpful, lacks many crucial details that could help us shape our teaching and alert us to potential challenges as we move through the semester. It’s a clear case of not knowing what we don’t know.

Participants at a Learning Analytics Summit workshop grapple with definitions of student success and with how they might use data to better understand teaching and learning.

That became clear to me last week at the first Learning Analytics Summit at Indiana University. The summit drew more than 60 people from universities around the country to talk about how to make more effective use of academic data. I led workshops on getting started with data projects for analyzing courses, curricula, student learning and student success. As I listened and spoke with colleagues, though, I was struck by how little we know about our courses, curricula and students, and how much we stand to gain as we learn more.

Let me provide examples from the University of California-Davis and the University of New Mexico, two schools that have been piloting electronic systems that give instructors vast amounts of information about students before classes start.

Marco Molinaro, assistant vice provost for educational effectiveness at UC-Davis, showed examples of a new system that provides instructors with graphics-rich digital pages with such details as the male-female balance of a class, the number of first-generation students, the number of low-income students, the number of underrepresented minorities, the number of students for whom English is a second language, the number of students who are repeating a class, the most prevalent majors among students in a class, previous classes students have taken, other courses they are taking in the current semester, how many are using tutoring services, comparisons to previous classes the instructor has taught, and comparisons to other sections of the same class.

For privacy reasons, none of that data has names associated with it. It doesn’t need to. The goal isn’t to single out students; it’s to put information into the hands of faculty members so they can shape their classes and assignments to the needs of students.

That data can provide many insights, but Molinaro and his staff have gone further. In addition to tables and charts, they add links to materials about how to help different types of students succeed. An instructor who has a large number of first-generation students, for instance, receives links to summaries of research about first-generation students, advice on teaching strategies that help those students learn, and an annotated bibliography that allows the instructor to go deeper into the literature.

Additionally, Molinaro and his colleagues have begun creating communities of instructors with expertise in such areas as working with first-generation students, international students, and low-income students. They have also raised awareness about tutoring centers and similar resources that students might be using or might benefit from.

Molinaro’s project is funded by a $1 million grant from the Howard Hughes Medical Institute. It began last fall with about 20 faculty members piloting the system. By the coming fall, Molinaro hopes to open the system to 200 to 300 more instructors. Eventually, it will be made available to the entire faculty.

Embracing a ‘cycle of progress’

Providing the data is just the first step in a process that Molinaro calls a “cycle of progress.” It starts with awareness, which provides the raw material for better understanding. After instructors and administrators gain that understanding, they can take action. The final step is reflection, which allows all those involved an opportunity to evaluate how things work – or don’t work – and make necessary changes. Then the cycle starts over.

“This has to go on continuously at our campuses,” Molinaro said.

As Molinaro and other speakers said, though, the process has to proceed thoughtfully.

For instance, Greg Heileman, associate provost for student and academic life at the University of Kentucky, warned attendees about the tendency to chase after every new analytics tool, especially as vendors make exaggerated claims about what their tools can do. Heileman offered this satiric example:

First, a story appears in The Chronicle of Higher Education.

“Big State University Improves Graduation Rates by Training Advisors as Mimes!”

The next day, Heileman receives email from an administrator. The mime article is attached and the administrator asks what Heileman’s office is doing about training advisors to be mimes. The next day, he receives more email from other administrators asking why no one at their university had thought of this and how soon he can get a similar program up and running.

The example demonstrates the pressure that universities feel to replicate the success of peer institutions, Heileman said, especially as they are being asked to increase access and equity, improve graduation rates, and reduce costs. On top of that, most university presidents, chancellors and provosts have relatively short tenures, so they pressure their colleagues to show quick results. Vendors have latched onto that, creating what Heileman called an “analytics stampede.”

Chris Fischer, associate professor of physics and astronomy at KU, speaks during a poster session at the analytics conference in Bloomington, Indiana.

The biggest problem with that approach, Heileman said, is that local conditions shape student success. What works well at one university may not work well at another.

That’s where analytics can play an important role. As the vice provost for teaching and learning at the University of New Mexico until last fall, Heileman oversaw several projects that relied on university analytics. One, in which the university looked at curricula as data for analysis, led to development of an app that allows students to explore majors and to see the types of subjects they would study and classes they would take en route to a degree. That project also led to development of a website for analysis and mapping of department curricula.

One metric that emerged from that project is a “blocking factor,” which Heileman described as a ranking system that shows the likelihood that a course will block students’ progression to graduation. For instance, a course like calculus has a high blocking factor because students must pass it before they can proceed to engineering, physics and other majors.

Better understanding what classes slow students’ movement through a curriculum allows faculty and administrators to look more closely at individual courses and find ways of reducing barriers. At New Mexico, he said, troubles in calculus were keeping engineering students from enrolling in several other classes. The order of classes also created complexity that made graduation more difficult. By shifting some courses, students began taking calculus later in the curriculum. That made it more relevant – and thus more likely that students would pass – and helped clear a bottleneck in the curriculum.

Used thoughtfully, Heileman said, data tells a story and allows us to formulate effective strategies.

Focusing on retention and graduation

Dennis Groth, vice provost for undergraduate education at Indiana, emphasized the importance of university analytics in improving retention and graduation rates.

Data, he said, can point to “signs of worry” about students and prompt instructors, staff members and administrators to take action. For instance, Indiana has learned that failure to register for spring classes by Thanksgiving often means that students won’t be returning to the university. Knowing that allows staff members to reach out to students sooner and eliminate barriers that might keep them from graduating.

Data can also help administrators better understand student behavior and student pathways to degrees. Many students come to the university with a major in mind, Groth said, but after taking their first class in that major, they “scatter to the wind.” Many find they simply don’t like the subject matter and can’t see themselves sticking with it for life. Many others, though, find that introductory classes are poorly taught. As a result, they search elsewhere for a major.

“If departments handled their pre-majors like majors,” Groth said, “they’d have a lot more majors.”

Once students give up on what Groth called “aspirational majors,” they move on to “discovery majors,” or areas they learn about through word of mouth, through advisors, or through taking a class they like. At Indiana, the top discovery majors are public management, informatics and psychology.

“Any student could be your major,” Groth said. That doesn’t mean departments should be totally customer-oriented, he said, “but students are carried along by excitement.”

“If your first class is a weed-out class, that chases people away,” Groth said.

Indiana has also made a considerable amount of data available to students. Course evaluations are all easily accessible to students. So are grade distributions for individual classes and instructors. That data empowers students to make better decisions about the majors they choose and the courses they take, he said. Contrary to widespread belief, he said, a majority of students recommend nearly every class. Students are more enthusiastic about some courses, he said, but they generally provide responsible evaluations.

In terms of curriculum, Groth said universities needed to take a close look at whether some high-impact practices were really having a substantial impact. At Indiana, he said, the data are showing that learning communities haven’t led to substantial improvements in retention or in student learning. They aren’t having any negative effects, he said, but they aren’t showing the types of results that deserve major financial support from the university.

As more people delve into university data, even the terms used are being re-evaluated.

George Rehrey, director of Indiana’s recently created Center for Learning Analytics and Student Success, urged participants to rethink the use of the buzzword “data-driven.” That term suggests that we follow data blindly, he said. We don’t, or at least we shouldn’t. Instead, Rehrey suggested the term “data-informed,” which he said better reflected a goal of using data to solve problems and generate ideas, not send people off mindlessly.

Lauren Robel, the provost at Indiana, opened the conference with a bullish assessment of university analytics. Analytics, she said, has “changed the conversation about student learning and student success.”

“We can use this to change human lives,” Robel said. “We can literally change the world.”

I’m not ready to go that far. University analytics offer great potential. But for now, I’m simply looking for them to shed some light on teaching and learning.

Data efforts at KU

KU has several data-related projects in progress. STEM Analytics Teams, a CTE project backed by a grant from the Association of American Universities, have been drawing on university data to better understand students, programs and progression through curricula. The university’s Visual Analytics system makes a considerable amount of data available through a web portal. And the recently created Business Intelligence Center is working to develop a data warehouse, which will initially focus on financial information but will eventually expand to such areas as curriculum, student success and other aspects of academic life. In addition, Josh Potter, the documenting learning specialist at CTE, has been working with departments to analyze curricula and map student pathways to graduation.

Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

By Doug Ward

A peer review of teaching generally goes something like this:

An instructor nears third-year review or promotion. At the request of the promotion and tenure committee, colleagues who have never visited the instructor’s class hurriedly sign up for a single visit. Sometimes individually, sometimes en masse, they sit uncomfortably among wary students for 50 or 75 minutes. Some take notes. Others don’t. Soon after, they submit laudatory remarks about the instructor’s teaching, relieved that they won’t have to visit again for a few years.

ChangHwan Kim (left), Tracey LaPierre and Paul Stock discuss their plans for evaluating teaching in the sociology department. They gathered with faculty members from four other units at the inaugural meeting of the Benchmarks for Teaching Effectiveness Project.

If your department or school has a better system, consider yourself lucky. Most peer evaluations lack guidelines that might offer meaningful feedback for a candidate and a P&T committee, and they focus almost exclusively on classroom performance. They provide a snapshot at best, lacking context about the class, the students or the work that has gone into creating engagement, assignments, evaluations and, above all, learning. Academics often refer to that approach as a “drive-by evaluation,” as reviewers do little but breeze past a class and give a thumbs-up out the window.

Those peer evaluations don’t have to be a clumsy, awkward and vapid free-for-all, though. Through the Benchmarks for Teaching Effectiveness Project, we have begun a process intended to make the evaluation of teaching much richer and more meaningful. The project is financed through a five-year, $612,000 National Science Foundation grant and is part of a larger NSF project that includes the University of Colorado, Michigan State, and the University of Massachusetts, Amherst.

We have used the NSF grant to distribute mini-grants to four departments and one school that will pilot the use of a rubric intended to add dimension and guidance to the evaluation of teaching. Faculty members in those units will work with colleagues to define and identify the elements of good teaching in their discipline, decide on appropriate evidence, adapt the rubric, apply it in some way, and share experiences with colleagues inside and outside the department and the university. Evidence will come from three sources: the instructor, students and peers, with departments deciding how to weight the evidence and to weight the categories in the rubric.

Not surprisingly, the instructors involved in the project had many questions about how the process might play out as they gathered for the first time in February: What types of evidence are most reliable? How do we reduce conscious or unconscious bias in the evaluation process? How do we gain consensus among colleagues for an expanded evaluation process and for application of a new system of evaluation? How can we create a more meaningful process that doesn’t eat up lots of time?

Those are important questions without simple answers, but the departments that have signed on in this initial stage of the project have already identified many worthy goals. For instance, Sociology, Philosophy and Biology hope to reduce bias and improve consistency in the evaluation process. Chemical and Petroleum Engineering plans to create triads of faculty members who will provide feedback to one another. Public Affairs and Administration sees opportunities for enriching the enjoyment of teaching and for inspiring instructors to take risks to innovate teaching.

All the units will use the rubric to foster discussion among their colleagues, to identify trustworthy standards of evidence, and, ultimately, to evaluate peers. Philosophy and sociology see opportunities for better evaluating graduate teaching assistants, as well. Chemical and Petroleum Engineering hopes to use the rubric to guide and evaluate 10 faculty members on tenure track. Sociology plans to use it to guide peer evaluation of teaching. Public Affairs and Administration plans to have a group of faculty alternate between evaluator and evaluee as they hone aspects of the rubric. Biology plans to explore the best ways to interpret the results.

That range of activities is important. By using the rubric to foster discussion about the central elements of teaching – and its evaluation – and then testing it in a variety of circumstances, instructors will learn valuable information about the teaching process. That feedback will allow us to revise the rubric, create better guidelines for its use, and ultimately help as many departments as possible adopt it for the promotion and tenure process.

All of the faculty members working in the initial phase of the Benchmarks project recognize the complexity and challenge of high-quality teaching. They also recognize the challenges in creating a better system of evaluation. Ultimately, though, their work has the potential to make good teaching more transparent, to make the evaluation of teaching more nuanced, and to make teaching itself a more important part of the faculty evaluation process.

Work your way through college? Not anymore

Kansas students would need to work nearly 30 hours a week at minimum wage to pay for college, even if they received grants and scholarships, according to an analysis by the public policy organization Demos.

In only eight other states would students need to work more hours to pay for college. New Hampshire, which would require more than 41 hours of work a week, was No. 1, followed by Pennsylvania (39.8 hours) and Alabama (36 hours).

Students attending college in Washington State would need to work the fewest hours (11.6), followed by California (12.6) and New York (15).

“In the vast majority of states, the idea of working your way through college is no more than an antiquated myth,” Demos writes. “A combination of low minimum wages and high college prices make borrowing an inevitability for students.”

The average yearly cost of attending Kansas universities is $16,783, Demos says. That’s 86 percent higher than it was in 2001, putting Kansas at No. 32 in average cost of attendance for public universities. New Hampshire had the highest average cost ($26,008), followed by Vermont ($25,910) and New Jersey ($25,544). Utah ($13,344) had the lowest average cost, followed by Wyoming ($13,942) and Idaho ($14,211).

Demos, which tilts liberal in its ideology, calculated the rankings using data from the federal government’s Integrated Postsecondary Education Data System and the Department of Labor. It created a “net price” for each state by subtracting average scholarship and grant aid from the average tuition and fees for four-year colleges in each state.

That approach has many flaws. In Kansas, for instance, tuition and fees vary widely among four-year universities and even within schools at those universities. Averaging also masks a wide variance in the amount of financial aid students receive. Looking only at cost of attendance skews the picture even further, as housing, food and other expenses generally exceed the cost of tuition and fees, especially in the Northeast and West Coast.

Even so, the study offers a reality check about college costs. State investment in higher education has declined even as the number of students attending college, and the diversity among those students, has grown. In Kansas, tuition now covers an average of 53 percent of a university’s costs, compared with 28 percent in 2001. Even that looks good compared with states like New Hampshire, where tuition accounts for 79 percent of university revenue, Delaware (75 percent) and Pennsylvania (73 percent).

Then again, in Wyoming, tuition dollars account for only 13 percent of college budgets. That is considerable lower than the states that follow: California (21 percent) and Alaska (30 percent) . In all states but Wyoming, tuition dollars now account for a greater share of university budgets that they did in 2001.

As Demos writes, “our state and federal policymakers have been vacating the compact with students that previous generations enjoyed.” It’s no wonder students have sought to put political pressure on schools and legislators.

The disinvestment in higher education began in the 1970s as a political message of lower taxes and smaller government started gaining ground. It accelerated during economic downturns and has only recently begun to ease. To compensate, colleges and universities have cut staff, hired fewer tenure-track professors, increased class size, and relied increasingly on low-paid adjunct instructors for teaching. Students and their families have taken on larger amounts of debt to finance their education.

As Demos writes: “When states do not prioritize higher education as a public good, students and families generally bear the burden.”

Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

By Doug Ward

Two vastly different views of assessment whipsawed many of us over the past few days.

The first, a positive and hopeful view, pulsed through a half-day of sessions at KU’s annual Student Learning Symposium on Friday. The message there was that assessment provides an opportunity to understand student learning. Through curiosity and discovery, it yields valuable information and helps improve classes and curricula.

The second view came in the form of what a colleague accurately described as a “screed” in The New York Times. It argued that assessment turns hapless faculty members into tools of administrators and accreditors who seek vapid data on meaningless “learning outcomes” to justify an educational business model.

As I said, it was hard not to feel whipsawed. So let’s look a bit deeper into those two views and try to figure out what’s going on.

Clearly, the term “assessment” has taken on a lot of baggage over the last two decades. Molly Worthen, the North Carolina professor who wrote the Times op-ed article, highlights nearly every piece of that baggage: It is little more than a blunt bureaucratic instrument imposed from outside and upon high. It creates phony data. It lacks nuance. It fails to capture the important aspects of education. It is too expensive. It burdens overtaxed instructors. It generates little useful information. It blames instructors for things they have no control over. It is a political, not an educational, tool. It glosses over institutional problems.

Dawn Shew works on a poster during a session at the Student Learning Symposium. With her are, from left, Ben Wolfe, Steve Werninger and Kim Glover.

“Without thoughtful reconsideration, learning assessment will continue to devour a lot of money for meager results,” Worthen writes. “The movement’s focus on quantifying classroom experience makes it easy to shift blame for student failure wholly onto universities, ignoring deeper socio-economic reasons that cause many students to struggle with college-level work. Worse, when the effort to reduce learning to a list of job-ready skills goes too far, it misses the point of a university education.”

So if assessment is such a burden, why bother? Yes, there are political reasons, but assessment seems a reasonable request. If we profess to educate students, shouldn’t we be able to provide evidence of that? After all, we demand that our students provide evidence to back up arguments. We demand that our colleagues provide evidence in their research. So why should teaching and learning be any different?

I’m not saying that the assessment process is perfect. It certainly takes time and money to gather, analyze and present meaningful evidence, especially at the department, school or university level. At the learning symposium, an instructor pointed out that department-level assessment had essentially become an unfunded mandate, and indeed, if imposed from outside, assessment can seem like an albatross. And yet, it is hardly the evil beast that Worthen imagines.

Yes, in some cases assessment is required, and requirements make academics, who are used to considerable autonomy, chafe. But assessment is something we should do for ourselves, as I’ve written before. Think of it as a compass. Through constant monitoring, it provides valuable information about the direction and effectiveness of our classes and curricula. It allows us to make adjustments large and small that lead to better assignments and better learning for our students. It allows us to create a map of our curricula so that we know where individual classes move students on a journey toward a degree. In short, it helps us keep education relevant and ensures that our degrees mean something.

New data about assessment

That view lacks universal acceptance, but it is gaining ground. Figures released at the learning symposium by Josh Potter, the university’s documenting learning specialist, show that 73 percent of degree programs now report assessment data to the university, up from 59 percent in 2014. More importantly, more than half of those programs have discussed curriculum changes based on the assessment data they have gathered. In other words, those programs learned something important from assessment that encouraged them to take action.

That’s one of the most important aspects of assessment. It’s not just data we send into the ether. It’s data that can lead to valuable discussion and valuable understanding. It’s data that helps us make meaningful revisions.

The data that Potter released pointed to challenges, as well. Less than a third of those involved in program assessment say that their colleagues understand the purpose of assessment, that their department recognizes their work in assessment, or that they see a clear connection between assessment and student learning. Part of the problem, I think, is that many instructors want an easy-to-apply, one-size-fits-all approach. There simply is no single perfect method of assessment, as Potter makes clear in the many conversations he has with faculty members and departments. Another problem is that many people see it as a high-stakes game of gotcha, which it isn’t, or shouldn’t be.

“Assessment isn’t a treasure hunt for deficiencies in your department,” Potter said Friday.

Rather, assessment should start with questions from instructors and should include data that helps instructors see their courses in a broader way. Grades often obscure the nuances of learning and understanding. Assessment can make those nuances clearer. For instance, categories in a rubric add up to a grade for an individual student, but aggregate scores for each of those categories allow us to see where a broad swath of students need work or where we need to improve our instruction, structure assignments better, or revisit topics in a class.

Assessment as a constant process

That’s just one example. Individually, we subconsciously assess our classes day by day and week by week. We look at students’ faces for signs of comprehension. We judge the content of their questions and the sophistication of their arguments. We ask ourselves whether an especially quiet day in class means that students understand course material well or don’t understand at all.

The goal then should be to take the many meaningful observations we make and evidence we gather in our classes and connect them with similar work by our colleagues. By doing that on a department level, we gain a better understanding of curricula. By doing it on a university level, we gain a better understanding of degrees.

I’m not saying that any of this is easy. Someone has to aggregate data from the courses in a curriculum, and someone – actually, many someones – has to analyze that data and share results with colleagues. Universities need to provide the time and resources to make that happen, and they need to reward those who take it on. Assessment can’t live forever as an unfunded mandate. Despite the challenges that assessment brings, though, it needs to be an important part of what we do in higher education. Let me go back to Werther’s op-ed piece, which despite its screed-like tone contained nuggets of sanity. For instance:

“Producing thoughtful, talented graduates is not a matter of focusing on market-ready skills. It’s about giving students an opportunity that most of them will never have again in their lives: the chance for serious exploration of complicated intellectual problems, the gift of time in an institution where curiosity and discovery are the source of meaning.”

I agree wholeheartedly, and I think most of my colleagues would, too. A college education doesn’t happen magically, though. It requires courses to give it shape and curricula to give it meaning. And just as we want our students to embrace curiosity and discovery to guide their journey of intellectual exploration, so must we, their instructors, use curiosity and discovery to guide the constant development and redevelopment of our courses. That isn’t about “quantifying classroom experience,” as Werther argues. It’s about better understanding who we are and where we’re going.

Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

By Doug Ward

Student motivation is one of the most vexing challenges that instructors face. Students can’t learn if they aren’t engaged, and serious classroom material often fails to pique the interest of a generation that has grown up with the constant stimulation of smartphones, social media and video on demand.

Some instructors argue that motivation should be up to students, who are paying to come to college, after all. Most certainly, instructors can’t make students learn. Students have to cultivate that desire on their own. Instructors can take many steps to stoke that desire to learn, though, by drawing students into subject matter and into learning in general.

student sleeping — Photo by Cassandra Hamer, Unsplash

In a pedagogy class I’m teaching this semester, students and I worked through some of the steps we can take to motivate students. This is hardly a comprehensive list, but it touches on concrete steps that any instructor can take to draw students into class material and into learning.

Find links. Helping students make connections among seemingly unrelated topics deepens their thinking and expands their ability to learn. By tying their interests (say, music) to more challenging subject matter (the workings of the brain, for instance, or American history), we can motivate students to further their exploration and broaden their learning. As John Bransford, Ann Brown and Rodney Cocking write in How People Learn, helping students understand the usefulness of a subject can improve learning, as can making sure material is neither too difficult nor too easy and providing opportunities to share with others.

Vary class time. Approaching class in the same way every time lulls students into a routine that can lead to their tuning out or shutting down. Put yourself in students’ shoes: They may have three or four classes in a single day. That alone makes concentration a challenge. Things like breaking a 75-minute class into three or four small topics, playing a short video or audio clip at some point, or even having students stand up for a minute or two can break a routine and refocus attention.

Give students choices. We all need some sense of control over what we do and how we do something. Giving students choices on project topics, readings or quiz questions gives them at least some sense of control and ownership.

Use hands-on activities. Evidence is clear that active learning, in which students engage in discussions, work on problems, or take on questions in groups, is a far better means of instruction than lecture. All instructors need time to explain things to students, but the real learning begins when students engage with material in authentic ways.

Move around the room. Moving about the classroom or encouraging students to move about and talk with classmates can help maintain students’ attention. This also helps instructors get to know students better.

Encourage students. A few words of encouragement can go a long way in keeping students engaged. Remind students that learning takes time and that their peers struggle, as well. Don’t resort to false praise, but point out good elements in students’ work and help them build on those elements.

Make individual connections. Show your humanity and help students understand who you are as a person. That doesn’t mean befriending students, but learning their names, remembering faces, and talking to students about their interests and aspirations helps personalize the learning process and helps draw students into that process.

Use humor. Instructors don’t have to be stand-up comedians, but displaying a sense of humor makes them more relatable, diminishes anxiety and sends a message that learning can be fun.

Use games. The gamification of learning has grown considerably since the turn of the century, but games that help students learn have been part of learning for as long as there have been games. So using a game strategy in a class doesn’t require great technical know-how. For instance, I have created “Jeopardy” games in PowerPoint to help students learn grammar, and crossword puzzles to help them practice research skills. Those strategies require preparation, but I’ve found them very effective.

There are many other approaches to engaging students. Some require prep time and trial and error from instructors, but many others require little more than an open mind. We’d love to hear the strategies that work best for you.

Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

By Doug Ward

The University of Kansas has made many gains in its recruitment of minority students, who now make up 20.6 percent of the student body. By at least one measure, though, the university still has considerable work to do.

According to an analysis by The Hechinger Report, there is a substantial disparity in the number of Latino students who enroll at KU compared with the number who graduate from state high schools. Hechinger looked at enrollment rates for Latino and black students at public flagship universities in each state. KU had the 15^th largest gap in Latino students.

Here’s what that means: About 16 percent of Kansas high school graduates in the spring of 2015 were Latino. That fall, 8 percent of KU’s freshman class was Latino, a figure that rose to 8.7 percent by Fall 2017.

The university fared better in a comparison of black enrollment, ranking 31^st among the states (a lower ranking was better). About 7 percent of high school graduates in the spring of 2015 were black, while blacks made up 4.3 percent of the university’s freshman class that fall. (That fell to 3.9 percent among freshmen who started in Fall 2017.)

The highest disparities between the number of black high school graduates and blacks enrolling in flagship universities were primarily in the South, Hechinger said, with Mississippi showing the largest gap. Black students made up more than 50 percent of Mississippi high school graduates in the spring of 2015 but only about 10 percent of the freshman class at the University of Mississippi that year.

Among Latino students, the largest disparities were in the west: California, Texas, Nevada and Colorado. For instance, Latinos made up more than 50 percent of high school graduates but only about 12 percent of the freshman class at the University of California, Berkeley.

This graph from The Hechinger Report shows the percentage of high school graduates who were Latino and the percentage of Latinos among the freshman class at state flagship universities. http://hechingerreport.org/disparities-state-flagships/

An Amazon move worth watching

Inside Higher Ed speculates that Amazon may be preparing for a move into higher education. That’s because the company has hired the Stanford researcher Candace Thill, who has taken a leave of absence from the university to become Amazon’s director of learning science and engineering. Amazon and Thill had little to say beyond that.

Thill was a founding director of the Open Learning Initiative at Carnegie Mellon before moving to Stanford. She has helped create online learning materials based on findings from learning science, arguing that such materials can tailor feedback to individual needs, thus speeding up learning and leading to better scaling of classes.

The Open Learning Initiative is a competency-based system, meaning students work at their own pace, moving into new material only after demonstrating their understanding of previous material. The online system provides data to instructors and course designers, helping them improve course design and make better use of class time.

Using online learning to scale classes and reduce costs has been a dream of administrators and educational technology companies for years. Results have been mixed at best, with tech companies proclaiming grand breakthroughs even as instructors find that high-quality online teaching often takes more time than in-person teaching.

Higher education still sees digital technology as an important means of innovation and transformation, Jim Hundrieser, associate managing principal at AGB Institutional Strategies, said last month at the annual meeting of the Association of American Colleges and Universities. Colleges are struggling to find a sustainable business model, he said, and that could lead to a hard fall, much as publishing, textiles, music, steel, trucking, telecommunications and other industries have taken.

Hundrieser predicted that the number of online courses would continue to grow, especially because of their ability to reach students in remote areas, make learning more convenient, and allow for collaboration across time and space.

He’s right, although universities can’t simply toss out lackluster materials online and expect students to respond enthusiastically. Good online teaching requires a rethinking of pedagogy, course structure, student interaction, and learning itself. Universities still have some time to improve and expand their online offerings, but that time is drawing short as competition increases. If Amazon puts its enormous resources and brainpower behind educational technology and online learning, they had better be ready.

Colorado’s fee experiment

Course fees add hundreds or even thousands of dollars to the cost of a college degree. They are calculated separately from tuition, so they can hit hard when students’ bills come due each semester.

Starting this fall, the University of Colorado will eliminate most of those fees. Students will still pay fees for such things as the university bus system, recreation center and health center, but they will no longer pay course fees that range from $1 a credit hour to $1,255 a semester. That will save students $8.4 million a year, the university said.

The university is also spending $1 million on a pilot program that will provide open online textbooks to students at a fraction of the cost of publisher-created books.

The university system’s chancellor, Phil DeStefano, said in a university address that CU hoped to increase graduation rates by reducing educational costs.

Both the elimination of course fees and the investment in open educational resources are excellent moves. Of course, the university will have to absorb the costs, essentially cutting its income by $8.4 million a year. This is at a university system that ranks near the bottom nationally in state funding.

So how can it do that? The university cited rising enrollment and retention rates. CU expects 40,000 applicants (up from 37,000 in 2017) for this fall’s freshman class of 6,500. In contrast, KU has about 15,000 applicants each year. It accepts more than 90 percent of those students, but only about 4,000 eventually enroll.

More students are also transferring to CU, the Daily Camera of Boulder reports, and the university has increased its freshman retention rate to 87.5 percent, from 84 percent a few years ago. Those two things alone account for a substantial increase in revenue. Growth almost always makes budgeting easier.

CU also charges nearly $28,000 a year in tuition for in-state students and $52,000 a year for out-of-state students. In contrast, in-state residents pay $19,600 a year at KU; those from outside the state pay about $25,500.

It doesn’t hurt that the Denver to Boulder corridor is one of the country’s fastest-growing technology and biotechnology hubs, bringing employers and research dollars to the area. And then there are the mountains and the, uh-hum, weed.

For every Colorado, though, there is an Illinois, which lost more than 19,000 students to other states in 2016, The Chronicle of Higher Education reports. The University of Illinois system has frozen tuition to try to keep more students in the state, but the number has risen for five consecutive years.

Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

By Doug Ward

The criticism of liberal education often carries a vicious sting. For instance, listen to Jeb Bush, the former Florida governor:

“Universities ought to have skin in the game. When a student shows up, they ought to say, ‘Hey, that psych major deal, that philosophy major thing, that’s great. It’s important to have liberal arts … but realize, you’re going to be working at Chick-fil-A.’”

Examples of posters created during the workshop.

Or Gov. Matthew Bevin of Kentucky as he describes his budget priorities for higher education:

“There will be more incentives to electrical engineers than to French literature majors. All the people in the world that want to study French literature can do so. They are just not going to be subsidized by the taxpayer like engineers.”

Those sorts of disparaging comments certainly demonstrate an ignorance of higher education, but they also reflect the use of higher education as a political foil as the cost of college – and student debt – rises. Those simplistic characterizations have power. They stick in people’s minds and play into stereotypes of academia as an ivory tower separate from society at large and out of touch with the vast majority of Americans. They also reflect a growing emphasis on college as a job factory rather than a place to help citizens learn to think more deeply and more critically, and to expand their understanding of a complex and ever-changing world.

Higher education has done a poor job of pushing back against those criticisms, as I wrote earlier this week. Faculty members and administrators are eager to do better, though, as I found last week in a workshop I led at the annual meeting of the Association of American College and Universities in Washington. I gave participants a handout in which I had categorized common criticisms of liberal education and provided examples like the ones above. After a brief discussion, I asked them to identify an audience and create their own messages to address one or more of the criticisms. The results were excellent, showing a steely resolve to reclaim the reputation of higher education.

Categorizing criticisms

I generally see six types of criticisms of liberal education. Most come from outside the academy, but some come from inside. There are overlapping aspects among all of them, and no doubt there are others. (For instance, one workshop participant pointed out the complaint that the liberal arts focuses heavily on the ideas of long-dead white men.) These are the common ones that I’ve identified, though, and that I shared in the workshop:

College costs too much to waste on “impractical” subjects
The study of the liberal arts has become an anachronism
Liberal education is out of touch with the “real world”
Liberal education isn’t keeping up with a changing world
Liberal education has lost its meaning
Identity consciousness has tainted liberal education

I asked workshop participants to work in pairs or groups, choose one or more of those criticisms, and create both a soundbite and more substantial messages that highlight the strengths of liberal education. Some rejected the idea of soundbites. That’s understandable. Matching soundbite to soundbite can easily devolve into the equivalent of a playground brawl rather than a meaningful conversation. Nonetheless, I think it is important that we distill the importance of liberal education into key elements to use when talking with students, parents, donors, community members, politicians, and even colleagues.

Here are examples of how workshop participants rose to that challenge:

Change is a constant. Liberal education provides the means to create and navigate that change.
Liberal education is a pedagogy and an ethos, not a set of disciplines.
Finding a path and a voice in the world.
Your life is better when we think better together.
Get a career, get a purpose, get a life, get a college education.
Build a team that knows how to think.
Liberal arts will get your promotion.
Pivot for your next opportunity.
Invest in the long run.
We teach essential skills for living fully and freely, everything you need for citizenship and prosperity, self-fulfillment and self-determination.

Two groups focused specifically on Republican donors, drawing on the language of business to make a connection:

Liberal education builds workplace skills: adaptability, flexibility, communication skills, evaluation and analytical skills, interpersonal skills in diverse populations. It also instills ethics and fosters curiosity.
The liberal arts yields effective communication skills in multiple modes, which is core to successful messaging, interaction, negotiation, innovation, collaboration, creative problem-solving, sales and marketing, global perspective, diverse audiences and cultures.

As I said, there are dangers in trying to compress the complexities of liberal education into soundbites or even more substantial talking points. We will never do it justice. By thinking in those terms, though, we can better identify the components of higher education we want to emphasize and better prepare ourselves for conversations with a broad range of constituencies.

So let’s keep talking.

Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

By Doug Ward

American higher education has taken a beating over the last 40-plus years.

Many of those blows came from the outside. Many others were self-inflicted. I won’t rehash those here, other than to say that higher education has done a poor job of fighting back. Much of the time, it has seen itself as above the fray. Its arrogance not only blinded it to its own shortcomings but let critics paint an unflattering portrait that has lingered in the minds of millions of Americans.

A board at the AAC&U meeting asked participants to share their thoughts about higher education. The theme of the meeting was “Can Higher Education Recapture the Elusive American Dream?”

Thankfully, colleges and universities have awakened from their slumber and started to realize that they must live within the broader society, not separate from it, and that they must make a case that higher education plays a vital role in democracy and the American dream. Yes, that sounds lofty. But it is crucial if we hope to maintain our colleges and universities as places of knowledge, aspiration, and above all, hope.

That sentiment was clearly evident last week in Washington, D.C., at the annual meeting of the Association of American Colleges and Universities. Speakers were alternately determined, defiant, pragmatic, searching, and hopeful. Like so many others, I came away energized by conversations with colleagues who are determined to reinvigorate higher education, and by sessions that focused on the core elements of AAC&U’s new strategic plan:

Championing sustainable models for high-quality undergraduate education
Advancing equity
Articulating the value of liberal education
Pushing for innovative approaches to change higher education

Speakers at the conference’s opening plenary were blunt about the problems that higher education faces. The United States used to be the world leader in degree holders, Lynn Pasquerella, president of AAC&U, told participants. It now ranks 15^th. Public higher education was once a truly public venture financed mostly by taxpayer dollars. Now it is public in name only as colleges and universities rely increasingly on private fundraising, tuition dollars, and grants to pay the bills. That, in turn, pushes institutions to obsess about rankings, which pushes them to seek students with higher test scores, which pushes them to build luxury facilities, which forces universities to seek private financing and push up tuition costs, which puts college further out of reach for more and more families.

That chain of events has led to both a financial and moral crisis in higher education, said Linda Martin Alcoff, a professor at City University of New York. Privatization has turned students and faculty into “human capital,” she said. Rankings have “infected” every faculty search as departments seek out stars who can improve rankings, Alcoff said. Faculty achieve star status by attracting private grant money, which has deteriorated the civic nature of higher education, she said.

“We’ve become beggars at the table,” Alcoff said. “Every time there’s a search, our chairs are beggars at the table with deans and provosts for positions that are ultimately decided by corporate boards of trustees and ranking mechanisms. … We’re all quite aware of the problem, but we have been lulled into quietude.”

New pressures on a college degree

Tamara Draut, a vice president at the public policy organization Demos, said that we in higher education must work to “unleash that era of possibility” that allowed so many people to get through college without enormous debt. Debt has poisoned higher education by creating an obsession with rankings and a need to recruit increasing numbers of out-of-state and international students, who pay higher tuition.

“There’s a lot of perversion that has happened in the academy because it has become connected to debt,” Draut said. “It has put pressure on a college degree to do something it was never supposed to do, which is show some ROI for the degree you get.”

She gave the example of a young woman who called in to an NPR show that Draut participated in. The young woman, who had an art degree and was working at a community center teaching art to children, was having a difficult time paying off her college debt. The next caller ranted about the young woman for “daring to get an art degree” rather than an engineering or technical degree.

Debt, Draut said, is “putting all kinds of burdens on institutions and on degrees that they were never meant to carry. And it’s making us devalue the learning and the doing that are the high marks of civilization: art, music, philosophy, education, doing good for others. That is what we should be lifting up. But the reality is, if you get an art degree and can’t pay back your student loans, we are saying to people that you did something bad and you should have studied something different.”

AAC&U members clearly took an inclusive view of higher education, as they should. College was once only the purview of the elite, and the rising cost of attending is clearly pushing it that way again.

“What happens is a lot of working class and poor people hear us saying you need to go to college,” Draut said. “The reason you are struggling is because you didn’t go to college. You made bad choices.”

That either/or narrative only sours people on higher education, she said. College is important, she said, but it is not a solution to poverty, prejudice or the growing gap between the ultrawealthy and everyone else.

“Higher ed is great, but it’s not all we have to do to fix society’s economic and racial inequality,” Draut said.

The importance of access

Panelists throughout the conference issued a call for educators to push for policies that provide broader access to higher education but also help re-establish a broad middle class.

“Teaching the poor should not be a niche market in higher education, but that’s what it has become,” Alcoff said.

She added: “The goal should be social justice for all so that those who engage in any kind of labor can have financial security.”

Wes Moore of the Robin Hood Foundation urged educators and alumni to tell their stories about the importance of higher education. Statistics can be helpful, he said, but they can also be manipulated.

“Make sure people understand the human implications of what we do,” Moore said. “It’s important to remind people not just what we are talking about but who we are talking about.”

Alcoff offered a similar point, saying that we must espouse the importance of higher education without alienating those who choose not to – or can’t – get a degree. By linking a college education to social mobility, we leave out a large portion of the American population.

“The goal of social mobility is the wrong goal in the United States today,” she said. “The goal should be social justice for all so that those who engage in manual labor – or any kind of labor – can have lives of dignity, can own a home, can send their kids to a good state university, and can have financial security.”

We must also make room for less-than-perfect students who aspire to the intellectual challenges of college, Alcoff said. With what she described as a “checkered past,” she never would have made it through college in today’s environment, she said. She was on her own financially at age 16, earned a GED, dropped out of college, found her way back, and eventually graduated. College is no longer forgiving for such students, she said, especially with costs that weigh on students for years.

Naomi Barry-Pérez, director of the civil rights center for the Justice Department, tied decreased funding of higher education and many social programs to a backlash against the civil rights and women’s movements in the 1950s, ’60s and ’70s. Government programs aimed at making society fairer were demonized once women and people of color gained more power, she said. We are the richest nation on earth, she said, but we life in a perpetual state of austerity.

“We have to be champions of reinvesting in ourselves,” she said.

Conflicting ideals

As passionate as the speakers were, they offered few solutions to entrenched problems that have been exacerbated by polarized politics. In most cases, there are no clear answers.

In the closing plenary, the author William Deresiewicz told conference-goers that higher education had been in crisis “since the beginning, perhaps since 1636.” The problems have changed, though, with the biggest today being the decline in education funding.

He said, though, that academics needed to delve more deeply into their own beliefs and actions. We talk about freedom, equality and justice, he said, but rarely think about the conflicts inherent in them. Equality often demands the diminishment of freedom, he said. We want to encourage creative expression, but at the same time, we have a need for all people to feel safe. That, in turn, often requires restrictions. Dealing with those conflicts is difficult and troubling, he said. Nobody wants to think about their own beliefs, values, and assumptions. At colleges and universities, that inaction silences voices and distances academia from the rest of society, he said.

“We live at a time when progressive opinion, which dominates most campuses, has hardened into something approaching religious dogma,” Deresiewicz said. “There’s a right way to think, and a right way to talk, and a right set of things to think and talk about. Secularism is taken for granted. Environmentalism is a sacred cause. Issues of identity occupy the center of discourse.”

There really is nothing to debate, he said, saying that he shared those beliefs, but “the fact that it’s inconceivable to think otherwise is precisely the problem.”

“The assumption on the left is that we are already in full possession of the moral truth,” Deresiewicz said. “We already know what’s good, what’s bad, what’s right, what’s wrong. There really is nothing to discuss, except how to put a belief into practice. Dogma makes for ideological consensus, and consensus is enforced through social means.”

He told of a recent experience in teaching a writing class for college juniors and seniors. All of the students were ill-prepared to read deeply, analyze others’ work, or to make strong arguments. These were smart students, he said, but they had learned only a technocratic form, one in which difficult question could be worked out in predictable ways. They also thought of writing as “something that just happened,” rather than as a process that requires persistent revision and questioning.

He pointed to several things he said were at the heart of the problem: social media and its fast-paced, anything goes mentality; grade inflation; adjunct instructors who can’t afford to spend time with student papers; and professors who lack incentives to take the time. If we spend all our time focusing on skills that can be scaffolded and measured, he said, we miss opportunities to delve into bigger questions like values, purpose and meaning that can transform students during their time in college. All too often, the humanities converts open-ended questions into things that can be assessed and tested, he said. As a result, students think fundamental questions about life and meaning have been settled. They learn to spout opinions, but recoil at the idea of public argument. They talk about things like patriarchy, intersectionality, trigger warnings, and microaggressions, but they are lost when they have to think outside those categories or are asked to examine what they mean or how others might feel differently.

“Big questions are big questions because no one has the answers,” Deresiewicz said.

What he failed to mention is that the dogma that afflicts the left also afflicts the right, making meaningful conversation and compromise even more difficult. Like other speakers at AAC&U, though, he was spot-on in calling for higher education to take a deep look inside itself. That’s the only way we will find a way forward.

Doug Ward is the associate director of the Center for Teaching Excellence and an associate professor of journalism. You can follow him on Twitter @kuediting.

From the deck of the Enterprise to classrooms around the world

An entrepreneur endorses the liberal arts

Using data to illuminate teaching and learning

Embracing a ‘cycle of progress’

Focusing on retention and graduation

Data efforts at KU

Making peer review of teaching more meaningful

Work your way through college? Not anymore

It’s time to move beyond a bogeyman view of assessment

New data about assessment

Assessment as a constant process

9 easy ways to improve student engagement

Analysis shows continued disparities in minority enrollment

An Amazon move worth watching

Colorado’s fee experiment

Countering the criticisms of liberal education

Categorizing criticisms

Taking on hard questions about education’s future

New pressures on a college degree

The importance of access

Conflicting ideals

Recent Posts

Archives

Recent Posts

Recent Comments

Archives

CTE’s Twitter feed

Search form

An entrepreneur endorses the liberal arts

Embracing a ‘cycle of progress’

Focusing on retention and graduation

Data efforts at KU

Work your way through college? Not anymore

New data about assessment

Assessment as a constant process

An Amazon move worth watching

Colorado’s fee experiment

Categorizing criticisms

New pressures on a college degree

The importance of access

Conflicting ideals

Recent Posts

Tags

Archives

Recent Posts

What we write about

Recent Comments

Archives

CTE’s Twitter feed