I woke up this morning to a flurry of Facebook links to Do the Best Professors Get the Worst Ratings? on Psychology Today. Everyone also seemed to be excerpting the same summary, and I now follow suit here:

To summarize the findings: because they didn’t teach to the test, the professors who instilled the deepest learning in their students came out looking the worst in terms of student evaluations and initial exam performance. To me, these results were staggering, and I don’t say that lightly.

Bottom line? Student evaluations are of questionable value.

The Psychology Today post starts by observing that a professor’s livelihood depends on positive course evaluations, which is part of why this matter is of concern to professors. Admittedly, the author, Nate Kornell, teaches at the nation’s top-ranked liberal arts college, where teaching performance is taken as a larger part of evaluation than it is at a research university.

But arguably, that fact gives the course evaluation even greater perceived value at research institutions. Fewer metrics are used to evaluate teaching in those contexts, and those metrics that appear “scientific” (that is, get averaged as a number) are more likely to be taken as a representation of fact. I’d hardly boast “best professor” status, but I can say that in my last promotion review I had to cite anonymized feedback from senior capstone essays citing the positive long(er) term impact of one of my lower-level classes, to “apologize” for lower average ratings immediately after it. After all, course evaluations are set up to solicit immediate feedback during the most stressful time of the term, while the most intense emotions are at work.

In that respect, course ratings are not much different than Yelp ratings or YouTube comments. They’re great for immediate rapture or grouse, but less useful as actual criticism or feedback. It is a perturbing and frightening fact that these ratings are still taken so seriously in light of what the Internet has taught us about numeric ratings and comments more generally.

published June 2, 2013

Comments

  1. Shawn Kerr

    To me this shows not only that using numbers & metrics can be “broken” but also that the emphasis on grades and tests has created a poor mindset in undergraduates of America. Teachers getting poor performance reviews because they don’t “teach to the test” is demonstrable of the fact that students are not valuing their learning and are valuing the grade they will be given on tests and at the end of the course. Students are focusing on the wrong part of education (not necessarily their fault), are putting value on the wrong things, and are hurting good teachers because of it.

    It seems to me that if this is what the data proposes there is a lot more to be learned from the data than whether student reviews are valid in their judgements of educators.

  2. Ian Bogost

    Oh, that’s certainly true, Shawn. However, that mindset starts so early and proceeds so deliberately that it’s hard to disrupt it at the age of 17.

  3. Mike Rentas

    Most students are interested primarily in maintaining a high GPA in order to look good when applying for jobs. As long as employers and schools stay focused on numbers, tests, and performance metrics, nobody is going to care about actually learning anything.

  4. Ian Bogost

    The upshot being, GPA is less likely to be relevant to employers than other things, particularly demonstrated, synthetic expertise in the chosen field.

  5. I. Read

    I applaud the intended goals of academia. However, from a third person perspective, the culture seems peculiarly rotten around the edges even if there is good fruit near the center. Culpability for this is surely shared between professors, the administration and the students.

    What saddens me most is that while the internet has a created an autodidact’s paradise in almost every way, certain varieties of information, experience and interchange can only be obtained or transpire by going to college. This conflict is what killed Aaron Swartz, almost literally.

    The misguided administrative methodology detailed above is saddening, sure, but it is not surprising. It’s prevalent in every major social institution essential to maintaining the status quo. Education isn’t an island, it just thinks it is… and a certain kind of heat map would show a lot of accomplished thinkers in the vicinities of colleges. But being in the proximity of lots of thinking does not make one imply one is morally impervious or capable.

    I’m sorry if my schadenfreude is showing. I’m really crying on the inside. I hate to see education fail, to see good intentions and good people suffer because of imbeciles in charge of policy and a shallow student body. Nonetheless, my frustration at the culture surrounding college and the disdain grads seem to carry in their front pockets for the “unclean” like myself is so great, I’d happily watch the institution burn until all that remains is Wikipedia, Khan Academy, Codeacademy. That is, were it not for all the academic journals and genuinely talented and innovative professors out there who are gatekeepers to the world’s knowledge, gatekeepers beholden to an antiquated method of knowledge dissemination, corrupt politicians and even more corrupt benefactors.

    What I mostly get from this is that idiot students hate their good professors as much as they look down upon people who didn’t go to college. It smells like the stench of class privilege.

  6. Ian Bogost

    “An autodidact’s paradise in almost every way” seems like a fairly massive overstatement.

  7. I. Read

    Ok, you got me. I really need to stave off propensity for late night hyperbole. I’m thinking the insertion of the word “potential” somewhere in that sentence would better support what I was trying to express.

    Still, the availability of useful information on the internet to the layman has increased by several magnitudes since I was looking up fatalities for Mortal Kombat II on Usenet in ’94. I can only imagine the sport I would have been making of my high school teachers if I had even a fragment of Wikipedia available to me at the time. That’s probably just a pointless revenge fantasy, however. Eternal September sprawled forth in magnitudes as well, who am I to speculate what side of the coin I would have favored had I been exposed to the more virulent forms of internet culture that grew in tandem.

    Suffice to say, a wash of warm sentiment relating how far we’ve come lead to counterproductive rhetoric when trying to (melodramatically?) assert how far there is to go vis a vis privileged resources unavailable to the committed autodidact. And this is all more anarchistically related than tangentially to your – obviously valid – observations about class ratings. I’m sort of cursed with a “kill ’em all” mentality at the moment, I hope you consider my comment to hold a little more weight than trollish polemic; I’m personally a fan, I find your discourse to be elevated, conscious and meta the way few things are – I hope my first comments on your site aren’t detracting from this tradition despite expressing some deeply held convictions.