Student evaluations have been criticized in recent years for gender and racial bias. And yet, such reviews are critical tools in the savvy student's toolbox. So what should we do about it?
RateMyProfessors.com didn't exist when I was a student in college, but it became a powerful force when I taught as a graduate student instructor and later as an adjunct professor. I remember regularly checking my reviews when I first started teaching, and I wondered if I was alone.
To find out, I sent an email to my colleagues from graduate school. Everyone who responded admitted to reading their reviews on RateMyProfessors.com at some point or another. We reflected on the wild west days of the "chili pepper" ratings — only dropped by the site in 2018 — and then we got vulnerable, sharing the two central reasons we read what you said about us.
Your Online Reviews Provide Validation
Professors are people too, and words of affirmation from our students send us to the professorial moon. As Dr. Devon Madon, a teacher at a public magnet high school, noted, "I got positive reviews and was always happy to see my page pop up when I Googled my name." (Yes, we Google our names, too.)
That said, we're also professionals. We generally use more formal tools — including the evaluations we ask students to complete and submit to our colleges and universities at the end of each class — to improve our course designs and teaching semester over semester.
Online Reviews Provoke Anxiety — Especially for the Untenured
Graduate student instructors, adjunct instructors, and non-tenured professors occupy a particularly tenuous professional space. Whether legitimate or not, the specter of a negative online review can loom large over our hopes of securing our dream jobs — or any job at all.
"It's purely voluntary, so only folks with strong feelings are likely to use it — you're basically getting one or both ends of the bell curve of student reactions, with little of the middle." - Dr. Sean O'Brien
In an email exchange, Dr. Sean O'Brien, a current clinical associate professor, reflected on the site's power and his feelings of vulnerability: "As a grad student, this was a public-facing evaluation of me that prospective employers could easily see, so it was a cause of anxiety regarding the job market." O'Brien added, "It's purely voluntary, so only folks with strong feelings are likely to use it — you're basically getting one or both ends of the bell curve of student reactions, with little of the middle."
And when only a handful of students submit reviews for a given instructor, this small sample size, combined with the assimilation effect, can lead to the dreaded confounded face emoji (😖) for that professor's page pretty quickly.
What Is the Assimilation Effect?
"An effect in which participants' judgments shift toward an anchor after it is introduced," according to the American Psychological Association. In other words, it's when people are primed to have a response or opinion after observing someone else's.
RateMyProfessors.com Fills a Valid Need
While this particular rating website might not have a meaningful impact on how professors structure and lead their courses, it can help students make more educated decisions when choosing sections of a particular class — or even when deciding whether to take a particular course at all.
Not every student has the luxury of choosing their professors for each of their classes, but most students like the opportunity to make an informed decision when registering. Back in our day, before the advent of the internet, we did essentially the same thing. Here's one version:
When I was an undergrad, a junior gave me a course catalog — which was a phone-book-sized paper object — and rated the professors he'd had on a scale of 1-5.
I used this reference when scheduling my classes, and I added to it with my own reviews each semester and invited a few of my friends to do likewise.
A couple students replicated our guides into their own catalogs, and I passed my catalog on to another student when I graduated.
—Dr. Sean O'Brien
This private and inconvenient analog method was worth the hassle because of the valuable insights it offered — and because the reviews were authenticated and trustworthy.
Given the ever-increasing cost of college, it strikes me that students are smart consumers if they do their homework before registering for a class.
RateMyProfessors.com is not the only way to digitize the rating experience. One teacher has seen an "unofficial" Facebook page her students use to rate their teachers, and I'm certain students still use their word-of-digital-mouth networks to help younger counterparts pick which professors to choose or avoid.
But the Evaluation Process Is Marked by Systemic Bias
Over the last decade and more, RateMyProfessors.com — and all evaluation tools, including those completed in the classroom — have rightly come under fire for reflecting systemic bias, including:
- Gender bias: the way in which men - especially white men — tend to receive preferential treatment over women
- Sampling bias: the way in which only students motivated by love or hate will take the time to craft voluntary online reviews, leaving little representation of more moderate takes
- Confirmation bias: the way in which we tend to like information that supports what we already believe
- Hasty generalization: students reach conclusions about a professor based on potentially tiny sample sizes — as few as one or two reviews
Simply put, RateMyProfessors.com is not a forum designed for independently imagined, measured, or fair evaluations — especially for women.
Simply put, RateMyProfessors.com is not a forum designed for independently imagined, measured, or fair evaluations — especially for women. Women on the site tend to be critiqued more harshly than their male counterparts. Men tend to receive more superlative feedback regarding their intelligence while women are often judged on non-academic concerns, such as their "maternal" or "nurturing" nature, whether they have children, and their overall appearance. Moreover, there remains a dearth of research around racial bias in evaluations.
In response to the presence of systemic bias, some schools are moving away from formal course evaluations. However, this doesn't help students find the right classes and professors for them.
So How Do We Fix It?
Just as we shouldn't have asked students to single-handedly preempt the COVID-19 onslaught in the fall, neither should we ask them to individually sacrifice informed decision-making in pursuit of eliminating systemic biases.
Although RateMyProfessors.com didn't create these systemic biases, it does occupy an extremely privileged space: that of a tech company in charge of its own user interface.
Unlike traditional evaluations conducted by universities themselves, RateMyProfessors.com can use its own platform to counteract bias in ways that more bureaucratic organizations, like colleges and universities, could only dream about.
The designers of this valuable database should take a hard look at the data they collect and how they collect it.
Ever get a pop-up notice from Gmail that says: You wrote, "I've attached" in your message, but there are no files attached. Send anyway?
How about one from RateMyProfessors.com asking, You wrote, "This professor is terrible!" Did you just receive your final grade? Do you want to reflect on the course a bit before posting your review?
Or perhaps: You mentioned your professor's "maternal instinct." What does that have to do with her ability to successfully teach the subject matter? Would you say this about a male professor?
Similarly, why not reward students for evaluating all their professors in a semester, not just the ones they loved or hated? Perhaps offer an annual scholarship, awarded via lottery, to students who submitted reviews for all their professors?
With the world moving online, a look at the systems that reinforce bias is past due. But, as I used to tell my students who slid into their seats after the bell: Better late than never.
Sarah Eilefson received her Ph.D. from Loyola University Chicago and teaches composition at the University of Minnesota-Duluth. Her scholarly articles have appeared in Text & Presentation and Scholarly Editing, and she has contributed to a number of publications in the fields of healthcare and law enforcement.
Header Image Credit: Epitavi, Tab62 | Shutterstock