Preposterous Universe

Monday, May 23, 2005
Sexy? Sure. Easy? Never.

Via a list of links (Volokh to Prawfsblawg to Pub Sociology to Andrew Gelman), we are led to a study that addresses one of the crucial pressing issues of the contemporary academy: how (or why) do some people get good teaching evaluations? Answer: by giving good grades and by being sexy. Here's the abstract of the paper, by Felton, Mitchell, and Stinson:
College students publicly rate their professors' teaching at RateMyProfessors.com, a web page where students anonymously judge their professors on Quality, Easiness, and Sexiness. Using the data from this web site, we examine the relations between Quality, Easiness, and Sexiness for 3,190 professors at 25 universities. For faculty with at least 10 student posts, the correlation between Quality and Easiness is 0.61, and the correlation between Quality and Sexiness is 0.30. Using simple linear regression, we find that about half of the variation in Quality is a function of Easiness and Sexiness. Accordingly, these results suggest that about half of the variation in student opinion survey scores used by universities for promotion, tenure, and teaching award decisions may be due to the easiness of the course and the sexiness of the professor. When grouped into sexy and non-sexy professors, the data reveal that students give sexy-rated professors higher Quality and Easiness scores. Based on these findings, universities need to rethink the use of student opinion surveys as a valid measure of teaching effectiveness. High student opinion survey scores might well be viewed with suspicion rather than reverence, since they might indicate a lack of rigor, little student learning, and grade inflation.
Now, I admit I haven't read the paper itself very carefully. And I'm always reluctant to criticize credentialed experts in fields outside my own (really, I am). But this abstract doesn't fill me with a great deal of confidence.

First, the data come from RateMyProfessors.com. You're kidding, right? Perhaps there might be some sort of selection bias in the students who take the time to fill in entries on the web site? More importantly, the authors seem to take for granted the existence of a priori categories of "easiness" and "sexiness," and claim that their existence is distorting the sought-after measure of "quality." It seems not to occur to them that, for example, sexiness and quality might correlate because they both are caused by some third external factor. Or that a class might be subjectively rated as relatively easy, not because the grade distribution was actually higher, but because the students came away with the feeling that they had really understood the material. Or, most likely of all, that students found certain professors to be sexy because they were good teachers. Effective pedagogy, you don't need me to remind you, is hot.

(And no, I don't have an entry at RateMyProfessors.com. But my teaching evaluations are pretty good. Draw your own conclusions.)

Ideas on culture, science, politics.
Sean Carroll

Preposterous Home
Atom Site Feed (xml)
RSS Feed
Technorati Profile
Bloglines Citations
Blogroll Me

About Last Night
Alas, a Blog
The American Sector
Asymmetrical Information
Big Brass Blog
Bitch, Ph.D.
Body and Soul
Brad DeLong
Chris C Mooney
Collision Detection
Creek Running North
Crescat Sententia
Crooked Timber
Daily Kos
Daniel Drezner
Deepen the Mystery
Dispatches from the Culture Wars
Dynamics of Cats
Electron Blue
Ezra Klein
The Fulcrum
Girls Are Pretty
Jacques Distler
James Wolcott
John and Belle
Julie Saltman
Lawyers, Guns and Money
Leiter Reports
The Loom
Matt McIrvin
Matthew Yglesias
Michael Bérubé
Michael Nielsen
Mixing Memory
Mr. Sun
Not Even Wrong
Obsidian Wings
Orange Quark
Paige's Page
Panda's Thumb
Playing School, Irreverently
Political Animal
The Poor Man
Quantum Diaries
Quark Soup
Real Climate
Roger Ailes
Rox Populi
Shakespeare's Sister
Simple Stories
Sisyphus Shrugged
Smijer & Buck
TPM Cafe
Uncertain Principles
Volokh Conspiracy

Powered by Blogger
Comments by Haloscan
RSS Feed by 2RSS.com

February 2004
March 2004
April 2004
May 2004
June 2004
July 2004
August 2004
September 2004
October 2004
November 2004
December 2004
January 2005
February 2005
March 2005
April 2005
May 2005
June 2005
July 2005