College rankings – like those published by U.S. News & World Report – can be misleading, skewed, and they even can drive schools to alter their services, scholarships and missions, according to a University of Iowa researcher.
Michael Sauder, a UI associate professor in sociology and fellow-in-residence with the UI Obermann Center for Advanced Studies, spent the last semester analyzing the unintended consequences of rankings – especially those in the educational realm.
He and his co-researcher, Wendy Espeland with Northwestern University, conducted extensive interviews and gathered qualitative information about 10 years of rankings going back to 2003. They focused mostly on law school rankings, but also looked at business schools and undergraduate colleges and universities.
What they found were administrators who cared deeply – maybe too much – about where their schools ranked in national publications like U.S. News & World Report, which has become a go-to guide for many would-be students.
“Some become kind of obsessed with rankings,” Sauder said.
Almost every school has a strategy to improve its standing on the lists, which rate institutions based on things like tuition and fees, admission, retention and graduation rates.
“But sometimes those strategies don’t seem to be the best for the education at the school,” Sauder said. “Some schools are more concerned about the numbers than the educational mission.”
Regarding law school rankings, Sauder said, peer reviews weigh heavily, and he found that many law schools spend thousands to millions of dollars trying to raise their reputational scores among peers. They might send more promotional materials and brochures to administrators at other schools than to prospective students, he said.
“In a few cases, schools have spent millions of dollars effectively advertising themselves to other schools,” Sauder said.
Some schools have gone so far as to alter their missions if it means improving their standing, according to Sauder’s research. For example, a school’s mission to enroll a diverse populace of students might be overshadowed by a desire to admit only students with high test scores.
“They are making the decision between being an inclusive law school and just taking students with high LSAT scores, even if it goes against their mission,” Sauder said. “That is the type of effect that rankings have on schools. For everything, they think, ‘What is this going to do to my ranking?’”
Schools also might employ “gaming strategies” when providing data to the ranking organizations, skewing numbers to make them look better, Sauder said.
Looking again at law schools, he said, institutions provide the percentage of employed graduates, which traditionally counted just graduates with jobs in the legal field. But a few began including any form of employment – including jobs at restaurants or shelving books at the library.
Those schools saw their rankings improve, Sauder said, and now almost every school counts all forms of employment.
“Now, employment numbers in the U.S. News & World Report are all very high because they are counting everyone who has a job,” he said.
From time to time, Sauder said, reports surface of institutions simply lying for the rankings.
“You have that level of cheating too,” he said. “But you never know how much of that goes on because you only hear about it when they get caught.”
Effects of the rankings aren’t all bad, Sauder said, reporting that they do provide a basic level of guidance for prospective students.
That information is meant to supplement additional research and college visits. And, Sauder said, the U.S. News & World Report rankings – at least when they first debuted – filled a void.
But he said, the ranking system could work better if there were more well-respected and well-read lists to compare.
“If a school ranks fourth in one system and 45th in another, it becomes pretty clear that something is off kilter,” he said.
Sauder and his colleagues are publishing the research in a book tentatively called, “Fear of Falling: How Media Rankings Changed Legal Education in America.” He expects it to be published later this year or in 2015 with the goal of making students and administrators more savvy about the rankings.
“It’s more important to choose a school that feels like a good fit instead of based on rankings,” he said.