Judging the rankings: Results give just part of the college picture
February 16, 2012 3:00 PM
The Ellis School senior Priyanka Amirneni values campus visits more than rankings.
By Taryn Luna Pittsburgh Post-Gazette
Nearly every college is ranked in something.
Carnegie Mellon University ties for No. 23 in the national university rankings of U.S. News & World Report.
Penn State University's main campus lands in a five-way tie for No. 45 in the same ranking, but it usually fares better in the Princeton Review's list of top party schools, where it currently resides in seventh place.
And University of Pittsburgh students are apparently the 20th-happiest student body in America, also according to Princeton Review.
But does rank matter?
The answer varies widely depending on whom you ask.
Sophia Rosenfeld, a senior at The Ellis School in Shadyside, said rankings didn't factor into her decision to attend the University of Chicago next fall.
"I didn't think a ranking would show me what the best fit for me was," explained the native Pittsburgher.
When asked if she knew the university's position in the U.S. News & World Report rankings, the most commonly referenced and promoted assessments of their kind on the market, she was quick to respond with "No. 5" -- and she was right.
"Rankings, whether people like them or not, are part of the college selection culture," said Paul-James Cukanna, associate provost for enrollment management at Duquesne University.
Students like Ms. Rosenfeld may not rely on rankings to determine what school is best for them, but they do use them for reassurance, or to determine a pool of schools to apply to, he said.
"Most consumers, in any industry -- we're looking for validation that our choice is accurate," he said. "Students and parents are no different when it comes to the higher education market."
He describes rankings as one "part of a toolkit to gather information to make a decision" about which college to attend.
Universities pay attention to rankings for a different reason.
Mr. Cukanna said his college uses rankings as "a way to get more students interested in your university" and gauge their own competition in the higher education marketplace.
The Dukes don't hesitate to promote their standing.
The school's website has a page dedicated to tout its rankings from the U.S. News & World Report, Princeton Review and other publications.
Likewise, the public relations team routinely distributes press releases to plug the latest favorable nod.
Mr. Cukanna thinks it's important that the school remains cognizant of where it falls. "We always try to understand why we've increased and why we haven't," he said.
But not everyone in education places as much stock in rankings.
Joanna Schultz, director of college counseling at The Ellis School, an independent girls' school that boasts a 100 percent college attendance rate among its graduates, advises students to ignore rankings.
"We don't want students to have a ranked list of colleges that are good and some better than others," she said. "We want them to have a list of colleges that are really good for them, OK for them or not good for them."
Ms. Schultz is among many education professionals to question the credibility of the U.S. News & World Report ranking methodology, which dedicates 22.5 percent of a school's score to reputation.
Reputation is heavily weighted on how provosts, presidents and upper administration at universities rate each other's institutions.
Howard Ishiyama, vice president for academic affairs and academic dean at La Roche College, said he's proof that the U.S. News approach has its flaws.
In the 10 years he's been in higher education, he's become familiar with about 5 to 10 percent of the schools he's asked to rate.
Some he's heard of but knows very little about their quality.
Mr. Ishiyama either chooses not to rank these schools or he scores them somewhere in the middle of the road: not too high, but not too low.
"It brings in a lot of guesswork, and it kind of codifies it as if it were more than that," he said. "It accumulates guesswork."
When he rates the schools of colleagues that he knows well, a natural bias comes into play.
"The other circles that I'm part of are personal relationships with chief academic officers, and some of that can -- to the extent to which I'm friendly, or like or respect those people -- tend to have a side effect on how I rate them," he said.
"I am exactly the kind of person who has been rating La Roche and every other school on that list," he said.
Robert Morse, director of data research for U.S. News & World Report, stands behind the credibility of the methodology he created as "one way to evaluate schools."
The academic reputation portion, he said, asks experts in the field for their assessments on anywhere from 100 to 120 other schools, not the full list of 250.
He said he doesn't expect university presidents to have a superb detailed knowledge of all of the schools they rate.
The methodology assumes that "you've had experience in higher education and you've risen through the ranks and you have some knowledge of other schools" to provide a credible perspective, he said.
There's no doubt that reputation plays a vital role in how students select a college.
Mr. Morse cited a recent UCLA study that asked 203,967 first-year students at 270 four-year colleges and universities in the nation to list important influential factors in their college search.
Rankings came in as the 11th most-influential factor, but a school's academic reputation topped the list, which he suggests supports his methodology's weight on reputation.
"Research shows that the perspective students in the freshman survey think that the reputation of a school is very important," he said.
David Hawkins, director of public policy and research at the National Association for College Admission Counseling in Arlington, Va., said he doubts the impressions of university executives best suit the students who use the surveys.
That's true at least in the case of Priyanka Amirneni, of Marshall. Priyanka, a senior at The Ellis School, said she values what she hears about colleges through her academic and social networks.
She said rankings weren't a factor in the schools she applied to, particularly because she feels they are too fluid. She placed a greater emphasis on college visits.
"I know that rankings go up and down," she said. "If a college is a reputable college, I guess you hear about them. That's good enough for me."
But even then, a school with a good "reputation" doesn't necessarily make it good fit for Priyanka, or any other individual.
"At some point, you have to question what does reputation mean?" Mr. Hawkins said. "Does it mean the college that does the best at what you're interested in?"
NACAC members generally agree that more emphasis should be placed on what students get out of their education -- such as graduation and retention rates, faculty resources and financial resources-- than the skills they bring to the school as evident in average scores on SAT and ACT college entrance exams, incoming freshman grade point averages, acceptance rates and so on.
Mr. Hawkins suggests pushing away from a system that creates a sense of order and rank with one college above another, which might not be supported by the reality of the best school for the individual, to a more expansive view of how to judge the quality of a college.
To do this, he explains, would require U.S. News and other ranking organizations to de-emphasize their editorial take on the methodology and allow individuals to determine the information about a school that matters most to them.
"What they are doing now is much too subjective," he said. "If they were able to tap into the vast public interest, they could almost turn it around."
He said students could become involved via an interactive Web tool that allows them to determine the percentage of a score devoted to reputation, graduation rates or other factors that matter to them in a college search.
"In some ways we're asking them to relinquish their grip on the Best Colleges mantra," he said. "You still get U.S. News data driving the process, but you get an endless array of individualized methodologies that allows you to see what people really want."
The verdict is out on whether this is the type of ranking approach for the future, but Mr. Morse said U.S. News is studying the possibility of providing a personalized rankings tool on usnews.com.
"It is appealing because it would give students and parents [the] ability to set priorities with the data," he said.
The challenge is creating a user-friendly interface.
Mr. Morse, whose ranking system is described by Mr. Hawkins as one his "members love to hate and hate to love," said that despite the fact that many colleges speak with "a forked tongue" about his rankings, they continue to use them.
"They are using our rankings as a validation or an indicator to the public as their standing against their peers," he said. "The institutions that we're ranking think [the rankings] are credible."