Playoff system maybe not the slam dunk the NCAA wants
January 5, 2014 11:49 PM
David J. Phillip/Associated Press
Auburn head coach Gus Malzahn, left, and Florida State head coach Jimbo Fisher shake hands in front of The Coaches' Trophy during a news conference Sunday for the BCS National Championship in Newport Beach, Calif. Florida State plays Auburn today.
By Mark Dent / Pittsburgh Post-Gazette
It's OK as a college football fan to admit you got so mad at the Bowl Championship Series these past 15 years that you cursed furiously at an object so vague and inanimate that it wasn't actually an object, just a series of rules comprising an algorithm.
So many of us did. Utah's attorney general threatened a lawsuit. Congress formed a Playoff PAC. Then-candidate Barack Obama discussed a need to lobby for a college football playoff as part of his 2008 presidential platform, focusing his wrath on the faceless, non-breathing aspect of the system he loathed.
"You know, I'm fed up with these computer rankings," he said in an ESPN interview in 2009.
Always the computers. When people thought BCS, they thought computers. And when people thought BCS, they thought of the corruption of a strained, authoritarian government puttering along desperately and arbitrarily as it attempted to maintain control over a system everyone wanted to see gone in favor of a playoff.
After tonight's championship matchup between Auburn and Florida State is played at the Rose Bowl in Pasadena, Calif., the BCS is gone, replaced by a four-team playoff for next year that is almost unanimously regarded as a better alternative.
But the computers are gone, too. College football has handed complete power to 13 humans, known as the College Football Playoff Committee, spearheaded by Bill Hancock, the guy in charge of the BCS for the past four years. The committee members have been charged to use whatever means they believe to be most useful to select the four best teams.
The rest of society thrives using quantitative analysis. Companies comb for big data to best suit their products for consumers. Statistician and writer Nate Silver's election projections were nearly dead-on and upset the establishment. College football, though, is moving away from these practices, writing a neo-romantic script, leading to a question of whether the sport has cast too much blame on quantitative analysis for its past problems and is not emphasizing it enough as the playoff system begins.
"I thought the computers were always put in the position of the fall guys," said Hal Stern, a statistics professor at the University of California-Irvine who, in a 2006 paper, implored all quantitative analysts to disassociate from the BCS because of the system's mismanagement. "If the two polls [writers' and coaches'] disagree with the computers, then it's always the computers' fault."
Five of the people in charge of formulating the six computer rating components of the BCS -- Ken Massey, Jeff Sagarin, Richard Billingsley, Jeff Anderson and Wes Colley -- spoke with the Pittsburgh Post-Gazette. Not wanting to sound bitter, they do have a message: Life might not be as easy without the computers.
"We were never intended to really contribute to the formula, just to be a lightning rod," Massey said. "Anytime something went wrong, they would have someone to blame."
Sports journalist Dan Wetzel, author of "Death to the BCS," co-written with Jeff Passan and Josh Peter, would prefer a 16-team playoff, but with the conditions set as they are for the new four-team playoff, he would suggest having a computer rating system select the teams. Let one transparent formula grade the four best teams. Let every fan, coach and player know what will lead to a top ranking.
"That obviously has zero traction," Wetzel said. " ... I find the people that run college athletics to not really be into the college part."
The people in charge of the BCS -- and they are now in charge of the College Football Playoff Committee -- weren't just averse to academia. In recent years, they arguably committed atrocities against the concept of mathematics.
The trouble started in 2000 after Roy Kramer, the first BCS director, stepped down. Two years in a row, first with the inclusion of Florida State in the national title game in 2000 and then with Nebraska in 2001, the BCS faced controversy because the human polls had other teams ranked higher.
The public outcry was loud enough that Mike Tranghese, then-director of the BCS and now a member of the College Football Playoff Committee, ordered all the rating systems to remove margin of victory from their formulas. Anderson's, Hester's and Colley's rating systems didn't account for margin of victory. Anderson said he still thinks the change made the BCS better.
Other computer pollsters such as David Rothman and Herman Matthews saw manipulation and withdrew their ratings after Tranghese's decision. Because people didn't trust the computers, the BCS essentially neutered them. Stern, the California-Irvine professor, describes the decision to remove margin of victory as a means for the BCS to "achieve whatever they wanted."
Sagarin, who computes what he considers a superior ranking that includes margin of victory on the USA Today website, said a win-loss method without margin of victory tends to agree with the human polls, which account for two-thirds of a team's overall BCS standing. Sagarin said Kramer wanted computer ratings that would balance out the bias of the polls, and Tranghese's decision countered that purpose.
Fans and the media trusted the polls, not the quantitative analysis. The computers were now programmed to help the BCS avoid controversy.
Despite the top two teams in the final BCS standings matching the top two teams in the polls every year since 2004, the media has continued to make the same, often circular, complaints against the computers, chastising them at times for not including margin of victory.
"The media were the people who caused that to happen," Massey said.
The advantages of computers were rarely discussed, partially because computer rating systems are complicated and unexciting for most. Stern said they lack the built-in biases that, for instance, lead poll voters to believe historical powers like Oklahoma will be better than Baylor. The computer ratings can better account for complicated variables like strength of schedule and better predict and measure the quality of a team that is unheralded in the human polls. In the first week of the BCS standings this season, for instance, the average computer rating of Auburn was No. 7 while the other polls rated it No. 16.
The rating systems had flaws on their own, too, aside from the BCS' mismanagement. Billingsley's formula has been criticized for carrying over a team's end-of-year rating from the previous season into the next season regardless of personnel changes. Colley missed a score while entering his final ratings of 2010, causing LSU to be ranked 10th in the BCS standings and Boise State 11th when it should have been the other way around.
Colley is the only person who releases the formula for his ratings publicly; others don't for proprietary purposes. Transparency has never been the BCS' strong suit.
The confusing new order
The College Football Playoff Committee has revealed little about its plans. It is made up almost entirely of former or current college athletic administrators or coaches, aside from former Secretary of State Condoleezza Rice and retired journalist Steve Wieberg. Hancock, executive director of the College Football Playoff, has promised integrity and transparency -- eventually.
"We will be as transparent as possible," he said in an October teleconference. "That was one of the first steps that our management committee took when they created the playoff, was insisting on transparency. Now, we don't know what that means exactly, but there will be a defined set of procedures that will be made public."
Jeff Long, committee chairman and former Pitt athletic director, said on the teleconference the committee will have a "bunch of data" to digest and analyze. Colley questioned the efficacy of such a practice if there is no clear baseline -- the BCS had six defined computer rating systems -- or whether the committee, which lacks anyone with a background in quantitative analysis, could properly utilize the data they gather.
The lone clear method of the committee is one known well to college football fans: the poll. Each member of the committee will make individual ballots throughout the year -- they will not be released publicly -- to form a single poll of the top 25 teams.
"The idea of ranking your teams alone is ... I don't know," Wetzel said. "I don't even know what to say to those guys anymore. It's like people telling you two plus two is five and slapping each other on the back."
Next season, the College Football Playoff Committee will have no computers to fall back on -- or blame. Sagarin sees the removal of the computer ratings as a move away from fairness. Computers don't cave to politics. Humans do.
"There is going to be huge institutional pressure on that committee not to rock the boat," Sagarin said. "And the boat is the boat that's created by the media."
Done the right way, Sagarin, Stern, Massey, Anderson and the other quantitative analysts say there is a place for computer ratings in college football's newest postseason incarnation, as an option or even as a necessity to prevent too much bias and politics from creeping in. Colley said fans have gotten smarter and more open to data, and less likely to send him emails questioning his ability to evaluate sports and tell him to "go back to playing Magic."
To report inappropriate comments, abuse and/or repeat offenders, please send an email to
firstname.lastname@example.org and include a link to the article and a copy of the comment. Your report will be reviewed in a timely manner.