ESPN's new Total Quarterback Rating

[Arles]

ESPN is coming out with a new formula to grade QBs (they're running a special at 8 PM EST to go through it). Here's some links to how it works:

Explaining the statistics behind the Total Quarterback Rating - ESPN (http://espn.go.com/nfl/story/_/id/6833215/explaining-statistics-total-quarterback-rating)

NFL - Total Quarterback Rating shifts the way we see the position - ESPN (http://espn.go.com/nfl/story/_/id/6835090/nfl-total-quarterback-rating-shifts-way-see-position)

It seems like the goal is to isolate how well a QB plays and his effect on whether a team wins or loses. I'd be interested in what this community thinks of the new measure.

My initial comment is I don't like a QB getting "penalized" for hitting a WR on a slant or corner route where the WR runs for 50 yards to get a 70-yard gain. IMO, being able to not only pick the right WR to pass to and hit that WR in stride (even between coverages) is a skill that doesn't seem rewarded in this system. That said, it's a pretty minor gripe and it seems like a huge leap forward when compared to QB rating and other mainstream measures.

Here's the current rating list for 2010:
Rank Season QB Team Action Plays QBR
1 2010 Tom Brady NE 607 76.0
2 2010 Peyton Manning IND 779 69.5
3 2010 Matt Ryan ATL 709 68.6
4 2010 Aaron Rodgers GB 627 67.9
5 2010 Michael Vick PHI 547 66.6
6 2010 Drew Brees NO 760 65.9
7 2010 Eli Manning NYG 654 64.3
8 2010 Josh Freeman TB 626 63.5
9 2010 Philip Rivers SD 667 63.2
10 2010 Ben Roethlisberger PIT 500 59.8
11 2010 Tony Romo DAL 251 58.1
12 2010 Joe Flacco BAL 647 58.1
13 2010 Matt Schaub HOU 678 57.8
14 2010 David Garrard JAC 510 57.3
15 2010 Kerry Collins TEN 342 56.0
16 2010 Matt Cassel KC 566 51.2
17 2010 Ryan Fitzpatrick BUF 551 48.7
18 2010 Mark Sanchez NYJ 619 47.4
19 2010 Carson Palmer CIN 720 46.7
20 2010 Colt McCoy CLE 290 46.6
21 2010 Kyle Orton DEN 612 46.6
22 2010 Jon Kitna DAL 409 46.1
23 2010 Shaun Hill DET 499 44.8
24 2010 Jason Campbell OAK 479 43.8
25 2010 Jay Cutler CHI 596 42.6
26 2010 Matt Hasselbeck SEA 547 42.4
27 2010 Chad Henne MIA 604 41.4
28 2010 Donovan McNabb WAS 596 41.0
29 2010 Sam Bradford STL 732 41.0
30 2010 Alex Smith SF 426 40.0
31 2010 Derek Anderson ARI 387 35.9
32 2010 Brett Favre MIN 459 25.8
33 2010 Jimmy Clausen CAR 397 11.7

[Easy Mac]

perhaps adjust it to include yards only if a certain number of tackles are missed. i.e. if you hit a guy in stride and he doesn't have to break tackles, you're credited. Marshawn Lynch (I think, right) barrels through 6 or 7 guys, its on him.

[JediKooter]

So are they going to count interceptions against the QB, that should have clearly have been caught by the receiver, but, it bounced out of his hands and the defender ends up with the ball?

[Radii]

GO PANTHERS. Fuck.

[RPI-Fan]

This is no better (and is not necessarily worse) than Football Outsider's DVOA and related ratings. Except this one does not have an actual mathematical explanation offered as its basis.

[DanGarion]

This is no better (and is not necessarily worse) than Football Outsider's DVOA and related ratings. Except this one does not have an actual mathematical explanation offered as its basis.

MATH?! Pfft who needs it!

[JAG]

This is no better (and is not necessarily worse) than Football Outsider's DVOA and related ratings. Except this one does not have an actual mathematical explanation offered as its basis.

by Aaron Schatz :: Fri, 08/05/2011 - 12:22pm
Let's see if I can give my thoughts concisely.
The rating is based in part on a lot of FO theories -- remember, we were the first people to ever rate football players based on play-by-play analysis compared to a baseline for each play -- but takes things to the next level. In many cases, it is what we would do if I was able to pay 30 game charters every Sunday instead of relying on volunteers who have things like "school" and "work" and "families" to deal with.
The analysis of how much to attribute YAC to the quarterback as opposed to the receiver is very similar to the study we did on how to attribute rushing yards to a back vs. the offensive line for Adjusted Line Yards. It's something I've always wanted to do and just never had the time. It's not just subjective.
I have two problems with the rating.
1) The lack of opponent adjustments, although I understand why they aren't including them. Remember, they are trying to produce ratings on the fly, whereas I produce ratings after all the games of a certain week are over. And you know even we don't include opponent adjustments early in the season, and we slowly ramp up the strength of those adjustments. So I understand their arguments against including them.
2) The "clutch" thing. I still think this gives a bonus to quarterbacks who happen to play with bad defenses. But they say they have worked it out so that no quarterback gets a bonus for BEING in more "clutch" situations. They only get a bonus for playing well in those high-leverage situations. Still, we've found that offensive DVOA in first quarter correlates best with winning, not other quarters. There is an advantage to getting off to an early lead, dictating the pace of the game, and an even bigger advantage to blowing your opponent away out of the gate.
To answer Jeff's question in comment 31, yes, quarterback B will have the higher Total QBR. I actually asked Dean Oliver this specific question.
It's not really supposed to be predictive. I asked Dean about that, and he said "We both know that the best predictive stat isn't a stat, it's an algorithm." Which is true.

[kcchief19]

Here's a smell teat question for this rating vs. the traditional QB rating. Was Peyton Manning really the second beat QB in the league last year? From a number, performance, wins and observation stand point, I don't last season was anywhere near his best. By the traditional rating, he was the ninth best QB. ESPN says he's number two.

The traditional rating stinks, but I haven't read enough to see this is a major improvement. In some ways, as noted above, it may overvalue the so-called clutch rating and not reward QBs who put games away early and don't end up in close and late situations.

[kingfc22]

Glad we are sticking with Alex Smith....AGAIN:banghead: :banghead: :banghead:

[stevew]

Eli Manning throws 19 more interceptions than Freeman, and 20 more than Big Ben, yet is ranked ahead of them? 25 picks is Favre-ian, and the rating already doesn't pass the smell test.

[Lathum]

I'm on board with any rating that puts Eli ahead of Romo, Rivers and BigBen.

[albionmoonlight]

THE LIST:
__________________________

(1) {Every Other Quarterback in the NFL}
(2) Jimmy Clausen
_____________________________

Looks spot-on to me.

[sabotai]

Eli Manning throws 19 more interceptions than Freeman, and 20 more than Big Ben, yet is ranked ahead of them? 25 picks is Favre-ian, and the rating already doesn't pass the smell test.

Ignoring that Eli had better stats than them in most other categories.....

Many of Eli's picks last year were balls that were tipped into the air by his own receivers. The whole point of doing stats the way the people at FO do them is to get beneath the stat sheets and to access credit and blame appropriately. A QB who throws 15 INTs all straight into the hands of the defenders performed worse than a QB who throws 20 picks with half of them bouncing off the hands of his receivers (read: should have been caught) and being snagged by the defenders.

[Lathum]

We all know I am completely biased when it comes to the Giants, but a huge portion of Elis picks last year were the receivers fault

[BillJasper]

Ignoring that Eli had better stats than them in most other categories.....

Many of Eli's picks last year were balls that were tipped into the air by his own receivers. The whole point of doing stats the way the people at FO do them is to get beneath the stat sheets and to access credit and blame appropriately. A QB who throws 15 INTs all straight into the hands of the defenders performed worse than a QB who throws 20 picks with half of them bouncing off the hands of his receivers (read: should have been caught) and being snagged by the defenders.

But aren't you really beginning to get into personal interpretation when you start trying to assign blame for interceptions on tipped passes.

Hell... you could have an absolutely terribly thrown ball that the receiver somehow manages to get his fingers on, but not catch and that turns into an INT.

Who do you blame? In the NFL we still can't even decides what constitutes a catch and yet we're going to try to base stats on personal interpretation of whether or not the ball is well thrown?

EDIT: Peyton Manning being number 2 just doesn't work for me. Yeah he led a lot of late comebacks, but many of those deficits were created by him earlier in the game.

[sabotai]

But aren't you really beginning to get into personal interpretation when you start trying to assign blame for interceptions on tipped passes.

Which is why, hopefully, they have more than 1 person doing it. But since football plays 16 games instead 162, just having counts and averages for what happened isn't a very good measure of how well a player played.

It's also impossible to tell if a QB made a bad pass or if the receiver ran the wrong route (unless after the game if one decides to throw the other under the bus). It's impossible to tell if a running back ran his assignment poorly of it the OL blocked the wrong guy. This is why I doubt there will ever be anything close to what baseball or even basketball and hockey have when it comes to stats.

But having said that, just counting how many times something happened (a TD pass, an interception, etc.) and saying that the QB is 100% responsible for each and every one is probably the worst way to do it. If they played 162 games a year, I'd be fine with it since most would average out over time, but a football player would be lucky to get near that number for his career, so doing something to assign credit and blame to plays would be a better.

Even if you did something as simple as say "If the ball hits the receiver in the hands and it ends up being picked, regardless of how or why the ball got there, 50% blame to QB to cover the times it was a bad throw" would be better. The same goes for TD passes. I doubt the QB should ever get any credit for a TD scored on a screen or shuffle pass, but in the traditional QB Rating, a QB who gets 10 TD passes on screen plays gets 100% credit for those, the same as a QB who throws 10 perfectly placed passes into the endzone.

[BillJasper]

Even if you did something as simple as say "If the ball hits the receiver in the hands and it ends up being picked, regardless of how or why the ball got there, 50% blame to QB to cover the times it was a bad throw" would be better. The same goes for TD passes. I doubt the QB should ever get any credit for a TD scored on a screen or shuffle pass, but in the traditional QB Rating, a QB who gets 10 TD passes on screen plays gets 100% credit for those, the same as a QB who throws 10 perfectly placed passes into the endzone.

One, the play they showed for it being the receivers fault was, quite frankly, bullshit. Eli Manning to Hakeem Nicks... when Manning threw the ball, there were three Cowboys defenders within two yards of the ball. Yeah Nicks should have probably caught it (and would've gotten blown up), but Manning should have never thrown it to begin with.

Which brings me to decision making, I'm watching on ESPN saying the QB shouldn't get credit for 'bubble screens' but I disagree. The QB still has to make the correct read and get the ball to the receiver. Then how do you weight the QB's ability to get his team into the correct play when the defense is in an alignment you didn't count on?

Then there are the anomalies of the position like a Peyton Manning or Michael Vick.

I think there has to be a better way to rate QB's, I just don't think ESPN came close to finding it.

EDIT: Why can't you simply break the current QB rating down by quarter, red zone and third down categories? Then weight them by importance.

[larrymcg421]

It seems as though Eli's Interceptions were heavily balanced to blowouts. He threw 15 in 7 games decided by 15 or more points (either way). However, he only threw 5 in 4 games that were decided by 7 points or less. Given the way the formula works, it seems as though the system simply doesn't penalize him that heavily for his 25 INT's because they mostly happened in situations where that didn't hurt his team's chances that much.

[DaddyTorgo]

I don't buy their new rating. I think they just wanted something to try to generate some press and to have a stat that would get people to tune in to their different broadcasts while they debate how good it is among the analysts that have actually played and those who are just talking heads.

Basically...just another ESPN gimmick. Except this one since they created it they can "trademark" it and try to make some dough off of it.

[Noop]

Chad Henne sucks.

[BillJasper]

Chad Henne sucks.

Bet you figured that out without any statistical data at all! :lol:

[Blackadar]

It's lousy statistics, plain and simple. You can't have things like "expected points" or "dividing credit" in a true statistical analysis. There's far too much subjectivity in this to be considered anything but a gimmick. Low ball? Hmmm...was it a bad throw, a safe throw to keep the defense from picking it or does the WR just like the ball low? Before, it's a catch. Now, someone is going to try to subjectively evaluate that? Uh, no.

The GIGO principle is at work here in a large way. As such, this is a pretty worthless stat.