Ratings Vs. Real Performance

[bmerryman]

I read somewhere in here a quote from Jim basically saying you should look at the stats as well as the ratings of the players. It's easy to know when your QB is underperforming because the QB rating is a very good measure in my opinion. I looked at some of the other stats and thought, "I have no idea what a good sack % is for a guard. Or, what the hell is a good tackle % for a DE?"

So, I took 4-5 years of statstics from approximately 200-300 starters per position in the interior line positions (C,G,T,DT,DE). I'm a fat ex-lineman and have more than a few soft-spots for those guys. I chose 5-6 year increments (like 2007,2012,2017,2023). I'm far from a statistician but it seemed appropriate to avoid analyzing data from basically the same set of starters in four consecutive years. I made a crude excel calculator that I use to rate the actual performance of my lines relative to the 4 or 5 years of statistics that I gathered. Here's a current chart for one of my MP teams:

			Game 1 -2007			Game 10 -2007
Player #	Pos	Exp	Current Estimate	Future	2006 Stat Based	Current Estimate	Future	2007 Stat Based
Willis, Ernie (M)	C	9	17	19	N/A	17	19	N/A
Grijalva, Cornell	C	3	63	89	29	71	89	65
Wonders, Claude	C	2	29	56	32	32	56	13
Gerhardt, Brant	LG	7	61	61	36	64	64	64
Frederick, Damon	RG	12	81	81	65	81	81	79
Clemons, Willie	LT	5	53	55	47	61	61	60
Bennetts, Rusty (M)	RT	10	26	26	65	26	26	55
Fulcher, Brad	RT	2	47	71	38	61	71	65
Schwartz, Bernie	RT	2	21	36	81	22	36	26
Disbro, Deion	P	1	32	48		34	48
George, J.B.	K	2	38	54		44	54
Copeland, Les	RDE	7	29	29	71	29	29	45
McWilliams, Joseph	LDE	9	43	43	54	43	43	81
Gordon, Ian	LDE	3	20	37	17	20	37	24
Amari, Mike	RDE	4	38	46	26	42	46	66
Klech, Butch	LDT	5	42	52	48	46	46	37
Mena, Marshall	LDT	2	30	58	39	33	58	66
Cooper, William (M)	RDT	7	41	41	63	41	41	36

As you can see, there are some differences between my scout's ratings of some of my players and their actual performance. We all know this occurs. SkyDog documented the incredible QB rating of a guy who's scouted rating never seemed to crack 50/50. I picked up (LDE) Les Copeland as a free agent prior to week 1 because his Stat-Based Rating the prior year was so high - even though his scouted ratings were and are very low. Now, he hasn't performed at last year's level but he's still a servicable cheap fill-in that on the surface appears to be awful. You can also see that my veteran backup Tackle Rusty Bennetts' Stat-Based Rating is quite a bit higher than his scouted rating.

Obviously, I've only been doing this a few weeks and the Stat-Based Ratings may end up being meaningless. In 6 more weeks their ratings will probably be very different but my guess is that over time I'll be able to better identify who's over/under rated.

Thus far, my D-Line has performed very well in this league with the #1 PrPct, #5 Rushing D, #12 Passing D. The Stat-Based Ratings seem to reflect that fact, and help me identify who's actually contributing and who isn't. It's easy for me to identify why my team is 4-5 - it's because my quarterback's QB rating is 65.6! Damn him.

I'm interested in your thoughts/opinions. Does this approach help us understand our players more?

[gstelmack]

The problem you may have here is that stats for offensive linemen aren't very accurate. See the KRB/KRO thread. Defensive Linemen should be a bit better.

[bmerryman]

That thread has a lot of speculation about what KRO/KRB mean but I'm not sold on any particular theory. From what I can tell, generally, highly rated run blocking lineman get more KRO's which is probably a good thing for the team, and is a stat that's clearly tracked. I'll check this a little closer. Thanks for the feedback g.

[bmerryman]

As I suspected KRO% appears to be more closely related to good run blocking linemen than KRB%. In fact, KRB% appears to be fairly random but does trend toward highly rated linemen. Here's what a quick study of 144 linemen found:

KRB%
	Top 25%	26-50	51-75	76-100
Median Rating	61.5	56.0	54.5	49.5
Average Rating	56.9	55.9	52.4	53.1
KRO %
	Top 25%	26-50	51-75	76-100
Median Rating	73.0	54.5	44.5	32.5
Average Rating	73.4	68.3	43.1	34.4

[bmerryman]

I'm going to check to see if the team's overall running statistics are related. I'll respond to myself once I get an indication!

[bmerryman]

Here's the news: An individual lineman's KRO% is not related to overall team yards per carry. In other words, any particular lineman that is a part of a team that has a high yards per carry average generally does not have a higher KRO% when compared to a lineman on a team with a low average yards per carry.

KRO%
	Top 25	25-50	51-75	76-100
Team Median YPC	3.98	4.16	4.17	4.02
Team Avg YPC	3.97	4.08	4.03	3.99

However, KRB% is related to overall team yards per carry. In other words, a team with a high yards per carry average does have a higher KRB%.

KRB%
	Top 25	25-50	51-75	76-100
Team Median YPC	4.32	4.06	3.96	3.64
Team Avg YPC	4.31	4.05	4.00	3.73

We've already identified that linemen with higher run block ratings tend to have a higher KRO%; therefore, this suggests that KRO's are "created" by good linemen, while KRB fulfillments are probably tied to good running backs.

This leads me to believe we should focus on KRO% to identify run blocking performance of our O-Lines. Sack percentage is an obvious pass blocking performance measure. I also factor in Pancake %. I haven't tested pancakes but they overtly seem to indicate a blocker's strength.

When I rate a lineman, I use a weighted average of all 3 of these. I factor sack% slightly less for Centers and Guards when compared to Tackles. On defense I factor Pass Rush slightly more for DE's versus DT's.

Comments/thoughts?

[QuikSand]

On OL stats -- I believe the evidence is perfectly clear that the presence of a "key run block" on a given play is 100% a function of the outcome of the play, and not just the effectiveness of a particular blocker. On a successful standard running play, there is a KRB awarded, and on an unsuccessful running play, there is no KRB awarded. Period.

Any sense that this is an OL-only stat is completely upended by this. That doesn't mean it's a useless stat -- but it does mean that using it to compare across teams (and maybe even across positions) is getting into pretty shaky territory, since you are inherenbtly adding in factors that trace back to a player or players other than just the particular OL -- even on a play when RT Jones was selected for the KRO and received the KRB, it's not clear that the effectice play was due to a good block by that one lineman, good overall blocking by the entire run-blocking group (all the OL and the FB/TE), a brilliant move by the ball carrier, or just a short distance needed to declare "success" by the game -- or in many cases, some combination of those things and maybe more.


As for using KRO% as an indicator -- I'm worried that this stat is team dependent as well. Let's say you're right (I have no reason to doubt this) that an OL with high run blocking skills gets more KRO chances than a player with lower skills. Okay -- but consider a 60-rated LG on a team with everyone else rated 40 -- he'll have a high KRO%. Compare him to an (in truth) equally-talented and equally effective 60-rated LG on another team, where he is surrounded by superior players, all rated 80. This guy would likely have a far lower KRO%, since he's the weak link on that strong team, rather than the strong link on a weak team. With relative, team-dependent issues like this, it again becomes awfully difficult to rely on these stats when measured across teams, as you're inherenty picking up residual effects of the broader situation.


I applaud the thinking here and the direction of your effort. My fear is that by starting to understand OL stats better, we're losing faith in what we can do with them. And it's possible that more digging into stats of DL or other positions might well yield the same sort of dispiriting results.

[Vinatieri for Prez]

I'm with QS. I applaud the work done. Everything is appreciated to better understand the game. I agree with all his conclusions (lord knows we discussed them at depth in the KRB thread), but I would add some add with this:

- a player's run blocking stats have a large team element to them; thus they are good for comparing players on the same team, but not for players on different teams

- pancake blocks are an independent player stat on which to compare players from different teams

The first one has some serious testing behind it. The second one not as much testing, but I firmly believe it to be the case.

[CU Tiger]

I agree with QS and V4P
And would also add that KRO, has a very strong tie to play calling.

Simply put a team that runs more will have a larger sample size of KRO to distribute amongst its 5 linemen.

[Sgran]

What has always confused me is that a lineman can get a key run block on a play in the opposite direction. Yes, guards pull, but tackles usually don't (correct me if I'm wrong), and you'll sometimes see a right tackle get a KRB on a sweep left. Could this mean that his block had the highest degree of difficulty? Like he disrupted Shawn Merriman's backside pursuit? Or does everyone chalk it up to a random roll for who gets credit for the run (since clearly noone botched an assignment)?

[Vinatieri for Prez]

A random roll is what I believe. Although it is not entirely random as the chances for each player are relative to their hidden true ratings.