Scout Error

[aston217]

Correct me if I am wrong -- a summary of what we know about scout error:

* It affects veterans more than it affects rookies
* For veterans the difference can be as much as 15% between two scouts

I don't see any reason to doubt this. However, I do think that something very different happens when you are talking about players on your own team, of any year. I had the chance to take a look at data from Changetracker run by both me and another member in the league.

I put my Utah players and their two changetracker data sets side-by-side and calculated the difference in ratings seen by his scout and mine, for each of the four columns: PRE-TC CUR, PRE-TC FUT, POST-TC CUR, POST-TC FUT.

70 players in all
* 57 with discrepancies in one or more of the four columns
* 12 of the 57, my scout saw higher
>>>Of these 12, 10 of them are droppers (either TC or negative cut test).
* 45 of the 57, my scout saw lower
>>>Of these 45, 44 I know to be creepers. The lone exception, I think has been about as scouted, although it's hard to tell. If he is moving in either direction, it isn't much.

thenewchuckd has always liked hearing what other people saw his rookies as, as he saw it as kind of similar to the cut test.

Now it seems that is not only pretty accurate, but it also applies to everybody on your team.

There doesn't seem to be nearly as much rhyme or reason to it when looking at the FA list or other teams. I did check another team and while I don't know the history of his players, a look at the rookies:

9 rookies in total on Team #2 (St. Paul Bandits)
* 2 rookies my scout saw higher
>>>Of these two, one had a -5 camp and one had a -4.
* 5 rookies his scout saw higher
>>>Of these four, the camp results were -2, +4, +8, -9, and -4.
* Of the two where there were no discrepancies, the camps were -1 and -2.

Just for comparison, the 13 rookies on my team
* All 13 had discrepancies.
* 6 rookies my scout saw higher.
>>>My camp results: -3, -3, -2, -1, +1, -1. The +1 I know for sure isn't that accurate, as the guy has a -1 cut test.
* 7 rookies my scout saw lower.
>>>My camp results: +3, +1, +2, +3, +6, +5, +3. All of these, creeping, per the cut test.

Maybe it is common knowledge to others, but I wasn't ever too sure how "solid" of a sign it was when another GM saw your rookies rated higher. Now I am fairly confident, and the side note is just that it seems to apply to your entire roster.

Do our own scouts tend to see masked players extra masked, throughout their careers, compared to everybody else? If the latter, it has the added consequence of knowing when you see a discrepancy for another team's player on a trade block vs your scout, it might not be scout error at play, but a sign of which direction he is going.

A note on scout ratings
There were four discrepancies in the scout-scout comparison for my roster: the three guys who didn't follow suit (i.e, the 2 of 12 and the 1 of 45), and another guy that we both saw the same, even though he is for sure a significant creeper.

It's worth noting that these are all offensive linemen, and my scout is rated Poor there, while his is rated Good. I dunno, just a note, I don't have definite thoughts about this.

In brief: Thinking about "scout error" and checking your scout's skills may only be useful when you are talking about free agents or players on a 3rd-party team. If you are comparing how two scouts see guys that are on one of the two teams, it *appears* that you're dealing with masking. Maybe not exclusively, but it does seem to be the larger factor.

[Ben E Lou]

Scout error implementation is just plain wonky. The most concrete and obvious is example is that the better your scout is with youngsters, the LEAST accurate he will be on young masked players. For example, my heavily masked IHOF QB.

MY SCOUT'S VIEW: (VG QB, VG Young Talent..used to be Exc. in both) QB Jackie "Big-Time" Collier Player Details (http://108.59.255.76/~benelou/ihof/ben/playercard.php?playerid=24969)

CUERVO'S SCOUT'S VIEW: (confirmed by a couple of others over time to be closer to what they see than my guy) http://www.fof-ihof.com/players/player.php?player=24969

[Ben E Lou]

Dola:

Based on his combines and the continued creeping, it's obvious that Collier is better than even the 78/78 that Cuerv's so-so scout is seeing, most definitely better than the 74/74 that my ZOMFG TEH AWESOME scout sees. *shurg* I suspect that the same may be true for those vets that your guy sees as 65/65 and the rest of the league sees as 57/57.

[aston217]

It seems to me that it doesn't particularly matter how good or bad your scout is: he will be bad, in the wrong direction, on masked players, relative to scouts on other teams, again regardless of how good or bad they are.

And all of that may only apply to guys on your own team.

It would be tough to pin down for sure, but I suspect that if looked at some solidly masked creeping rookies, your scout would see them as better than the host team's scout, even if the host team scout is really bad.

That might be something to test out, as you have an awesome scout -- find someone with a really bad one, and check their latest batch of rookies vs yours. Mine is a pretty mixed bag, and so is the one I compared it to (although, both were VG in YT).

[Ben E Lou]

It seems to me that it doesn't particularly matter how good or bad your scout is: he will be bad, in the wrong direction, on masked players, relative to scouts on other teams, again regardless of how good or bad they are.Well, you're wrong then. Your scout always will be worse than opposing scouts on masked guys on his own team, which is problematic. But what's worse is that the better the scout is, the worse he sees masked guys on his own team. That's just silly. I have seen this at work in multiple leagues on many players because over time I virtually always have one of the top handful of staffs in the league (mainly due to another wonky implementation--the financials).

[aston217]

Your scout always will be worse than opposing scouts on masked guys on his own team, which is problematic.


Oh, OK. That is all I was saying. You might be right that the better the scout is, the worse this effect becomes, although I wonder what you would have as a basis for comparison if you always have the best staffs.

I guess the main point is that this effect exists and I think you nailed why it works that way in the quote.

[Ben E Lou]

although I wonder what you would have as a basis for comparison if you always have the best staffs.By comparing the deltas with other owners. Mine are almost always greater for my great scouts than those that other owners see, which is a pretty obvious indicator. (Look at the 6-point delta there for Collier. How often do you see that?) I was already about 99.5% sure this was the case, and then I verified it by using a quick test MP league (just draft a bunch of obvious mega-creepers on one team, then sign a horrible scout and see how he's closer to their real ratings.)

[aston217]

That's interesting, and it makes what I noticed about OL make more sense (my scout being 'poor' there.)

Wonky, eh? (good way to describe it, I think) you want your scouts to be good for draft time so you see narrower bars, but then they get bad at judging guys once they hit your team. At least it is predictably bad, which ends up being a good thing again.

This should make it extra important to pay attention to who is creeping on your roster and who is dropping -- as your scout may be overestimating the 45/45 dropper and underestimating the 35/35 creeper to the point where they are equivalent. Or much closer than they look.