STRATEGY: The importance of interviews

[Ben E Lou]

OK. I don't have a huge sample size yet for 6.1, but given the three info points below, I'm pretty confident that the stuff I'm about to present is accurate:

1. I tracked lots of info on my draftees for 30 years in 6.0e.
2. There's no indication in the change log that interviews are any different in 6.1.
3. Having compiled roughly 1/6 of the amount of data that I had for 6.0e, trends are indicating that interviews are the same in 6.1.

So, that being said, here goes.

Here's a 33-player set of draftees in 6.1. Now, keep in mind that my career is currently in 2012, so the 2007 draftees are nearly fully unmasked, and the 2011 draftees are only beginning to unmask. However, it's pretty clear that the mechanisms involved are largely or fully unchanged, so I feel safe making some assumptions at this point about where these guys are headed. In the charts below, Cur1 and Fut1 represent the initial scouted ratings when on the roster. Cur2 and Fut2 represent the results of the first training camp. MaxCur and MaxFut represent the ratings as of now (FA1-1, 2012). 1stCamp is just a calculation: it's the delta in future potential observed in the initial training camp.

YR	Pick	POS	LAST	FIRST	Imp	Cur1	Fut1	Cur2	Fut2	MaxCur	MaxFut	CurChg	FutChg	1stCamp
2007	1(10)	CB	Lutt	Andrew	UR	57	72	55	72	76	76	19	4	0
2007	2(9)	QB	Reese	Scott	VU	20	54	23	48	37	37	17	-17	-6
2007	3(16)	T	Butler	Jeff	UR	27	40	28	42	48	48	21	8	2
2007	4(15)	TE	St. Louis	Charles	xxx	18	52	19	46	29	29	11	-23	-6
2007	5(14)	S	Daggett	Lewis	xxx	9	22	11	26	34	34	25	12	4
2007	6(13)	C	Blanchard	Richie	xxx	25	58	28	60	64	64	39	6	2
2007	7(12)	WR	Davis	Maurice	xxx	15	27	15	24	16	21	1	-6	-3
2008	1(20)	DE	Thompson	Jerome	VU	24	50	24	53	63	63	39	13	3
2008	2(24)	G (was C)	Madison	Corwin	VU	28	61	31	63	71	71	43	10	2
2008	3(23)	OLB	Ellard	Brett	VU	12	33	15	40	53	58	41	25	7
2008	4(22)	FB	Meyer	Wesley	xxx	33	51	35	52	56	56	23	5	1
2008	5(21)	G	Blanchard	Dwayne	xxx	16	39	18	39	26	41	10	2	0
2008	6(20)	QB	Benson	Sammy	VO	15	44	15	39	22	28	7	-16	-5
2008	7(24)	T (was C)	Sauer	Walt	xxx	18	43	18	38	16	33	-2	-10	-5
2009	1(23)	WR	Reeves	Junior	VU	19	39	22	45	54	54	35	15	6
2009	2(22)	S	Forbes	Hardy	VU	25	35	29	40	53	53	28	18	5
2009	3(25)	P	Holliday	Zack	UR	71	71	61	63	72	72	1	1	-8
2009	4(24)	G	Babin	Frankie	xxx	18	52	21	54	51	58	33	6	2
2009	5(23)	QB	Shinners	Lamar	VO	16	44	16	42	16	41	0	-3	-2
2009	6(22)	DT	Collier	Reuben	HTR	26	33	30	40	45	45	19	12	7
2009	7(25)	ILB	Reese	Ronald	OR	18	34	18	33	19	32	1	-2	-1
2010	1(32)	WR	Floyd	Willie	UR	23	44	24	47	51	51	28	7	3
2010	2(32)	OLB	Farrell	Fernando	UR	24	38	28	43	50	51	26	13	5
2010	3(32)	QB	Willow	Drew	HTR	18	46	17	44	16	42	-2	-4	-2
2010	4(32)	WR	Ferguson	Terrance	HTR	13	20	16	26	28	34	15	14	6
2010	5(32)	S	Oliveira	Xavier	xxx	10	21	13	26	24	30	14	9	5
2010	6(32)	RB	Pirtle	Jamie	UR	23	29	24	31	28	35	5	6	2
2010	7(32)	G	Newton	T.J.	HTR	16	45	16	41	19	36	3	-9	-4
2011	1(31)	OLB	Hutto	Horace	VU	18	29	23	39	30	46	12	17	10
2011	2(31)	OLB-->ILB	Henson	Vince	UR	23	51	27	51	31	51	8	0	0
2011	3(31)	QB	Burton	Kent	HTR	22	59	21	52	20	47	-2	-12	-7
2011	4(31)	C-->T	Langlois	Rodney	xxx	15	67	18	60	26	58	11	-9	-7
2011	5(31)	C-->DT	Barlow	Larry	xxx	17	55	20	52	21	51	4	-4	-3

I hit some. I missed some. Average ratings were 22.2/44.2, 23.6/44.6, 38.3/46.9. Overall my guys are gaining more than they lose, by not by much. Average change in first training camp is +0.39 points. I'm doing ok, but certainly could stand to improve. This is a nice chart, but it doesn't tell us all that much sorted like that. However, sorting it differently tells a story worth noting:

YR	Pick	POS	LAST	FIRST	Imp	Cur1	Fut1	Cur2	Fut2	MaxCur	MaxFut	CurChg	FutChg	1stCamp
2011	1(31)	OLB	Hutto	Horace	VU	18	29	23	39	30	46	12	17	10
2008	3(23)	OLB	Ellard	Brett	VU	12	33	15	40	53	58	41	25	7
2009	6(22)	DT	Collier	Reuben	HTR	26	33	30	40	45	45	19	12	7
2009	1(23)	WR	Reeves	Junior	VU	19	39	22	45	54	54	35	15	6
2010	4(32)	WR	Ferguson	Terrance	HTR	13	20	16	26	28	34	15	14	6
2009	2(22)	S	Forbes	Hardy	VU	25	35	29	40	53	53	28	18	5
2010	2(32)	OLB	Farrell	Fernando	UR	24	38	28	43	50	51	26	13	5
2010	5(32)	S	Oliveira	Xavier	xxx	10	21	13	26	24	30	14	9	5
2007	5(14)	S	Daggett	Lewis	xxx	9	22	11	26	34	34	25	12	4
2008	1(20)	DE	Thompson	Jerome	VU	24	50	24	53	63	63	39	13	3
2010	1(32)	WR	Floyd	Willie	UR	23	44	24	47	51	51	28	7	3
2007	3(16)	T	Butler	Jeff	UR	27	40	28	42	48	48	21	8	2
2007	6(13)	C	Blanchard	Richie	xxx	25	58	28	60	64	64	39	6	2
2008	2(24)	G (was C)	Madison	Corwin	VU	28	61	31	63	71	71	43	10	2
2009	4(24)	G	Babin	Frankie	xxx	18	52	21	54	51	58	33	6	2
2010	6(32)	RB	Pirtle	Jamie	UR	23	29	24	31	28	35	5	6	2
2008	4(22)	FB	Meyer	Wesley	xxx	33	51	35	52	56	56	23	5	1
2007	1(10)	CB	Lutt	Andrew	UR	57	72	55	72	76	76	19	4	0
2008	5(21)	G	Blanchard	Dwayne	xxx	16	39	18	39	26	41	10	2	0
2011	2(31)	OLB-->ILB	Henson	Vince	UR	23	51	27	51	31	51	8	0	0
2009	7(25)	ILB	Reese	Ronald	OR	18	34	18	33	19	32	1	-2	-1
2009	5(23)	QB	Shinners	Lamar	VO	16	44	16	42	16	41	0	-3	-2
2010	3(32)	QB	Willow	Drew	HTR	18	46	17	44	16	42	-2	-4	-2
2007	7(12)	WR	Davis	Maurice	xxx	15	27	15	24	16	21	1	-6	-3
2011	5(31)	C-->DT	Barlow	Larry	xxx	17	55	20	52	21	51	4	-4	-3
2010	7(32)	G	Newton	T.J.	HTR	16	45	16	41	19	36	3	-9	-4
2008	6(20)	QB	Benson	Sammy	VO	15	44	15	39	22	28	7	-16	-5
2008	7(24)	T (was C)	Sauer	Walt	xxx	18	43	18	38	16	33	-2	-10	-5
2007	2(9)	QB	Reese	Scott	VU	20	54	23	48	37	37	17	-17	-6
2007	4(15)	TE	St. Louis	Charles	xxx	18	52	19	46	29	29	11	-23	-6
2011	3(31)	QB	Burton	Kent	HTR	22	59	21	52	20	47	-2	-12	-7
2011	4(31)	C-->T	Langlois	Rodney	xxx	15	67	18	60	26	58	11	-9	-7
2009	3(25)	P	Holliday	Zack	UR	71	71	61	63	72	72	1	1	-8

It's now sorted by the change in the first trainign camp--the key indicator of unmasking direction (go here for more discussion of unmasking). Notice these numbers:

• Interview results continue to be quite accurate, despite some complaints to the contrary. So far, 14 of the 17 players who were indicated to be heading in one direction or another were given the correct direction. (For those who have complained, this is another case where tiny sample sizes get you in trouble. Sure, your scout might miss on 3 or 4 guys in a given draft. You can't conclude "it's all random" from one or two MP drafts, though. Over time, he'll be correct well over half of the time.)
• Further, despite only being a few seasons in, every VU guy who is increasing, has increased by 10 points or more, including the one who was drafted just last season. (I'll bet the same would be the case for the VOs if I'd played Lamar Shinners enough to get him unmasked more.)
• Exactly 50% of the players I've drafted so far with HTR or no interview increased (there's no particular reason I've drafted no AS guys...just small sample), and 50% decreased.

If you're concerned about my sample sizes, here are numbers from a much larger sample size in 6.0e.

• 89 drafted players were VU or UR. 77 of those gained points. (86.5%)
• 22 drafted players were VO or OR. 19 of those lost points. (86.4%)
• 17 drafted players were "As Scouted." 14 of those ended up within 4 points of the original estimate.
• 63 players were drafted who were either HTR or not interviewed at all. 40 of those (63.5%) increased in points. Seeing as I'm only at 50% in 6.1, I've got some work to do there.

If the accuracy of interviews has changed at all, it hasn't changed much. Success varies from scout to scout, but it's fair to say that over nearly 200 draftees in 6.0e, spanning 30 seasons (so several different scouts there), my scouts' interview results were 80% to 90% accurate, and that's where they appear to be heading in 6.1 as well.

CLOSING COMMENTS/CONCLUSIONS

• The "interviews-are-worthless" people are dead wrong. The results are still highly accurate indicators of which direction a player is heading, and by what degree.
• A MAJOR part of drafting is interviewing the right players. If you're coming up heavily tilted toward VO players in particular, you probably need to re-think your entire drafting strategy.
• The MP implications are that everyone has exactly 60 resources that appear to be more accurate than anything else in the game when it comes to evaluating rookies. This is the one area in the game where we're all on a truly level playing field every single season. How we use those 60 resources can make a significant difference in drafting success or failure.

[Ben E Lou]

These might be worth noting as well. These are the average overall changes by impression for my draftees:

6.0e
VO: -13.1 (max =-31)
OR: -6.4
UR: +6.8
VU: +11.1 (max= +32)

It's too early to post these numbers for 6.1, but, considering I already have 4 players who are +15 or better, what I SUSPECT we will see here is that the average gain/loss won't change terribly much, but that with the reduced scout error, the pool will contain fewer VU and VO players.

Two notes...

1. There are still cases where your scout is just dead wrong--identifying a VO player as VU, and vice versa. Note QB Scott Reese from above as an example of it happening in 6.1.
2. For less-experienced players, one note of caution about that VO average in 6.0e: it's skewed by the fact that I was able to identify three cases in which the scout was clearly wrong, the guys both increased significantly. Backing those two out, the average drops to -21.7.

[cuervo72]

# 89 drafted players were VU or UR. 77 of those gained points. (86.5%)
# 22 drafted players were VO or OR. 19 of those lost points. (86.4%)

Those seem to be rather close, as if the % were taken from a table or something. I'd be curious to see how these trended according to scout ability. Might you get an 86.5% hit rate on EX, 83.5% on VG, 80.5% on G etc? I would think they wouldn't be the same but would vary.

And yeah yeah on the small sample size. I know I complained a bit in IHOF, but that was more on my particular scout being an idiot than about scouting in general. (oh noes, elements of roleplaying! )

[Ben E Lou]

Originally Posted by cuervo72

Those seem to be rather close, as if the % were taken from a table or something. I'd be curious to see how these trended according to scout ability. Might you get an 86.5% hit rate on EX, 83.5% on VG, 80.5% on G etc? I would think they wouldn't be the same but would vary.

I didn't track it by scout ability, so I can't help you there, but I'd say that's a fair assumption.

And yeah yeah on the small sample size. I know I complained a bit in IHOF, but that was more on my particular scout being an idiot than about scouting in general. (oh noes, elements of roleplaying! )

Heh. I don't remember who complained, and I wasn't even thinking IHOF when I made that comment, either. I just know I've read a number of people commenting that they're not worth the time because they're so random.

[Draft Dodger]

it's interesting that some positions seem to be clustering together on the 2nd chart. The OLB, S and WR you've taken all seem to be booms, while the QBs and linemen don't seem to have faired as well. I wonder if those would match up with scout ability or if they can just be attributed to you looking for the right combine scores for some groups, and not so well with others. or, it's just random.

[Ben E Lou]

Originally Posted by Draft Dodger

it's interesting that some positions seem to be clustering together on the 2nd chart. The OLB, S and WR you've taken all seem to be booms, while the QBs and linemen don't seem to have faired as well. I wonder if those would match up with scout ability or if they can just be attributed to you looking for the right combine scores for some groups, and not so well with others. or, it's just random.

I think that's more based on me than something in the game. I *know* that I'm good at identifying safeties and WRs who are better than scouted. (I wouldn't have said that for OLB, but maybe I'm getting better there.) As far as QBs, it's a new career, and I haven't found a long-term solution there, so I've been drafting one nearly every year--even ones that I doubt are any good, kind of a "just in case" thing. As for linemen, there are some that have gained, and some that have lost. The losses are a little ahead of the gains now perhaps, but I suspect that's just a sample size thang.

[dj_morton]

great info!

[Mistwood19]

Thanks for the info Skydog!

[Hammer]

my scouts' interview results were 80% to 90% accurate, and that's where they appear to be heading in 6.1 as well.

Any thoughts as to just how much this changes with the quality of the scout? I presume "young talent" is the key variable here?

[Ben E Lou]

An update...

Originally Posted by SkyDog

Interview results continue to be quite accurate, despite some complaints to the contrary. So far, 14 of the 17 players who were indicated to be heading in one direction or another were given the correct direction.

I don't know if it has to do with the guys I'm interviewing, or if "reduce the scouting error" applies here (which I wouldn't have expected, given how accurate the interview results already were). The number is now 29 for 32.