DeltaWhiskey
09-15-2008, 05:15 AM
BACKGROUND:
I began applying statistical analysis procedures to draft data with the hope of improving my draft success and to learn a bit about how FOF works. I conducted 2 sims to provide me 2 draft classes (1600+ players). I used this data to create regression equations to predict the CUR/FUT ratings for Pre TC and Preseason WK1 (before/after TC). After presenting some general information regarding the findings as they relate to the PRE TC CUR scores in another thread here, I learned that there seems to be little interest in this particular rating, and that in particular, interest in predicting ratings much further in the future (up to 4yrs) was desired. Someday, I may put together a data set to explore this. For now I'm happy predicting Preseason WK1 FUT ratings.
In this thread, I'm going to present the results of some additional data mining from this data set. Essentially, I was curious as to what, if any, are the differences between players I'll call the Wheat and the Chaff. The variable of interest is the Preseason WK1 FUT scores. Wheat players are players who at this point are showing the potential to be Excellent, while the Chaff refers to those who are Good to Very Good.
METHOD:
The Wheat category consisted of all players w/ Preseason WK1 FUT rating > 65, and the chaff was defined as any player w/ a Preseason WK1 FUT rating ranging from 50 to 64.
Wheat: n = 31
Chaff: n = 130
To assess differences between the Wheat and chaff, I conducted a series of one-tailed t-Tests on the pre-draft data (i.e. 40yd dash, agility, etc.), including an additional variable, Average Ability (AVG ABI). AVG ABI is simply the average of all of the numbers for bar scores from Extractor output for each player.
RESULTS:
After running the t-tests for height, weight, volatility, Sol, 40Y, Bench, Agility, Broad Jump, Pos Drill, % Developed and AVG ABI, only average AVG ABI yielded a significant difference at the p<0.05 level. This suggests that any differences between the Wheat and chaff groups on Combines (e.g. 40Y, Bench, etc.) and other measures (e.g. Sol, Height, Weight, etc.) are most likely due to chance, while there is a less than 5% chance that the difference between the Wheat and chaff on AVG ABI is due to chance.
For AVG ABI:
Mean Wheat = 69.87 (SD = 8.11)
Mean chaff = 56.83 (SD = 7.45)
Twenty-four players scored greater than 69 on AVG ABI. Seventeen (70.8%) of those had a PreWK1 Fut score of 65 or greater, and of the remaining seven, all were rated as 55 or better.
>69 AVG ABI = 24
>69 AVG ABI = 24 and >= 65 PreWK1 FUT = 17
>69 AVG ABI = 24 and < 65 PreWK1 FUT = 7 (64,50,53,63,62,62,55)*
*PreWK1FUT ratings for 7 players w/ AVG ABI > 69 and PreWK1 FUT < 65)
DISCUSSION:
These results suggest that on average, selecting a player whose AVG ABI is greater than 69 will yield a player with a more than respectable Preseason WK1 FUT rating 70% of the time and a decent player 30% of the time. Undoubtably, there are going to be random situations where this is not the case, and a true dud may slip through from time to time; however, this should be farely rare. Additionally, the quality of the scout may play a role into the utility of this measure (AVG ABI).
There is also the possibility, that because of how I conducted the data analysis (multiple t-tests), I may be capitalizing on chance associated with running multiple t-tests, and there is no difference between Wheat and chaff; however, looking at the difference between means and the standard deviation for each group, I suspect this not the case.
Lastly, it is unclear if the Preseason WK1 FUT rating is of any value. At this time I prefer it as a rating of draftees, as it is the most "pure" measure available. That is, while being able to predict a players end of year CUR ratings four years out is desirable, there are potentially many interfering factors to account for (playing time, multiple training camps, mentors, coaches, etc.) with this measure. After the first training camp, the scout has gotten a good look, and only one other event (Training Camp) has occurred to effect player development.
I began applying statistical analysis procedures to draft data with the hope of improving my draft success and to learn a bit about how FOF works. I conducted 2 sims to provide me 2 draft classes (1600+ players). I used this data to create regression equations to predict the CUR/FUT ratings for Pre TC and Preseason WK1 (before/after TC). After presenting some general information regarding the findings as they relate to the PRE TC CUR scores in another thread here, I learned that there seems to be little interest in this particular rating, and that in particular, interest in predicting ratings much further in the future (up to 4yrs) was desired. Someday, I may put together a data set to explore this. For now I'm happy predicting Preseason WK1 FUT ratings.
In this thread, I'm going to present the results of some additional data mining from this data set. Essentially, I was curious as to what, if any, are the differences between players I'll call the Wheat and the Chaff. The variable of interest is the Preseason WK1 FUT scores. Wheat players are players who at this point are showing the potential to be Excellent, while the Chaff refers to those who are Good to Very Good.
METHOD:
The Wheat category consisted of all players w/ Preseason WK1 FUT rating > 65, and the chaff was defined as any player w/ a Preseason WK1 FUT rating ranging from 50 to 64.
Wheat: n = 31
Chaff: n = 130
To assess differences between the Wheat and chaff, I conducted a series of one-tailed t-Tests on the pre-draft data (i.e. 40yd dash, agility, etc.), including an additional variable, Average Ability (AVG ABI). AVG ABI is simply the average of all of the numbers for bar scores from Extractor output for each player.
RESULTS:
After running the t-tests for height, weight, volatility, Sol, 40Y, Bench, Agility, Broad Jump, Pos Drill, % Developed and AVG ABI, only average AVG ABI yielded a significant difference at the p<0.05 level. This suggests that any differences between the Wheat and chaff groups on Combines (e.g. 40Y, Bench, etc.) and other measures (e.g. Sol, Height, Weight, etc.) are most likely due to chance, while there is a less than 5% chance that the difference between the Wheat and chaff on AVG ABI is due to chance.
For AVG ABI:
Mean Wheat = 69.87 (SD = 8.11)
Mean chaff = 56.83 (SD = 7.45)
Twenty-four players scored greater than 69 on AVG ABI. Seventeen (70.8%) of those had a PreWK1 Fut score of 65 or greater, and of the remaining seven, all were rated as 55 or better.
>69 AVG ABI = 24
>69 AVG ABI = 24 and >= 65 PreWK1 FUT = 17
>69 AVG ABI = 24 and < 65 PreWK1 FUT = 7 (64,50,53,63,62,62,55)*
*PreWK1FUT ratings for 7 players w/ AVG ABI > 69 and PreWK1 FUT < 65)
DISCUSSION:
These results suggest that on average, selecting a player whose AVG ABI is greater than 69 will yield a player with a more than respectable Preseason WK1 FUT rating 70% of the time and a decent player 30% of the time. Undoubtably, there are going to be random situations where this is not the case, and a true dud may slip through from time to time; however, this should be farely rare. Additionally, the quality of the scout may play a role into the utility of this measure (AVG ABI).
There is also the possibility, that because of how I conducted the data analysis (multiple t-tests), I may be capitalizing on chance associated with running multiple t-tests, and there is no difference between Wheat and chaff; however, looking at the difference between means and the standard deviation for each group, I suspect this not the case.
Lastly, it is unclear if the Preseason WK1 FUT rating is of any value. At this time I prefer it as a rating of draftees, as it is the most "pure" measure available. That is, while being able to predict a players end of year CUR ratings four years out is desirable, there are potentially many interfering factors to account for (playing time, multiple training camps, mentors, coaches, etc.) with this measure. After the first training camp, the scout has gotten a good look, and only one other event (Training Camp) has occurred to effect player development.