Importance of different positions

[Daimyo]

I know this is something that several people have been looking at lately. I have been running several experiments over the last few days and I know at least a few others have been as well. Today I started a new project to look at the various positions and their relative importance.

In this series of experiments I will start a new game with a team created via the allocation draft. I will then use this same team with the exception of one only starter that will be varied. I use approximately 15-20 different players (carefully chosen so as to provide an even disbursement of overall ability) for each position and 10 seasons for each player to give an average number of wins per player per season. Using this data I should be able to determine the relative importance for each position. Of course this is not an exact science so the results may or may not be very useful, but I am bored at work so I will try to do it anyway. Hopefully I can get through at least a few positions before my fiance gets back on wednesday...

[Daimyo]

The first position I decided to examine is the starting quarterback. It is well believed the QB position is the most important and most studies similar to this one have confirmed this belief.

<a href="http://pubweb.acns.nwu.edu/~sws057/fof/qb.csv">Here is my data (a comma-delimitted spreadsheet)</a>

Basically, I found the QB position to be important and even moreso than previous experiments have shown. I found the <a href="http://pubweb.acns.nwu.edu/~sws057/fof/qb.gif">relationship between the scout's ratings of the quarterback (0-100 scale) and average wins</a> by the team to be

<font face="courier" size="3">AvgWins = 0.081 * ScoutsRating + 4.77
with an R² of 0.90</font>

The R² value of 0.90 is significantly higher than any I have seen for this sort of exeriment in FOF2k1. I believe this is due to the fact that every player besides the QB stays constant whereas in other experiments (including my own) the entire team is recreated each time.

[This message has been edited by Daimyo (edited 02-19-2001).]

[ez]

so you're re-simming the 2000 season 15 times, with 15 different QBs on an otherwise identical, average roster?

[Daimyo]

yes... that's the gist of it. But I plan to do the same thing for different positions as well if I have the time.

Actually... resimming the 2000 season 10 times each for 15 different starters.... It was a total of 150 simmed seasons.


[This message has been edited by Daimyo (edited 02-19-2001).]

[Morgado]

Originally posted by Daimyo:
<font face="courier" size="3">AvgWins = 0.081 * ScoutsRating + 4.77
with an R² of 0.90</font>

The R² value of 0.90 is significantly higher than any I have seen for this sort of exeriment in FOF2k1.

Actually my first guess is it's because all the other experiments to date have been flawed by bad design and poor data (extremely low # of obsrevations). My question here is what the standard error or t-statistic on your coefficients were (the intercept and the beta).

Oh heh nevermind. I just looked at your graph and goodness of fit appears to be very good indeed. http://dynamic.gamespy.com/~fof/ubb/biggrin.gif

Oh yeah, I forgot to mention it before: Good job, Daimyo.

------------------
The 64 Dollar Question: What *is* The FOF Journal (http://www.geocities.com/bobkemp01/journal.htm)?

[This message has been edited by Morgado (edited 02-19-2001).]

[Daimyo]

uh.. I'm gonna have to get back to you as soon as I find my old statistics books http://dynamic.gamespy.com/~fof/ubb/smile.gif

<a href="http://pubweb.acns.nwu.edu/~sws057/fof/qb.csv">Here is the data (csv)</a> from the QB portion if you want to have a look yourself in the meantime...

Thanks


[This message has been edited by Daimyo (edited 02-19-2001).]

[QuikSand]

Just to make a fairly worthless contribution to this thread, you can enter special characters into the UBB by using the alt-#### format. The code for the "squared" exponent is alt-0178.

Say fopr instance, typing r, then alt-0178 gives you:

r²

Our smug friend with the exponented name likes to think of this as his privilege, but not to worry... he's not much for on-topic discussion anyway. He won't be stopping by here.

Anyway, I really couldn't care less, but if you're curious-- that's at least one way to type the exponent. (Hold alt, then type the four digit number) There are similar codes for a whole range of oddball characters.

[Daimyo]

Back in my BBS days seven or eight years ago I used to know a lot of the useful alt-??? cobinations like that... alas I forgot them all and was too lazy to look for an ASCII chart. I tried several random combinations, but couldn't find it. Anyway, thanks Quik... now hopefully I can remember it next time I want to type R² http://dynamic.gamespy.com/~fof/ubb/smile.gif

[ez]

Originally posted by Daimyo:
Actually... resimming the 2000 season 10 times each for 15 different starters.... It was a total of 150 simmed seasons.


great experiment - no wonder your results are so good... http://dynamic.gamespy.com/~fof/ubb/smile.gif

[ibdb]

We're not worthy. . .We're not worthy. . .

Great stuff, Daimyo! I'm looking forward to the next installment.

(I've got to get a job that gives me more time to do stuff like that. http://dynamic.gamespy.com/~fof/ubb/biggrin.gif)

------------------
The city of Houston was named after Sam, not Dean.

[Daimyo]

RB is the next position to be examined. Most experiments have found very little correlation between the quality of the the starting RB and wins. I found more correlation than I have seen before, but still much less in comparison to the QB position.

<a href="http://pubweb.acns.nwu.edu/~sws057/fof/rb.csv">Here is my data (a comma-delimitted spreadsheet)</a>

Basically, I found the RB position to be roughly about 20% as important as the QB. I found the <a href="http://pubweb.acns.nwu.edu/~sws057/fof/rb.gif">relationship between the scout's ratings of the quarterback (0-100 scale) and average wins</a> by the team to be

AvgWins = 0.016 * ScoutsRating + 10.5
with an R² of 0.21

The R² value of 0.21 is significantly lower than the same for the QB position, but much higher than for similar experiments judging the importance of runningbacks. Even when all four RB's were "zeroed out," the team still managed to win an average of almost 10 games per season. While it does seem that a quality RB *can* help a team's performance, it is not as cut ad dry as with the QB. Also, success is possible even with the worst possible runningback.

[TheDawgsAreOut]

Daimyo - Fine work. The more I read this thread, the more impressed I am. I am afraid I'll have to give up my attempts to rank positions in a different manner because yours is much more accurate, clear, and most importantly does not take hours to write down. I am glad to see that the rb data confirms what I found that it is far less important than qb. Looking forward to more data.

------------------
The best damn announcer the fake FOFL has ever seen! Check out my cd "Who cares who let them out, I know they're out!" in stores everywhere next month.

[Daimyo]

Thanks... I should be able to finish up the primary WR today or tomorrow morning, but after that I won't be able to do any more until Monday. I would like to go through ever position, but I'm not sure that's possible... I'm not sure how long work will stay so slow for me. Of course, if you or anyone else wants to contribute any positions or any further analysis on the positions I've done, it would be more than welcome. http://dynamic.gamespy.com/~fof/ubb/smile.gif

[QuikSand]

Hmmm... maybe if the general forum/landfill stays shut down for a while, people will have no choice but to stop by and read some of this stuff.

Nah. Baywatch is on somewhere.

[Alf]

Thanks Daimyo ! Really impressive analysis.

So what we can deduce from your QB+RB analysis is that you don't need to spend stupid money on your Running-Back, but it is nearly mandatory to spend that stupid money on your QuaterBack !

Thanks again !

------------------

Alf

[Daimyo]

Next up is the primary WR. This experiment is the same as the previous two, except for a small change. Because there are two starting WR's and I wanted to test the the primary WR, I had to use near minimum WRs for the other four roster spots. Because of this, this experiment is perhaps a little less valid than the others because in normal situations the secondary WR will be a much higher caliber player.

<a href="http://pubweb.acns.nwu.edu/~sws057/fof/wr1.csv">Here is my data (a comma-delimitted spreadsheet)</a>

I found the quality of the primary WR to have absolutely no correlation to the average number of wins by the team. <a href="http://pubweb.acns.nwu.edu/~sws057/fof/wr1.gif">Here isa graph of the results.</a> The "relationship" is

AvgWins = 0.00007 * ScoutsRating + 8.4
with an R² of 0.000006

The R² value of 0.000006 is low enough to basically be considered to be zero. Clearly, in my experiment, the quality of the primary WR had absolutely no effect on the team's performance measured in wins.

What does this mean for us? Perhaps it means that having only one good receiver is worthless. My hypothesis is that you really need two solid receivers to see any effect on performance. My next experiment will examine the effect of the secondary receiver which should provide some more answers. Unfortunately I doubt I'll be able to get to it until Monday...

[This message has been edited by Daimyo (edited 02-21-2001).]

[ez]

interesting...

is your plan to add a second above-average WR to the team? or will your #2 WR be the only standout? (i suppose both experiments have merit.)

if you use only one strong receiver, i wonder if the AI is smart enough to swap his top CB to cover the split end or flanker as needed.

this may also have some bearing on your first WR experiment. i wonder if your solo stud was constantly double-teamed, greatly reducing his impact.

i think adding an above-average #3 WR (in a separate experiment) would be another interesting test. i have an old theory bouncing around here that a strong #3 WR has a (much?) larger than expected impact, because few/no AI teams have a third decent CB...

[This message has been edited by ez (edited 02-21-2001).]

[B & B]

i think adding an above-average #3 WR (in a separate experiment) would be another interesting test. i have an old theory bouncing around here that a strong #3 WR has a (much?) larger than expected impact, because few/no AI teams have a third decent CB...

Count me as a subscriber. When you run alot of 3 wide sets and tend to pass, Ive seen the three slot have equal/better numbers than the SE. Of course having roughly 20% of your cap on those high priced positions has been a problem. Often , one of the three I carry is a starting draftee who has yet to resign for big bucks.

[Daimyo]

For the secondary wide receiver test, I was planning to have the greatest WR I can find on the team as a constant and then vary the second receiver. The third through fifth receivers would remain the worst I can find. That way it lets me allow the computer to handle things during the season. As long as I keep the best receiver in the game as my primary receiver, the computer will never put the guy I'm testing in the first spot (except when the first guy is taken out to rest). I will need to discount seasons in which the priomary receiver misses any significant time due to injury.

Do you think using extremes (ie, the primary guy is a superstuf and all the backups are as weak as possible) like this compromises the validity of the experiment?

Also, I think I agree on the third receiver bit. All my previous experiments showed it to be more important than the first or second. However those experiments would also sometimes say that the always inactive, fourth string guard was important... that's why I'm trying to keep everything as controlled as possible in this series to try to weed out "fake" results.

[Daimyo]

Its been a while since I've been able to update this post between "intrusions" from work and my personal life... but I was finally able to finish the secondary Wide Receiver. In this experiment only the second best wide receiver varied. Marvin Harrison was chosen as the primary receiver (79 overall).

I found the quality of the secondary receiver to be more important than the RB, but less the QB. <a href="http://129.105.195.109/groupthink/wr2.gif">Here is the graph</a>. <a href="http://129.105.195.109/groupthink/fofwr2test.txt">Here is the data (tab-delimited)</a>.

AvgWins = 0.029 * ScoutsRating + 8.2
with an R² of 0.53

The R² value of 0.53 is very good for this type of experiment. It seems that WR's are required in pairs to be successful. While having two great WR's won't help as much as having one great QB, it will help more than having one great RB.

[This message has been edited by Daimyo (edited 03-02-2001).]

[Alf]

Excellent !! As usual.

------------------

Alf

[EagleFan]

Okay, I'm scared. Not by all the numbers mentioned, but by the time needed to get the numbers.

wow http://dynamic.gamespy.com/~fof/ubb/smile.gif

[OldGiants]

Great stuff, Daimyo.

I glad to see some people are still playing FOF besides me.

On the 2nd and 3rd WR discussion, let me introduce some consistent results (in anecdotal form, because I'm too lazy to dig out the numbers) from a several seasons I've run using my "3rd down clutch WRs, QBs and RBs" idea.

The rule is I don't draft skill players until at least the fifth round. I then sign skill position with the best 3rd down skills I can find. this results in a lot of KR-type WRs.

these guys usually have high 3rd down and yardage skills, with the others in the 25 or less range. Because stamina is crappy, everybody plays, and gets hurt, so 7 WRs per season is the norm.

The offense sucked until I picked up a breakout RB in the 6th round. He now leads the team in rushing and recieiving, and has been the MVP three years running, after being ROY.

Until he showed up, the #2 reciver was always the leading reciever (two different expansion teams in 10+ year careers). Often the guy would would catch 80+ balls, with the second best total being in the 30's. The next season, this guy would be a flop as #1 WR, and the newest FA rookie signing would catch the 60-80 balls.

the #3 guy was not significant because there were usually several players in the #3WR due to injuries.