Testing a Game v. a Season [Archive] - Front Office Football Central

View Full Version : Testing a Game v. a Season

damelloman888

08-25-2008, 09:01 PM

TheMeat

08-26-2008, 12:04 AM

I think that is a great idea, as I noticed it in the other thread on Def GPs and it got me to thinking. The chore of it is taking down the stats for each game, since I know nothing about parsing programs, but may be someone has a good idea on how to do this easily.

I think that running the single game a 100 times would give you a very good idea of your performance in a specific situation, and comparing that to other games run multiple times against different types of teams might lead to some kind of revelation about defensive gameplanning.

Dutch

08-26-2008, 12:35 AM

Sounds like a promising idea.

DeltaWhiskey

08-26-2008, 07:48 AM

Question for the statistics people:

Has anyone thought about the implications of testing a single game as opposed to a season?

It is a lot of work, but I thought it may help solve the defensive puzzle.

Let's say you want to test what bumping the SLB blitz by 10% does. If you ran that for the same season 10 times you would have 160 samples. The problem is though that the computer changes the offence for each team.

If however, you ran the same game 100 times you just might have a good sample of the 10% increase in the SLB blitz against the "type" of team you are playing against.

Any thoughts would be greatly appreciated.

My thoughts, having re-read the other Defensive Gameplan string and this one:

A. What are you going to measure? Changes in the number/success of SLB blitzes? Changes in the number of sacks? Changes in points allowed? Changes in yards allowed? Given the amount of work required to gather any useful data, this is a crucial question to answer up-front.

B. The study, as proposed, has limitations. In addition to being limited by the type of offense the team runs, it is also limited by the strengths and weaknesses of the personnel on the opposition.

C. If you wish to pursue this (i.e. repeatedly running the same game), would recommend the following:

Run in an MP league with two human controlled teams.
Manipulate both rosters such that all players are between 45 and 55.
Both teams should use the same offensive gameplan (recommend these be from the gameplan library).
Both teams run identical defense
For testing (e.g. increase in the SLB blitz') would recommend obtaining 3 or 4 levels of measurement: (1) a baseline test = SLB Blitz 0%, (2) a low-test = SLB Blitz +10%, (3) a medium test = SLB Blitz + 20% and (4) a high test = SLB Blitz +30% (or some series of setting to reflect baseline, low, medium and high) .
By doing 3 & 4, you're getting 2 data points/game
Next you can make changes to 3 or 4 to conduct further trials
Depending on what you measure, 100 trials is probably excessive. Twenty-five trials (50 data points) should be enough
Use a statistical software program to analyze. Should be able to either conduct a regression analysis or an ANOVA depending on your variable of interest.As you can see, and based on your own observations, in order to gain a large amount of meaningful data, this really would be tedious.

I suspect that within Quicksands multiple runs of a single season, there are valuable pieces of information that could be extracted with the proper statistical analysis techniques.

Celeval

08-26-2008, 07:50 AM

Running a single game - saving out the info each time - 100x would be excruciatingly tedious. :)

What may be more easily doable is setting up a MP league, giving all 32 teams the same offensive and defensive gameplans. Then running that season 10x... so you're looking at 16 games x 32 teams x 10 seasons = a lot of data with the same offense v. defense (obviously different personnel).

DeltaWhiskey

08-26-2008, 08:03 AM

I like that plan. Personnel differences should wash out over the course of a season, and rather than repeating the season, would recommend running consecutive seasons to further reduce any personnel effects.

damelloman888

08-26-2008, 08:46 AM

My thoughts, having re-read the other Defensive Gameplan string and this one:

A. What are you going to measure? Changes in the number/success of SLB blitzes? Changes in the number of sacks? Changes in points allowed? Changes in yards allowed? Given the amount of work required to gather any useful data, this is a crucial question to answer up-front.

B. The study, as proposed, has limitations. In addition to being limited by the type of offense the team runs, it is also limited by the strengths and weaknesses of the personnel on the opposition.

C. If you wish to pursue this (i.e. repeatedly running the same game), would recommend the following:

Run in an MP league with two human controlled teams.
Manipulate both rosters such that all players are between 45 and 55.
Both teams should use the same offensive gameplan (recommend these be from the gameplan library).
Both teams run identical defense
For testing (e.g. increase in the SLB blitz') would recommend obtaining 3 or 4 levels of measurement: (1) a baseline test = SLB Blitz 0%, (2) a low-test = SLB Blitz +10%, (3) a medium test = SLB Blitz + 20% and (4) a high test = SLB Blitz +30% (or some series of setting to reflect baseline, low, medium and high) .
By doing 3 & 4, you're getting 2 data points/game
Next you can make changes to 3 or 4 to conduct further trials
Depending on what you measure, 100 trials is probably excessive. Twenty-five trials (50 data points) should be enough
Use a statistical software program to analyze. Should be able to either conduct a regression analysis or an ANOVA depending on your variable of interest.As you can see, and based on your own observations, in order to gain a large amount of meaningful data, this really would be tedious.

I suspect that within Quicksands multiple runs of a single season, there are valuable pieces of information that could be extracted with the proper statistical analysis techniques.

Well, I guess I got my answer, yikes! Wish I read that before running a test.

FYI I had a team with good Bump and Run numbers. So I altered my pass defence by adding 10% to BnR and taking away 10% from MfM -- however I never let MfM drop below 3% so that the opposing QB would have a hard time locking on.

To keep it simple I looked at Wins, Points For, and Points againts. I did 50 samples. Most of the data was inconclusive, however there was a 33% increase in games that my opponent was held to less that 21 points.

So, if I limit it to saying that against the "type" of talent I was playing against, does it mean anything?

DeltaWhiskey

08-26-2008, 09:09 AM

Well, I guess I got my answer, yikes! Wish I read that before running a test.

FYI I had a team with good Bump and Run numbers. So I altered my pass defence by adding 10% to BnR and taking away 10% from MfM -- however I never let MfM drop below 3% so that the opposing QB would have a hard time locking on.

To keep it simple I looked at Wins, Points For, and Points againts. I did 50 samples. Most of the data was inconclusive, however there was a 33% increase in games that my opponent was held to less that 21 points.

So, if I limit it to saying that against the "type" of talent I was playing against, does it mean anything?

It's hard to say what, if anything you've found.

What is significant about 21 points? Is that an average number of points scored in this test? The average number of points scored in general in FOF?

What happened in the other 67% of games?

These are pieces of information that can help determine the value of what you've found.

QuikSand

08-26-2008, 09:39 AM

If however, you ran the same game 100 times you just might have a good sample of the 10% increase in the SLB blitz against the "type" of team you are playing against.

My honest answer is that I fear this would still be too small a sample size to tell much, especially for something as minor as the 10% tweak you're talking about. There might be some value in testing an entirely different defensive strategy this way, but I just don't know what you're going to get out of all this labor if all you test is a small thing like a modest bump in blitzing percentage.

I confess that I'm somewhat interested in the results or something like this, but I fear that what you'd see would end up being something that looks an awful lot like white noise. One team posts a PR% of 21.4, and the other a 21.8, and you end up saying "who knows?" or (possibly more harmful) you end up being drawn to a conclusion that B is better than A when in fact the difference was simply a function of the still-present randomness at work.

QuikSand

08-26-2008, 09:43 AM

I responded earlier before reading the whole thread, sorry...

Most of the data was inconclusive, however there was a 33% increase in games that my opponent was held to less that 21 points.

I think I'd just edit this to "All of the data was inconclusive."

damelloman888

08-26-2008, 09:46 AM

It's hard to say what, if anything you've found.

What is significant about 21 points? Is that an average number of points scored in this test? The average number of points scored in general in FOF?

What happened in the other 67% of games?

These are pieces of information that can help determine the value of what you've found.

I think I found.... nothing :confused: thus returning to my former conclustion that I wish I read your post before conducting the test.

damelloman888

08-26-2008, 09:48 AM

<TABLE style="WIDTH: 96pt; BORDER-COLLAPSE: collapse" cellSpacing=0 cellPadding=0 width=128 border=0 x:str><COLGROUP><COL style="WIDTH: 48pt" span=2 width=64><TBODY><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; WIDTH: 48pt; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" width=64 height=17>Comp</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; WIDTH: 48pt; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" width=64>Change</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>6</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>3</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>7</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>7</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>7</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>7</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>9</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>7</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>13</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>12</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>13</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>13</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>14</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>13</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>16</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>13</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>16</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>13</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>17</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>13</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>17</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>13</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>20</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>13</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>20</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>14</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>20</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>16</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>20</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>17</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>21</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>18</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>21</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>20</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>21</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>20</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>21</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>20</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>23</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>20</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>23</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>21</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>23</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>21</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>24</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>23</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>24</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>24</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>24</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>24</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>24</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>24</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>24</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>24</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>24</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>24</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>24</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>26</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>24</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>27</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>24</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>27</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>27</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>28</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>27</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>29</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>28</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>29</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>30</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>29</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>31</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>30</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>31</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>30</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>31</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>31</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>32</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>31</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>32</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>31</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>34</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>32</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>34</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>34</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>35</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>34</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>35</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>35</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>35</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>35</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>36</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>35</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>37</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>37</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>38</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>38</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>42</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>38</TD></TR><TR style="HEIGHT: 12.75pt" height=17><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; HEIGHT: 12.75pt; BACKGROUND-COLOR: transparent" align=right height=17 x:num>51</TD><TD style="BORDER-RIGHT: #ece9d8; BORDER-TOP: #ece9d8; BORDER-LEFT: #ece9d8; BORDER-BOTTOM: #ece9d8; BACKGROUND-COLOR: transparent" align=right x:num>38</TD></TR></TBODY></TABLE>

Yoda

08-26-2008, 11:58 AM

If I was going to do this kind of test, I would set up the player file with all 50's/mid level/what have you. So that all players for every position are rated the same (except for the values we cannot control and the hidden values).

Then I would set up each defensive team a little different and have every offense running the same offense.

Then again, I don't have that kind of time. :(

damelloman888

08-26-2008, 06:44 PM

DeltaWhiskey

08-27-2008, 01:30 AM

My honest answer is that I fear this would still be too small a sample size to tell much, especially for something as minor as the 10% tweak you're talking about. There might be some value in testing an entirely different defensive strategy this way, but I just don't know what you're going to get out of all this labor if all you test is a small thing like a modest bump in blitzing percentage.

I confess that I'm somewhat interested in the results or something like this, but I fear that what you'd see would end up being something that looks an awful lot like white noise. One team posts a PR% of 21.4, and the other a 21.8, and you end up saying "who knows?" or (possibly more harmful) you end up being drawn to a conclusion that B is better than A when in fact the difference was simply a function of the still-present randomness at work.

This is where statistical analysis tools becomes crucial. A difference between 21.4 and 21.8 could be statistically significant - even w/ a small sample size if the effect is particularly robust. Of course statistical significance can still have little to no real world relevance, but at least it becomes possible to assign a confidence interval regarding the impact of randomness (i.e. error).

After thinking about this issue a little more, I'm wondering if the cart isn't being placed before the carrot. In another thread, it was noted that there was a lack of clarity regarding the impact/importance/value of double coverage. This has led me to think that before tweaking the defense (i.e. double coverage, blitz %, etc.) it would be useful to know to what degree these variables result in positive plays for the defense. That is, what % of sacks involve a LB or DB blitzing or what is the completion % for double coverage v. single v. zone, etc.

I haven't spent much (any) time looking at game logs, so I'm unfamilar w/ the data available and therefore don't know if this type of analysis would be possible.

DeltaWhiskey

08-27-2008, 02:47 AM

I think I found.... nothing :confused: thus returning to my former conclustion that I wish I read your post before conducting the test.

If I'm understanding the data you posted:

Independent Variable (IV) = pass defence increased 10% to BnR and decreased 10% from MfM - never let MfM drop below 3%

Dependent Variable (DV) = points allowed (each number represents the points allowed in a single game correct?)
Comp = points allowed before applying IV
Change = points allowed after applying IV

Based on these assumptions, I ran a quick analysis of the data and found the following:

Comp AVG = 24.6 (SD = 9.31)
Change AVG = 23.22 (SD = 9.39)
Difference = 1.38

I next conducted a t-test, which yielded a p = 0.231, which is generally considered to be non-significant.

What all of this means (if I correctly understand the data you provided) is that: "adding 10% to BnR and taking away 10% from MfM" and "never let[ting] MfM drop below 3%" resulted in an average 1.38 reduction in points allowed. The probablility of this difference being due to chance (random or other factors) is less than 23% (i.e. we can say with only 77% degree of confidence the changes in defense were responsible for the 1.38 point reduction).

Possible causes for these findings:

Sample size to small - I took your data and repeatedly copied it until I achieved significance below 0.05 on a t-test. Somewhere between 200 and 400 data points was required.
Independent variable (IV) too small/not powerful enough - perhaps a 15% or 20% bump up is required to produce a change
Wrong dependent variable (DV) - points allowed may not be the best measure to detect an effect, perhaps Completion %, Passing YDS allowed or something more pass specific would be better.Comment: I think #3 may be the most salient/relevant place to start. While ultimately the goal is to have the defense prevent points, in this study, the relevant question is "What impact on my opponent's passing game do these adjustments have?" If you've got the data, I'd be happy to look at it.

In the end, I have to agree (unfortunately) that you've probably found nothing; however, without running the above sort of analysis, there is no way to really know with any degree of confidence, and perhaps with a different DV and a few more runs (e.g. 100 as originally proposed) something of interest might pop-up.

damelloman888

08-27-2008, 10:40 AM

If I'm understanding the data you posted:

Independent Variable (IV) = pass defence increased 10% to BnR and decreased 10% from MfM - never let MfM drop below 3%

Dependent Variable (DV) = points allowed (each number represents the points allowed in a single game correct?)
Comp = points allowed before applying IV
Change = points allowed after applying IV

Based on these assumptions, I ran a quick analysis of the data and found the following:

Comp AVG = 24.6 (SD = 9.31)
Change AVG = 23.22 (SD = 9.39)
Difference = 1.38

I next conducted a t-test, which yielded a p = 0.231, which is generally considered to be non-significant.

What all of this means (if I correctly understand the data you provided) is that: "adding 10% to BnR and taking away 10% from MfM" and "never let[ting] MfM drop below 3%" resulted in an average 1.38 reduction in points allowed. The probablility of this difference being due to chance (random or other factors) is less than 23% (i.e. we can say with only 77% degree of confidence the changes in defense were responsible for the 1.38 point reduction).

Possible causes for these findings:

Sample size to small - I took your data and repeatedly copied it until I achieved significance below 0.05 on a t-test. Somewhere between 200 and 400 data points was required.
Independent variable (IV) too small/not powerful enough - perhaps a 15% or 20% bump up is required to produce a change
Wrong dependent variable (DV) - points allowed may not be the best measure to detect an effect, perhaps Completion %, Passing YDS allowed or something more pass specific would be better.Comment: I think #3 may be the most salient/relevant place to start. While ultimately the goal is to have the defense prevent points, in this study, the relevant question is "What impact on my opponent's passing game do these adjustments have?" If you've got the data, I'd be happy to look at it.

In the end, I have to agree (unfortunately) that you've probably found nothing; however, without running the above sort of analysis, there is no way to really know with any degree of confidence, and perhaps with a different DV and a few more runs (e.g. 100 as originally proposed) something of interest might pop-up.

Wow, thank you for the info -- the whole thing lead to another idea.

Once again with 50 samples.

IV = human coaching (in this case me using everything I know so far)

DV = Win percentage
Comp = WP before applying IV
Change = WP after applying IV

Comp = 54%
Change = 66%

also FYI

DV = Points
Comp = 26.32
Change = 23.8

DV = Points allowed
Comp = 24.6
Change = 18.54

It is important to note that I did not fix the Red Zone defense bug in either case.

How much difference would be significant?
Could I have stumbled on a good way for a rookie coach to "practice"?

Thank you for working through this with me, it sure is interesting.

DeltaWhiskey

08-27-2008, 11:27 AM

Wow, thank you for the info -- the whole thing lead to another idea.

Once again with 50 samples.

IV = human coaching (in this case me using everything I know so far)

DV = Win percentage
Comp = WP before applying IV
Change = WP after applying IV

Comp = 54%
Change = 66%

also FYI

DV = Points
Comp = 26.32
Change = 23.8

DV = Points allowed
Comp = 24.6
Change = 18.54

It is important to note that I did not fix the Red Zone defense bug in either case.

How much difference would be significant?
Could I have stumbled on a good way for a rookie coach to "practice"?

Thank you for working through this with me, it sure is interesting.

No problem, this is interesting to me. I frequently don't have the time to do the data collection and the analysis I would like and I enjoy thinking this through.

RE: Win%, I've got to think a little on how to test this, but eyeballing it, you went from winning around 1/2 of the time (a coin toss = not much better than pure chance) to about 2/3 of the time. Without testing, that looks significant.

If you could provide me the raw data (like in the previous post) either in the string or PM, I could work it up pretty quick and see what's going on. Based on the earlier results, I think it will be significant, but would have to see.

Did you make any offensive adjustments?

Interesting to note that Comp pts allowed = 24.6 - did you run it again, or use the old data?

damelloman888

08-27-2008, 11:40 AM

Me coaching make all kinds of offensive and defensive adjustments.

Us Them Win
34 14 1
24 21 1
30 14 1
20 17 1
12 17 0
30 14 1
29 17 1
31 9 1
27 30 0
24 19 1
37 7 1
12 10 1
41 26 1
24 34 0
10 17 0
27 7 1
19 20 0
38 28 0
24 27 0
34 21 1
17 34 0
30 23 1
23 20 1
29 3 1
10 28 0
23 15 1
28 20 1
23 15 1
20 16 1
20 27 0
20 10 1
26 17 1
30 17 1
26 3 1
17 20 0
17 20 0
13 20 0
13 17 0
26 17 1
16 6 1
27 24 1
19 16 1
24 21 1
21 6 1
34 10 1
25 35 0
23 20 1
17 30 0
24 27 0
22 21 1

damelloman888

08-27-2008, 11:49 AM

Computer Coaching (same as before, I'm hopeing that 100 samples are not nessisary)

Us Them Win
24 38 0
24 27 0
28 21 1
14 27 0
28 23 1
29 32 0
17 20 0
17 13 1
20 21 0
34 16 1
31 28 1
21 24 0
33 9 1
38 36 1
26 23 1
41 24 1
45 51 0
34 35 0
27 21 1
41 14 1
34 24 1
10 31 0
38 24 1
26 24 1
35 37 0
31 21 1
24 31 0
34 17 1
9 35 0
31 32 0
20 42 0
24 13 1
21 24 0
31 30 1
30 7 1
16 34 0
34 20 1
37 17 1
9 24 0
7 24 0
26 6 1
37 31 1
13 7 1
26 35 0
24 20 1
20 34 0
17 24 0
37 16 1
16 20 0
27 23 1

DeltaWhiskey

08-28-2008, 03:57 AM

Looking at the data:

CONTROL (Computer Running the Show)
AVG PF=26.32 SD=9.22
AVG PA=24.6 SD=9.31
AVG Point Differential=1.72 SD=12.8

TEST (User Changes)
AVG PF=23.8 SD=7.23
AVG PA=18.54 SD=7.88
AVG Point Differential=5.26 SD=11.15

These numbers suggest that while your scoring went down with your changes, so did your opponent's. Additionally, there was less variability in scoring by both teams. That is there was greater consistency. I assume, that none of the TEST games used the same offensive or defensive gameplan.

T-Test (Comparing CONTROL to TEST)
PF p=.1121
PA p=.0005*
AVG Point Differential p=.0785

*Significant <.05

Conclusions: The difference in Points For (PF) and AVG Point Differential between the two tests are not statistically significant (generally the "p" value should be <0.05). Said another way, there is an 11% probability that the change in PF occurred by chance and there's a 7% chance that the change in point differential was due to chance. The change in PA is statistically significant with a less than 1% (0.05%) chance that the change was due to chance alone (i.e. there is a 99.95% probability that the difference in PA is due to gameplan changes).

Problematic is the fact that changes were made to both offensive and defensive gameplans, That is as QuickSand has pointed out, how an offense performs often dictates what the defense has to face. Therefore, the overall changes in game plan appear to be effective at reducing PA in this particualy situation (i.e. against this one team's offensive talent and style).

At this point, a description of the oppositon would be useful (the offense/defense they seemed to use,if known, their personnel, etc.) and the game plan changes made would be informative.

Additionally, it might be useful to take this as preliminary data, look at changes made, pick some, and run repeated tests. If defensive gameplanning is the test you wish to look at (vice overall gameplanning) then I'd recommend returning to doing nothing on offense and tinkering w/ the defense. Also, 50 runs seems to be enough to detect changes when they're robust.

Kodos

08-28-2008, 10:08 AM

damelloman888

08-28-2008, 04:56 PM

Anyone else think that it is ridiculous that one would have to go to this level of effort to try to figure out defense in a game that is supposed to be played for fun? This is not a slam on the researchers -- kudos to them. This is a slam on the unintuitiveness of the game as far as defense goes. Sure, put in stuff that people can dig around to gain a little bit of an advantage. But don't make the game so that NOBODY knows what they are doing on defense.

Kodos I agree, I have a few days left with this kind of time and hoped to be able to help everyone out. It is silly the amount of time it takes though, however if I find anything I will be happy to share.

damelloman888

08-28-2008, 05:27 PM

What about in a season:

Comp = computer game plan
Change = human game plan

Sample a whole season twice and find a team that has an average of 8 wins (.500 record)

Then try the same seaon with your game plan. I think 13 wins are significant. Over two season I think you need 23 wins. Is that correct?

QuikSand

08-28-2008, 09:17 PM

Seriously, if you're trying to gather information from testing, at least gather enough information that you have something to work with. That means more than wins.

Here's a tool that you might find useful: Excel-Based QuikTest Template - Front Office Football Central (http://www.operationsports.com/fofc/showthread.php?t=60105)

I know it got edited a couple of times with various upgrades, but I think you may find it useful -- at least this way you'll have data on things like points allowed per game, yards per carry, yards per attempt, and the like.

DeltaWhiskey

09-02-2008, 06:43 AM

Ben E Lou

09-02-2008, 07:32 AM

DeltaWhiskey

09-02-2008, 09:49 AM

And even more seriously, it would be wise to take into account the advice of those who have played this and other iterations of FOF for 10ish years.

GUIDE TO DEFENSIVE SUCCESS IN FOF2007

1. Acquire players on defense with lots of big, red bars.
2. Don't do anything extreme with your game plan.
3. Don't rex, specifically because of the overuse of goal line, and the switched columns bug.
4. Do well on offense.
5. Don't waste your time testing one defensive game plan versus another.

I can put this guide in PDF form for printing if anyone needs it that way. ;)

This is the best advice we get from 10ish years of experience?

1. What if you can't acquire the necessary number of players w/ big red bars?

2. Extreme means what?

3. So basically just follow #2, that helps how?

4. What if you have a team that won't/can't do well on offense? (Do I have to have both 1 & 4 or will one or the other suffice?)

5. Why, because after 10ish years nobody else has succeeded?

The fact that in an earlier post, the Point differential was statistically significant at around the 0.07 and the Points Against Differential was statistically significant suggests that whatever monkeying around damelloman888 was doing, he may have been having some pretty good success tweaking his defense..

All of the above suggestions regarding defense suggest that it is essentially random and/or out of the player's control - is this what the game designer had in mind when he designed the defensive aspect of the game?

Ben E Lou

09-02-2008, 10:05 AM

This is the best advice we get from 10ish years of experience?Unfortunately, yes. To be fair, it's more like two years with this game. Things changed drastically with this version.

1. What if you can't acquire the necessary number of players w/ big red bars?Then your defense will suck, unless you have a high-powered offense.

2. Extreme means what? Jim gave some guidelines in one of the Q&As. I'd start there.

3. So basically just follow #2, that helps how?It avoids the offense becoming familiar with the defense. (Again, see Q&A on defensive familiarity.)

4. What if you have a team that won't/can't do well on offense? (Do I have to have both 1 & 4 or will one or the other suffice?)Then your defense will suck, unless it has *really* big red bars.

All of the above suggestions regarding defense suggest that it is essentially random and/or out of the player's control - is this what the game designer had in mind when he designed the defensive aspect of the game?Random? No. Out of the player's control besides the acquisition of personnel? Yes. Did the developer design it this way? All indications are that yes, he did. We know for a fact that defenses changed significantly.

Look, I don't mean to throw a wet blanket over testing. I am *THRILLED* to see some new faces doing detailed tests. I appreciate the work you and others have put into it, and I hope you'll stay involved in unraveling some of the mysteries of FOF. I'd just hate for y'all to burn out testing something that I'm fairly certain is going to yield little/no return. I've spent a good bit of time on this (and usually when I spend a good bit of time testing out game plans, a patch or new version ends up coming out to nerf the stuff I discover :devil:,) and I'm convinced that:

1. You can improve your defensive performance only *very* marginally by game planning "well."
2. You can screw up your defensive performance significantly by doing too much of any one thing. (Again I'd refer you to the defensive familiarity Q&A for guidelines.)
3. A "good" game plan is most useful against a team with an offensive game plan that's somewhat imbalanced. Against a balanced offense, it's virtually useless.

Chubby

09-02-2008, 10:21 AM

Ben E Lou

09-02-2008, 01:15 PM

So saith yet another person whose game plan testing has caused code changes. :D

MacroGuru

09-02-2008, 02:02 PM

All of the above suggestions regarding defense suggest that it is essentially random and/or out of the player's control - is this what the game designer had in mind when he designed the defensive aspect of the game?

Random? No. Out of the player's control besides the acquisition of personnel? Yes. Did the developer design it this way? All indications are that yes, he did. We know for a fact that defenses changed significantly.

This to me and what everyone has been saying about it...really honestly is frustrating. However, I am able to create a defense without the big red bars and do well against other teams (except Ben's) and I feel that the right players and skill sets at the right position can make or break a defense.

However, if that is not the case, so be it...this game is flawed to me more than ever then and if Gary's DDS: PF is better at all, even a shred...I will move that way immediately.

Ben E Lou

09-02-2008, 02:06 PM

Is it flawed, or does the NFL work that way? (i.e. Is it all about talent and mixing/disguising schemes in the NFL, rather than defensive schemes?)

MacroGuru

09-02-2008, 02:37 PM

Is it flawed, or does the NFL work that way? (i.e. Is it all about talent and mixing/disguising schemes in the NFL, rather than defensive schemes?)

Depends on the route you take. But everyone is saying if I don't have a great offense and great talent on defense I can't do a thing on defense.

Talent wise a decent defense with the proper scheme can be devastating, even in the NFL. I should be able to game plan, scheme and do more defensively in order to succeed than what I am being told here.

Get a lot of players with red bars
have an offense that can score
don't rex (which I agree with)
don't do anything extreme

to me, that's a flawed system, obviously if I have the best talent on defense, I should do well. It's about being able to win with what you have, and I am being told regardless of what I have, if they do not have the bars it's not going to work.

gstelmack

09-02-2008, 03:53 PM

FWIW, you can Rex on defense with success (and I do all the time), you just need to manually correct its egregious abuses, mostly fixing the swapped columns bug, and toning down its tendency to call 4-deep zone.

But nothing beats the big red bars...

gstelmack

09-02-2008, 03:55 PM

It's about being able to win with what you have, and I am being told regardless of what I have, if they do not have the bars it's not going to work.

But you're actually describing having the correct bars in the correct places. If you don't have talent, how do you expect to succeed? You don't need all-around players, but you do need overall talent in key places. For example, I could care less if my DTs can pass rush, and I could care less if my CBs can defend the run. If they do, it's a bonus, but they don't need to to have success. But if my DTs suck at run defense and my CBs can't cover man-to-man, no amount of scheming is going to cover that up, either in the NFL or in FOF.

bmerryman

09-04-2008, 02:10 PM

Is it flawed, or does the NFL work that way? (i.e. Is it all about talent and mixing/disguising schemes in the NFL, rather than defensive schemes?)

Leans to the flawed side. Example: Tampa/Cover 2 teams play Tampa 2 on most downs in the NFL. The Monte Kiffins of the world believe the simplicity of their scheme gives them an advantage. If you tried to run Cover 2 80% of the time in FOF then you would fail miserably.

It would be a nice improvement if you could replicate that or any other type of defensive philosophy within FOF and target talent designed to fit the defensive system whether it be Cover 2, Zone blitzing (LeBeau), Bellichik, Johnson, etc...

DeltaWhiskey

09-05-2008, 01:22 AM

Leans to the flawed side. Example: Tampa/Cover 2 teams play Tampa 2 on most downs in the NFL. The Monte Kiffins of the world believe the simplicity of their scheme gives them an advantage. If you tried to run Cover 2 80% of the time in FOF then you would fail miserably.

It would be a nice improvement if you could replicate that or any other type of defensive philosophy within FOF and target talent designed to fit the defensive system whether it be Cover 2, Zone blitzing (LeBeau), Bellichik, Johnson, etc...

Do big bars let you "cheat" more in this direction? That is, if I want to go with a more specialized defense, will having the big bars allow me get away with over-using certain strategies...even a little?

WelshWizard

09-05-2008, 08:09 AM

Do big bars let you "cheat" more in this direction? That is, if I want to go with a more specialized defense, will having the big bars allow me get away with over-using certain strategies...even a little?

In my opinion as Ben and others have stated never go to the extreme. However i have seen success when a defense has big red bars in many areas can go to a more specialized defense more successfully. I recall playing a team recently where my offense on numerous occasions had very favorable matchups where i would be running the ball against an aggressive pass defense yet still struggled to run the ball and my team was a successful running team, the opposing defense had a lot of talent especially the front seven at stopping the run.

DeltaWhiskey

09-05-2008, 08:35 AM

Well that certainly speaks to the notion of having the right people in the right schemes. Begins to make me wonder if what has been perceived as a lack of effective control in defensive gameplanning can't partially be explained as a mismatch between scheming/planning and personnel? Personally, as a fan, I know less in general about defense and this ignorance is further fueled by a lack of robust stats to explain what happens on defense.