My gameplanning "eureka" moment

[Leonidas]

I've posted before about some gameplanning testing I've done. Let me refresh on my methodology. I run 20 season blocks (320 games) on the same team for one season, running the same games over and over again, with injuries turned off to ensure the same talent is on the field every sim. Every team and gameplan I test I first run a baseline sim with the AI doing the gameplan. This becomes my standard to beat to have a successful offense. I further broke my data down into comparisons of performance by the team when they had winning seasons vs losing seasons and compared offensive performance between seasons the team scored more points than average and the seasons they didn't. My purpose was twofold, first develop a better gameplan than the AI and second find out statistically what teams are doing better when they are either winning or scoring more.

So yesterday I was doing this pretty much all day long and yesterday afternoon I had my "eureka" moment. I found a strong correlation that so far has panned out. I also found some non-correlations that may be just as important.

So here's what 6 hours of simming got me yesterday.

Running
There is no correlation between running the ball well and winning or scoring. This was not my "eureka" moment, but it was pretty profound all the same. There was really no statistical difference in yards per carry, number of rushes, or yards gained between teams with winning or losing records or teams that scored more than average and teams that didn't. The rushing statistics were all over the board.

I decided to tweak run direction and see if that might bring about a correlation, and it did, but not one I expected. Using run directions set to predominantly outside running (single digit runs around the guards, double digits around the tackles, 15-18 for outside runs) produced between one and two tenths higher rush per carry averages than an offense that had balanced run directions (10-12 per slot, stacked and extra point or two behind the best blocker). But it did not correlate into a more efficient offense. In fact, the offenses that ran behind balanced run directions tended to score about 10 more points per season than the outside running teams. I have a theory for this. The theory being the outside run offense probably breaks more long runs, hence leading to a higher rush average, but the balanced run offense is just more efficient at things like red zone offense and third down running. I only glanced at those numbers and didn't collect it so it's just a theory for study on another day. But the bottom line correlations were clear. You can tweak the running game to get more yards per carry, but there is zero correlation that it will make your offense better, may even make it worse.

Passing
Now for the "eureka" moment. I was tweaking gameplans left and right, collecting tons of data (must have run at least 3000games) and just couldn't figure out how to make a better offense. Then it hit me. It stuck our like a sore thumb. There was on statistic that stood out every single time. I couldn't believe it at first, just didn't make much sense to me. But decided to go with it and tweak it and sure enough, it made a difference. What stat was it? Yards per pass attempt. Every single sim I ran, the teams with winning records and the teams scoring more than average had a higher yards per pass attempt at the end of the season. When you run a dozen 20 season blocks and have a perfect 12 for 12 record it seemed pretty conclusive. Thats nearly 4000 games and there it was. Pass percentage also gave me a secondary correlation. I'd say it worked on 10 of the 12 season blocks.

So I decided to tweak the gameplan to optimize long passing just a little more and improved the team's overall scoring on average by a shade over 20 points per season and one tenth of a win more. Also made the team much more consistent. The win loss and scoring variances between seasons were much steadier. And all this from just improving yards per attempt by about a tenth. I also dropped 1-2% in completion percentage to do it, but having the higher percentage still correlated to winning teams and teams scoring more.

I decided to press my luck further and tweak the long passsing settings another 5 percent across the board figuring more was better. Well I obviously found diminishing returns at this point. The team played atrociously. Winning and scoring both plummeted, yards per attempt dropped 2-3 tenths, passing precentage dropped 3-4 points.

So my bottom line conclusion is the most efficient offense is one that strikes the right balance between a higher yards per attempt rate and a good completion rate. And rushing success seems to have little to do with improving the offense.

I'm going to do some more testing today with different teams to see if the correlation is still there. Obviously different talent sets will react differently to different gameplans.

[kingnebwsu]

Interesting sims eh. Keep us posted with more of your results :)

[Vinatieri for Prez]

Amazing the time you put into this.

Interestingly, I believe that yards per pass attempt also has been proven in the last decade or more to one of the biggest single statistical indicators of regular season success in the NFL.

So, it is not surprising that this is the case at all and it goes to show that Jim has patterned FOF after the NFL well. But good work bearing that out in you test runs.

[Dutch]

Good analysis!

Couple of points, at least on the MP front,

1. With the right personnel, you've got the workings of a great offense.

2. If you have the wrong personnel, you're screwed.

:)

[Ben E Lou]

Good analysis!

Couple of points, at least on the MP front,

1. With the right personnel, you've got the workings of a great offense.

2. If you have the wrong personnel, you're screwed.

:) http://www.benelou.com/iB_html/non-cgi/emoticons/rock.gif (javascript:emoticon('???'))

[Leonidas]

Good analysis!

Couple of points, at least on the MP front,

1. With the right personnel, you've got the workings of a great offense.

2. If you have the wrong personnel, you're screwed.

:)Obviously. I did the majority of my work yesterday with Cincinnati's offense, so yeah it they were pretty effective.

I also worked some other things out. For a passing offense I think a 60-40 pass to run ratio works best. It certainly was best in my test runs yesterday. I also have a pretty good balanced offense that works at least as well as the AI, assuming you have the right personnel for it. It kind of underperformed with the Cincy offense, but I think Carson Palmer and friends need to open it up to maximize their effectiveness.

I also have been thinking more about the outside running. I think this might be an exploitable thing with the right RB. Ladanian Thomlison might be unstoppable on an outside run offense. I might do a couple 20 season blocks on the Chargers just to see if that works.

I also wanted to clarify my point about run effectiveness not impacting the offense. Understand that this data is all relative and I was testing gameplan options. Obviously if you have a stud runner and a strong line the running game will make your offensive efficiency more effective than a team with just average players at those positions running the same gameplan. My point is that changing running gameplans really doesn't have much effect on how the talent contributes to the offense. Obviously it's going to matter if you run different run to pass ratios, but otherwise it doesn't seem to matter much whether the team runs efficiently or not with relationship to its gameplanning. It seems improving pass efficiency is far more important in improving overall offensive efficiency. So it's pretty much a typical FoF game in that respect. Quarterbacks are still hugely important.

[JeffW]

I think the confounding factor you're running into is that FOF doesn't track situational success.

RBs on good teams will be running the ball in a lot of obvious running situations to eat the clock, so they will tend to have poor yards/carry compared to RBs on bad teams who are running in balanced or pass heavy situations.

FOF also doesn't track success rate--you'd much rather have a player who gains 4 yards/carry with a standard deviation of 1 than a player who gains 4 yards/carry with a standard deviation of 5. The latter player is going to kill many drives by putting you in unfavorable second and third down situations as well as outright forcing punts on third down failures. Success rate measures how often a back gains significant yardage towards a first down(this stat is adjusted for fourth quarter leads, etc, but that's needless complexity).

Yards/carry doesn't begin to tell the whole story. Yards/pass attempt is afflicted with these problems to a lesser extent, but it shows such a high correlation to winning% that it's a good metric as is.

[Leonidas]

I think the confounding factor you're running into is that FOF doesn't track situational success.

RBs on good teams will be running the ball in a lot of obvious running situations to eat the clock, so they will tend to have poor yards/carry compared to RBs on bad teams who are running in balanced or pass heavy situations.

FOF also doesn't track success rate--you'd much rather have a player who gains 4 yards/carry with a standard deviation of 1 than a player who gains 4 yards/carry with a standard deviation of 5. The latter player is going to kill many drives by putting you in unfavorable second and third down situations as well as outright forcing punts on third down failures. Success rate measures how often a back gains significant yardage towards a first down(this stat is adjusted for fourth quarter leads, etc, but that's needless complexity).

Yards/carry doesn't begin to tell the whole story. Yards/pass attempt is afflicted with these problems to a lesser extent, but it shows such a high correlation to winning% that it's a good metric as is.
That's why I eventually want to do a study on balances vs inside vs outside running games and see if there's a correlation with 3rd down efficiency and red zone efficiency. Those are the closest metrics the game offers to a situational statistic.

[Dutch]

http://www.benelou.com/iB_html/non-cgi/emoticons/rock.gif (javascript:emoticon('???'))

Over a 20 year test the players change, and therefore their strengths change. There just isn't "a best" gameplan out there when you break it down to individual seasons (such as in MP).

I'm not knocking the test or the results and believe that regardless of the personnel, this generic "sweet spot" adustment to passing will improve your team. I'm making a minor side point that in each individual season, that might hurt your team if there are other areas of particular strength.

[Ben E Lou]

Over a 20 year test the players change, and therefore their strenghts change. There just isn't "a best" gameplan out there when you break it down to individual seasons (such as in MP).

I'm not knocking the test or the results and believe that regardless of the personnel, this generic "sweet spot" adustment to passing will improve your team. I'm making a minor side point that in each individual season, that might hurt your team if there are other areas of particular strength.
Oh, I was agreeing with your assessment, based heavily on the attempts of Williamsburg and Ann Arbor to run a similar offense as mine in IHOF, and my own attempt to run that offense with the *exact* same gameplan in the CFL with Honolulu. There's really no big secret or some "best" game plan to explain Duncan's success, and the difficulties of emulating it, even when I used an *identical* set of passing distances, and very similar run/pass percentages in another league. My tests seem to indicate that the strengths and weaknesses of your personnel are just as important as ever in FOF2K7.

[henry296]

Over a 20 year test the players change, and therefore their strengths change. There just isn't "a best" gameplan out there when you break it down to individual seasons (such as in MP).

I'm not knocking the test or the results and believe that regardless of the personnel, this generic "sweet spot" adustment to passing will improve your team. I'm making a minor side point that in each individual season, that might hurt your team if there are other areas of particular strength.

I think he ran the same season 20 times not a 20 year sim. Therefore, he found a better game plan for the Bengals. It may not necessarily work for the Steelers.

[Dutch]

I think he ran the same season 20 times not a 20 year sim. Therefore, he found a better game plan for the Bengals. It may not necessarily work for the Steelers.

Ah, okay. So that particular set of players created this success. That changes this a bit because as you pointed out, it may not work for another team....or the same team one year removed.

[Dutch]

Oh, I was agreeing with your assessment, based heavily on the attempts of Williamsburg and Ann Arbor to run a similar offense as mine in IHOF, and my own attempt to run that offense with the *exact* same gameplan in the CFL with Honolulu. There's really no big secret or some "best" game plan to explain Duncan's success, and the difficulties of emulating it, even when I used an *identical* set of passing distances, and very similar run/pass percentages in another league. My tests seem to indicate that the strengths and weaknesses of your personnel is just as important as ever in FOF2K7.

I've tried to reverse engineer your gameplan in SP mode and came to the conclusion that either you really suck at gameplanning and Duncan and Ellard and your defense really is the shiznit, or I just horribly failed at recreating your gameplan. :)

[Izulde]

Yards per pass attempt *does* make a huge difference, I noticed.

Brad Pepper I'm looking at you.

[Leonidas]

Ah, okay. So that particular set of players created this success. That changes this a bit because as you pointed out, it may not work for another team....or the same team one year removed.Ran the same gameplan with Indy, and it killed. I thought with my Indy baseline no way it would improved anything. But I was wrong. The AI baseline is the first number, my gameplan results are what's in parentheses

Rush yards per season: 1906 (1799)
yards per carry: 4.2 (4.27)
passing yards per season: 4368.9 (4830.3)
completion %: 65.5% (62.3%)
yards per attempt: 7.75 (7.90)
TD passes per season: 31.85 (41.25)
int per season: 9.05 (9.6) while averaging 50 more pass attempts
points per season: 442.30 (484.65)
wins per season: 13.15 (13.85) I even had two undefeated seasons out of the 20 year run with my gameplan, none with the AI

So I got to thinking, maybe it's just a great offense for a team loaded at QB and receiver. So I turned it around and used it with San Diego, which Rivers ir really pretty terrible. The AI on SD average 497 rush attempts per season to 454 passes so it was a huge running offense that averaged over 2000 yards rushing per season and only 2800 yards passing on the AI baseline. So I ran it (and mind you, forgot to change the formations to match Rivers style, not sure what effect that might have had) and here's what I got (AI first, mine in parentheses)

Rush yards per season: 2034.75 (1636.9)
yards per carry: 4.09 (4.11)
passing yards per season: 2811.85 (3562.3)
completion %: 61.64% (54.41%)
yards per attempt: 6.19 (6.17)
TD passes per season: 15.15 (21.15)
int per season: 17.3 (25.2)
points per season: 280.7 (306.1)
wins per season: 7.80 (8.10)

Granted, the gains were more modest with the limited passing talent of San Diego. but it still improved around 400 yards total offense a season (about 25 a game) and scoring was up almost 26 points. I also had yards per attempt go lower with San Diego. I attribute that to the completion percentage dropping so much. I suspect if I turned it around to yards per completion it would be a huge gain. Maybe the better correlation is to go yards per completion, not per attempt. Might bear more studying.

My best advice from all this is try and find a balance between getting more yards per pass attempt and passing percentage. And I wouldn't get overally wrapped up on gameplanning for the run game. As long as you have a good line and a good runner I don't think there's a whole lot more you can do with it to improve the offense overall.

[mrsimperless]

Its posts like this that suck the fun out of the game for me.

It's not that I'm upset at you for performing your experiments. In fact, I admire what you did. And it's not that "breaking the code" takes the fun away from me. I think it more comes down to the fact that it reminds me that it is in fact fust a computer game and then this elaborate fantasy I've built in my head comes crashing down like a poorly constructed house of cards.

For example, in the real NFL no "one magic strategy" always seems to work. You have to tailor to what other teams are doing and your strengths and weaknesses. Its just disappointing to know that there is only one gameplanning strategy that will improve my winning chances.

Sorry, again I'm not trying to put down your efforts. I don't even have a proposed solution to my depression. I just wish my fictional leagues were all real!

[Vinatieri for Prez]

I have to disagree. Without the right personnel, it won't work. If you have injuries and didn't manage your cap, the lack of depth will not be saved by the gameplan. Because there is randomization, one season can always beat the odds no problem. What about turnovers. And so on. And if you're playing MP, then just go ahead and try this same gameplan over and over again. May it help you win more. Maybe. Is there any guarantee you will win the Superbowl. No. And don't forget, the other team may be doing the same to you.

While I find this post informative, I don't think it's near the gamebuster you think it is.

[Dutch]

I have to disagree. Without the right personnel, it won't work. If you have injuries and didn't manage your cap, the lack of depth will not be saved by the gameplan. Because there is randomization, one season can always beat the odds no problem. What about turnovers. And so on. And if you're playing MP, then just go ahead and try this same gameplan over and over again. May it help you win more. Maybe. Is there any guarantee you will win the Superbowl. No. And don't forget, the other team may be doing the same to you.

While I find this post informative, I don't think it's near the gamebuster you think it is.

Exactly. You can go back across the history of the NFL and find that teams with a high completion % and high YPA or YPC were more successful than those with lower #'s. But just like in the NFL, with FOF, there is way more to it than that. If you have a 99/99 RB (think Barry Sanders) and a 10/57 QB (Chris Simms) you do not abandon the run in an effort to get more of the magical "perfect passing ratio" from your sucky QB. It simply is not a one-size-fits-all code breaker. In any given year, you can modify your gameplan to meet your team's strengths. And in many of those years, it may be more wise to completely ignore the information of this thread, which is simply a law of averages for a particular team.

[albionmoonlight]

In IHOF, my starting QB went down for the season at the end of game 2. He was averaging a QB rating of over 125 for those two games. His backup, with a different skill set, came in, and I tried to continue to use the starter's game plan. We quickly lost two games.

Once, however, I changed gameplans to fit the backup, he ended up with a QB rating over 100, and the offense clicked under him for the rest of the season.

Gameplanning is really important. But I agree that one gameplan cannot fit all situations. You have to be able to adapt it to your roster, or you will get smoked.

[OldGiants]

Leonidas,

Nice work.

If I might make one small suggestion in your running game analyis, take a look at run direction on the annual summary screens on the scouting report of all the teams. I've found a reasonably high coincidence of higher 'run right' percentages and good teams. By that I mean, a simple sort of 'run right' percentages tracked 'wins' quite closely. The major differences were largely explained by injuries.

This data was taken from two SP seasons (2018 and 2019, so lots of new players) and had no injury control and was a test of AI game planning, not human. I have no idea why good AI teams run right more than bad teams. It could be because they have decent blockers at all five line positions and bad teams don't. It could have been, and likely was, simply random like so much of FOF.

Just a thought.

[albionmoonlight]

Leonidas,

Nice work.

If I might make one small suggestion in your running game analyis, take a look at run direction on the annual summary screens on the scouting report of all the teams. I've found a reasonably high coincidence of higher 'run right' percentages and good teams. By that I mean, a simple sort of 'run right' percentages tracked 'wins' quite closely. The major differences were largely explained by injuries.

This data was taken from two SP seasons (2018 and 2019, so lots of new players) and had no injury control and was a test of AI game planning, not human. I have no idea why good AI teams run right more than bad teams. It could be because they have decent blockers at all five line positions and bad teams don't. It could have been, and likely was, simply random like so much of FOF.

Just a thought.

Do Tight Ends in FOF always line up to the right side?

[Leonidas]

I have to disagree. Without the right personnel, it won't work. If you have injuries and didn't manage your cap, the lack of depth will not be saved by the gameplan. Because there is randomization, one season can always beat the odds no problem. What about turnovers. And so on. And if you're playing MP, then just go ahead and try this same gameplan over and over again. May it help you win more. Maybe. Is there any guarantee you will win the Superbowl. No. And don't forget, the other team may be doing the same to you.

While I find this post informative, I don't think it's near the gamebuster you think it is.You're missing the point of my whole study. It's not to find that "trick" gameplan that beats the game every time. It's to find a gameplan that runs the most efficient offense relative to how the AI runs it. Many people let the AI do the gameplanning, and that's fine because I think the AI does a pretty good job matching a team's talent with a gameplan. My goal was to devise a gameplan that beats the AI, therefore find a gameplan worthy of using in the game. If I can't beat the AI then why bother gameplanning at all? So my objective was to see if it could be done. And along the way I have found some fascinating things about how this game runs that may provide insight into even better gameplanning.

I didn't win everything all the time with this gameplan. OK, Indy came close. In fact with San Diego I was really happy to get the team to average just a hair over 8 wins a season. But I was better than the AI which was winning less than 8.

And it's not a simple plug and play gameplan. My situational play calling stays the same, but I have to tailor the passing lengths and run directions to match the personnel for each team. But so far, when I have tailored it properly I have eventually been able to beat the AI with every team I tried it with. Now I'm working on defense.

[Subby]

I think this provides a nice baseline for what a number of folks are having success with in MP...

[Warhammer]

I think there are a couple of caveats to what you are seeing. First, you're running these tests with Indianapolis and Cinncinatti. Offensively, those are probably the best squads in FOF2k7. You might find drastically different results if you are looking at a squad like Chicago or San Diego.

I think the premise is correct, but I think some of the conclusions are not. Higher YPA can be accomplished in two ways and very much depends upon the skill set of your roster. If you have a team with a low completion %, you probably want to increase the shorter passes to increase your completion % which should increase your YPA because you are completing more passes. If you are completing a high % of passes, you should increase the % of long passes so you sacrifice some completions for going further downfield.

The one problem I do have with FOF, is that the deep passing game is too powerful, but I think things might change with the new coverage options.