Ushiakwa's Big Reveal & Big Questions aka A long boring history

[Ushikawa]

Thanks to the great work from Squirrel, Quik and Guests are explaining some of the big things in the game. I learned some stuff and consider myself maybe not a top ten 10 GM but certainly someone who knows the game in and out.

However, they aren't going anywhere near what I have been struggling to examine since FOF8 came out. FYI, I am going to assume that the reader has read and understood all the help and philosophy docs ad nauseam as well as read all the posts here to pick up on other "proven" concepts. Explaining that stuff to people is the Lord's work but I am an impatient, arrogant jerk.

My greatest interest relies what are the mechanics of a play. How can I use a given set of players optimally? Do certain things work better than others a priori?


So when FOF8 first came out I started trying to exploit matchups and still do some of this for playoffs and occasionally for divisonal games. Here is a sample of what that might have looked like. Two Tabs: one for O and one for D.

We will start with O. Pretty much I copied the personnel for each package (very few GMs change personnel barring injury or maybe for DIME or something) and then identify weak and strong points. (I used to be underemployed and then spent 5 hours a day on conference calls babysitting nerds). Green is where I have an advantage and Red is where I may have a disadvantage. Generally I rated these things in the context of my own team so if I have a 95 RR receiver I wouldn't mark a 70 man-bump CB as red but if my WR had 50 RR then I would. Sometimes I even just did it on a piece of paper.

So what happened?

Well I still wasn't very good or at least I thought I could be better. I realized maybe I need to mix it up a bit can't run most of the time at the same holes and throw the short stuff at the RCB and the deep stuff at the LCB for example.

OK, added a bit more variety. Very mild improvement, still underwhelming. Worth noting that I almost always have a solid 350+ points of run blocking on my OL and exclusively use the high HoleRecog and lo Elus RBs. But if I am running at their weakest defenders then I should be leading the league in YPC not just top 5, I am certain no one else is doing this. And yes, I have read the Run Defense section in the help file and understand the distribution there but still.

And passing was worse, I would target the heck outta the spot where the liability was but I never felt I was significantly getting a significant advantage.

Also you will note that occasionally I will even log the Defensive GPs on my opponent (again, you would be surprised how many good GMs and teams do not change this much) and this definitely does make a difference especially knowing which formations to avoid your top route runner, are open to a shot in the RZ, etc.

For Defense, I still log my opponents a good bit and you can see that tab (once you get the muscle memory down this only takes 20 minutes). This definitely is an advantage and is also a huge enabler of having a specialist-focused defense. Where and when to play your Run Stuffers vs your rushing specialists. When to double and whom? I am gonna assume you can infer the advantages here.Not gonna bother explaining my shorthand and what the colors etc. mean, you get the point. (If I am invited on the show I would be happy to show others how I do this so quickly, with zilch coding).

OK, so I was a regular contender now in my very competitive league and running off an endless stream of Bowl appearances (Dawgfan-lite I would say) and such in my not-so-competitive league. But I still wanted to understand more...

[Ushikawa]

So I decided to look closer at a couple things, plays on a granular basis and even try to prove to what extent space (matchups) even exists in FOF.

I had noticed that many highly successful teams used the same dang plays every game all the way to the Conference Finals. What the heck, how could they beat me? I knew exactly what they were gonna call. Are certain plays better than others? Does my previously alluded to, learning a play increase its effectiveness.

First plays on a granular basis. Let's look at O.

Pretty straightforward, had to set some ground rules for the translation from the game logs to my Excel (even started learning some Python but then became overemployed all of a sudden) and again not gonna bore you with my shorthand or ruleset.

Worth noting before I continue:
- I design all my own plays, my house rule is it has to be Solid or better fit to my OC. NO EXCEPTIONS.
- I am always top 5 in rushing attempts and have many more rushing plays than I will ever need
- I tend to design plays as if space was real, so will run a lot of Curl/Flat, Floods, Slant/Wheel, designed to put stress on a particular coverage point/player
- I want a nice set of plays for 3rd downs where both receivers will go past the sticks, otherwise I generally have the Secondary receiver run a shorter route. I have changed this up some after seeing someone having a lot of success with Secondary receivers being the deeper route normally, think you need a Scrambler QB and strong pass pro though.
- I throw it past the sticks and am almost always top 5 in 3rd down conversion percentage (TIP:you want to do better, throw it past the dang sticks on 3rd down, and even on 2nd a good bit)
- I also throw a lot of INTs maybe due to depth of target (again almost always bottom 5 in short attempts and top 5 in medium attempts), implicit familiarity or just the QBs I have had...who knows (srsly, please tell me).

So what have I learned so far?

- some plays definitely seem to work better at least for a certain team. I have yet to try to take successful plays from one team and use them elsewhere.
- many times it seems to last for a season, one of my top plays for one season will not necessarily be as marginally successful the next season even with the same personnel
- Curls aren't better than Comebacks nor Outs than Slants or anything cheesy like that. However for your team/OC/year you may find certain routes over or underperform expectations. Then you load up on them too much and whoops now they suck and the other route is better.

Long ago I tracked who the receiver was to try to identify which bars were more important but now I jsut want to target so I can evaluate during the season which Plays have been successful in what situations more frequently.

I just manually do my analysis with the filters, at one point I made a Pivot Table to see what the PR% YAC% and even % to get N yards would be but I can't find it and suck at Excel. I have the inputs pretty much set up to make a neat little outcome distribution chart but still haven't got around to it. For others doing something similar I would just strongly suggest using something more like DVOA or capping YAC yardage at 20 or something, a couple big plays can really eff with your averages.

I do the same charting exercise for Defense sometimes when I feel like somethign aint working as it should but not every season. Plenty to learn from that exercise too though:
- Vanilla Cover 2 sucks donkey
- blitzing as you might imagine gives you more incompletes but also really opens you up to bigger YAC gains
- dropping 8 seems to work very well for some teams and not at all for others, reckon if you have 3 strong pass rushers then it is worth trying some

[Ushikawa]

So that leaves us with the big question: does space (matchups) exist in FOF? Even 38% of the time?

Super hard to test.

But I decided to try to understand the mechanics of a passing play.

So we have all noticed that if I target my Z 10 times, your LCB isn't going to have 10 targets against. So what si happening here...

Plays impacted by a pass rush have no target. So the pass rush die rolls happen before the receiver and pass defender get rolled I think. Perhaps Adjust to Ball can bring in some errant hurried passes but it seems unlikely given that on any hurry no defender is charged with a target

Do hurries or consistent pressure lead to INTs and the like, no effing idea, certainly seems like it. Are there degrees of bad throw or good throw (ever notice how many receptions don' thave anyone with a minus?) possibly?

Anyways, to oversimplify there is no pass rush and the QB releases the ball. To the Z running an Out and WTF the WLB or FS gets an interception. So obviously there is some sort of distribution going on here. Passes:
- to the Z, or
- that are Outs
- maybe just the X axis doesn't exist and any player in a Short Zone has a good chance to make a play on an Out

INTs in particular seem to me to be "randomly" assigned based on position and bar. So I limited my analysis to Passes Defensed.

I logged 149 plays: formation, target, route, D-call, D-target, D-target-coverage:
-21 of the plays made absolutely no sense i.e. the RCB playing Bump deflects the pass to the Y running a Corner
- 33 made very little sense i.e. the SS in a Cover-2 makes a play on the Comeback-3 route to the X
- 35 made some sense i.e. the NB in a Cover 2 (short zone) look deflects the pass to the Z running a Deep In-6
- 59 times it made sense i.e. not just the the LCB defends the Out to the Z but also things like the SS defends the Curl-4 to the Z out of the Cover 2

So there seems to be some sort of distribution at work here as well which is why my matchup offense (and defense) wasn't leading to me going Dawgfan against a competitive league.

What are the further implications?
- Should we just put our best players in at LCB/SS always since he will get the majority of the nonsense targets?
- Is the matchup real but just the assignment of stats like passes defensed distributed according to position and bars? I personally like this theory a lot, seems to happen with sack allowed

[Ushikawa]

One big thing that should be acknowledged and could be throwing all the hours I have put into this off.

What if there is an "error" in the code and left and right are reversed on D (even more complicated, some of the time).

I first grew suspicious of this, when the ideal weight for the 3-T was lower than the 1-T. Then that was fixed but in a 43-Over it lists 1T then 3T in the depth chart and on the game logs, but that should be reversed due to common sense and the philosophy docs.

Other pieces of circumstantial "evidence":
- when the blitz assignment came out it said something about the primary blitzer is the SLB but in the game logs and Philosophy it is the WLB
- the +- on plays with a pass rush are seemingly out of whack with who gets the hurry/pressure
- the fact that I don't run for 5+ YPC with my 400 point OL running at their weakest defenders or dominate teams with one lesser CB/S

[Ushikawa]

The next thing I am going to study I think is if I run the same running play 10 times against the same front/personnel: what is the distribution of the key blocks. Imagine the value of knowing which OL has the highest chance of being the key blocker (and against whom) for a given running play. I may finally get my 5 YPC haha.

[Ushikawa]

One final note (very easy to do):
Copy the plus minus from the game logs and create a Pivot
Sample link

- let's you know who is doing well (at least as you are using them)

[Squirrel]

Interesting stuff in here, appreciate you posting it up.

Have to be honest I have very little idea on how the game engine works, I find myself needing to think about it statistically more than anything specific to any given play or any given game...which only gets me so far.

I think that's the joy of American Football as a sport to some degree. The sample sizes are necessarily small, the ability to infer from the data you have is always limited, there is just a load of randomness in there and not much anyone can do about it. I'd view FOF as quite a bit less random than the NFL but nonetheless there is plenty of space for luck to play its part.

[Ternvig]

This is FANTASTIC Ushikawa! Thank you for taking the time and sharing it with us.

Vanilla Cover 2 sucks! he difference it did for me change my 4-3 D i CFL from C2 to C1 and Tamba.

Match-ups: in the CFL I have no rungame (yet) so I do a lot of screens to compensate. The match-ups on screen plays has baffled me. I have noticed that even ILBs and Safetys get Def and INT on screen plays.

I dont have the skills to do it, but I would really like to take a closer look at where the game mechanics deviates pass play from play action. I believe, that there is a 'dice roll' i regards of 'Ha! - did you really think we would run the ball suckers!" and what both negative outcome and positiv outcome of the dice-role effects. Is positiv equal less sack/hurries, more completions, better YAC, or fewer INT or all of them?

The same could be done with finess-running.

My reasoning behind is from the FOF7 days, where you would get an e.i 'The defense is expecting a run' and so on. There could be some coding in regards of this still in the game FOF8.

[Ushikawa]

Originally Posted by Ternvig

Match-ups: in the CFL I have no rungame (yet) so I do a lot of screens to compensate. The match-ups on screen plays has baffled me. I have noticed that even ILBs and Safetys get Def and INT on screen plays.

I dont have the skills to do it, but I would really like to take a closer look at where the game mechanics deviates pass play from play action. I believe, that there is a 'dice roll' i regards of 'Ha! - did you really think we would run the ball suckers!" and what both negative outcome and positiv outcome of the dice-role effects. Is positiv equal less sack/hurries, more completions, better YAC, or fewer INT or all of them?

The same could be done with finess-running.

I look to use PA and finesse against guys with low play recognition and yes the most common defender against a screen is a guy in short zone so LBs rack up plays there.

Finesse plays never seem as consistent to em which is likely by design much more likely to be a loss or even cloud of dust than a regular run but obviously are not totally useless.

I will pull my data later on Finesse vs Normal running and PlayAction vs Normal passing.

[bomber33bomber]

WHAT DID HE SAY????

[QuikSand]

I definitely like to see the additional contributions to the conversation. Not sure, I can unravel much of what's here...but any efforts to sort out whether a concept or approach "works" in this game seems useful to me.

[garion333]

Originally Posted by Ushikawa

One big thing that should be acknowledged and could be throwing all the hours I have put into this off.

What if there is an "error" in the code and left and right are reversed on D (even more complicated, some of the time).

...

Other pieces of circumstantial "evidence":
- when the blitz assignment came out it said something about the primary blitzer is the SLB but in the game logs and Philosophy it is the WLB
- the +- on plays with a pass rush are seemingly out of whack with who gets the hurry/pressure
- the fact that I don't run for 5+ YPC with my 400 point OL running at their weakest defenders or dominate teams with one lesser CB/S

I've looooooong wondered if running right/left on offense is broken. Or if it's the D that's broken. When I try and run at someone's weaker run defenders it doesn't work, as you allude to. I've long since stuck to ignoring the players on the field and focus on stats. If a team sucks at defending left/midd/right then I run at that primarily. (Yes, hard to do early in the season.)

Hurries, sacks, etc., I truly believe are done based on groupings and whoever ends up with the dice roll on a failed pass block. I've tried pass rushing up the middle against folks giving up 2%+ sacks but it doesn't seem to do a damn thing. It's better to pass rush from the edge it seems, despite certain guards being awful at pass blocking.

The pass rushing wlb/SLB thing in the documentation and in-game screens has bugged me for years. I assumed we all simply ignored it and moved on with life. WLB was previously the primary pass rusher. I believe SLB in depth and gameplanning is primary pass rusher in FOF8.

[Sharkn20]

I think it goes by groups, when is a run, RB + runner abilities against the Front 7 or 8 to determine if running back is stuffed at the line or advances to the 2nd or 3rd level, then last check to see if the Safeties tackle him.

Passing plays should work similar, you can target who to throw to and in what area, but after that is passing bars / receiving bars and defending bars of the unit.

[Front Office Midget]

Are we sure about vanilla Cover 2? I use it as my primary defense most of the time with success.

[Hammer]

Originally Posted by Front Office Midget

Are we sure about vanilla Cover 2? I use it as my primary defense most of the time with success.

Yeah I use it more than any other defense. Just a bend but don't break approach. My guess is it would produce a better ypp for the D than more aggressive calls. However the disruption of more aggressive calls can work for you.

For me it depends on the make up of your D. Is there an obvious WR for DC, can your LBs blitz well. For me vanilla Cover 2 is my standard fall back option when there is no obvious reason to take an aggressive approach. I will continue to use it as my preferred D.

When you look at Defensive Philosohpy in the help file it seems a good D with no obvious weakness. More so than any other D IMO.

[Sharkn20]

Vanilla Cover 2 is pretty good if you have the appropriate players to execute, obviously you are vulnerable to fly / fade / post / corner routes, specially if your back end is not very good, or your line not able to put pressure, but with a solid DL and Cover orientated DBs it's very good.

[Dawgfan19]

For defensive ratings against the run, I compiled a grid of responsibility for each defensive position. For instance, the left defensive end had 25 percent of his run rating based on his team's performance against runs around right end and only 4 percent of his rating based on runs around left end.


Each team has its specific strengths, and part of the success of your running game will depend on your ability to recognize the strengths of your offensive line and the weaknesses in opposing defenses.

[Dawgfan19]

The above post is a portion of the help text describing how run ratings were compiled. Jim seems to tease us with small tidbits without completely revealing how the game engine works.

I believe he is telling us the respective offense and defensive position groups play a role in the play outcome. However, the ratings of the players based on run direction are weighed more. So, an offensive run to the right, the LDE run rating plays a bigger role than the RDE.

[Ushikawa]

Exactly, my general idea is to run at the weaker defenders bc there is a greater chance their bars will be rolled for a given play but stay balanced of course and consider the overall shape of the run D front because 25% of the time is not a lot.

And for D, this is where logging run direction by formation can be very useful. You don't want that run-stopping DE playing a lot but if you see that they run to a certain edge out of a certain formation a good bit then it can be a nice advantage for those handful of plays.

[Ushikawa]

This from my last couple seasons in CFL, very good DL, excellent LBs, cover CBs and solid prototype Ss.

RUN D '58 - 43 Over
C1B P1B C1 P1 C2 C3/4 P2 T2
2.9 3.98 2.24 4.04 4.59 5 3.15 4.82

RUN D '57 - 43 Under (used 0 run D DE at times)
C1 C2 C34 MM P1 P2 T2
3.74 5.25 3.6 3.07 3.91 4.71 6.78

PASS D '58
C1bl C1B C1 P1bl P1B P1 C2 C3C P2 T2
2.96 3.08 4.73 4.18 4.1 5.43 8.54 5.79 2.78 5.64
C1Bbl P1Bbl C2bl P2bl T2bl T2D
10.57 9.57 2.29 2.89 7.5 4.22

Caveats:
- Sample size
- Roster Bias
- Problems with Mean
- Selection Bias as I am running certain Ds in certain situations and yes when I don't have a solid tactic to apply I run C2 like many of us
- I do change the raw yardage a bit in a similar way to ANY+ for TDs & INTs

Some pretty compelling data all the same.

[Ternvig]

Thank you for sharing

I can't even start to imagine the Excel voodoo that went into extracting that kind of data.

I'm surprised that Tampa 2 is that inefficient.