2006-2031: Madden 18 Sim Stats vs. Real NFL Data

[DeuceDouglas]

Back in July I posted a thread showing ten seasons worth of Madden 17 sim data and compared it to the real-life NFL numbers. I decided to do this again with Madden 18 to see how much, if anything, had changed. This time I also decided to sim an extra five seasons and do 15 seasons rather than 10 just to get a more complete picture of how things change over time.

Again I'll start with the league-wide numbers:



So it's very similar to M17 which I initially thought wasn't too bad but when I started looking at the numbers compared to all-time NFL seasons was when I realized just how bad some of these numbers are. Everything that is highlighted yellow would be an all-time NFL record for any given season. I'll start with the major issues:

Pass TD's, Rush TD's, Scoring.

This is something that was off in M17 and appears to be even worse with M18. Every season I simmed had an all-time high of both passing and rushing which in turn led to all-time highs in PPG for every season. The passing TD's weren't nearly as bad at the beginning but as time went on they started to become really out of control. Rushing TD's however were out of control from the beginning and didn't deviate too much but were absurdly high in every season. The most passing TD's in any season in NFL history was 842 in 2015 and the lowest I saw in any season was 893 with most seasons cracking 900 and the last four reaching the 1000's.

The rushing TD's is where it was really bad though. The most ever in a single season was 487 all the way back in 1979 and the most in the last eleven years was 476. The first season I simmed had 719. I don't know what causes it but there is just way too many. Looking at the leaders you'll see lots of weird stat lines with #2 RB's scoring tons of touchdowns on limited amounts of carries.

Both of these also results in absurdly high scoring from every team. I had an idea this was an issue playing 17 because of how many times you see scores like 52-45, 48-45, etc. scrolling through the feed but this confirms it. Scoring league-wide is pretty consistently around 3-PPG higher than the highest NFL season ever which may not seem like a lot but when you consider that over 256 games it is really, really high compared to anything ever seen in the NFL.

Where art thou Sacks...

So sacks. As you can see there is an extremely rapid drop off about halfway through the sim and by the end they were more than cut in half from the first season. The second season nearly broke the all-time record but there is no doubt that something is going wrong here. My guess is that there is a huge lack of pass rushing talent coming into CFM paired with a lot of bad AI decision making where the few good pass rushers end up out of scheme and such. Looking at the rosters, it didn't seem like the OL's were good enough or the DE's and OLB's were bad enough for there to be that significant of a drop off but looking at certain attributes it seems like there are very few players at all with good FMV. More players have good PMV. I thought that it might start correcting itself because in 2028 Bosa tied the sack record but it seemed to be an anomaly at that point any way. By the end of the sim the league leader in sacks per team was at around 30 and I saw two teams break the record for least sacks in a season as well. I'm kind of interested to see how bad it will continue to get over more time.

The Running Game

The running numbers as a whole are not very good. Carries are too high. They didn't fluctuate too much but there's just too many on a per game basis. Every year there is more carries league wide than any of the past eleven NFL seasons which in turn affects YPG. Obviously the YPG is going to be higher but similarly to sacks there is a rapid decline from the beginning on both yards and yards per carry. Yards per carry was already too low to begin with and just continued to dwindle over time. By the end of the sim teams were barely cracking 100 yards per game and were just under 3.3 per carry which is absurdly low. One thing I noticed as I got deeper into the sim as well is that long runs essentially became non-existent. This wasn't really a shock but some of the numbers I'd see were extremely odd. I was consistently seeing backs with hundreds of carries that had long runs of 5 or 6 yards. Broken tackles and YAC were very low for most RB's as well. Looking at OL and RB's didn't really seem to show anything out of the ordinary. I initially thought that because of the sack numbers paired with the rushing decline that teams would be built full of lopsided lineman but they are actually pretty balanced. Looking at RB's showed that very few RB's had above 80 for ELU and most of the ones that did were 3rd or 4th on the depth chart. JKM and SPM are also pretty low and it looks like either the AI pumps it's XP into TRU and SFA or the draft class RB's are largely weighted towards those two because there are a ton of players highly rated in both throughout the league.

Plays Per Game

This was another issue present in M17. Plays per game is too high. I doubt that this really affects too much with how putrid the running game in sim stats appears to be but there's about four plays per team per game more than what might be expected. This isn't a huge deal like I said but could play a part in the scoring although I doubt that is case. It's just something that when compared to the real NFL data really stands out as being significantly higher.

Miscellaneous Thoughts

Interceptions seemed perfect. Some seasons high, some seasons low but nothing out of the ordinary on the high or low end. Out of all the stats I tracked it was by far the best.

I'm not sure what the cause might be but as you can see after season 10 passing numbers were off the charts setting all-time records in basically every category. My guess is there's either way too many good QB's compared to the stock roster or something along those lines. I looked at DB's and they don't seem that poor when compared to the stock rosters but I can't be sure about it but the way the passing numbers start to skyrocket right around the same time all the real players are out of the league makes me think that's the case.

Individual Leaders

Here's the Top-10 leaders for a bunch of the major statistical categories. As you can see by looking at any of these, if you're just simming seasons and looking at leaders, not too much looks out of the ordinary. The sacks and rushing TD's are the most obvious and so are the passing yards to a certain degree but just looking at these numbers will give you the impression that there isn't much wrong. Again, anything in yellow is an NFL record.

Pass Completions

Spoiler

Pass Attempts

Spoiler

Passing Yards

Spoiler

Passing Touchdowns

Spoiler

Interceptions Thrown

Spoiler

Times Sacked

Spoiler

Carries

Spoiler

Rushing Yards

Spoiler

Rushing Touchdowns

Spoiler

Receptions

Spoiler

Receiving Yards

Spoiler

Receiving Touchdowns

Spoiler

Sacks

Spoiler

Interceptions

Spoiler

Nothing too out of the ordinary in any of that. The sack record was tied four times but I didn't have any seasons like I did in my M17 sim where a player was getting over 30 sacks. Like I said, just looking at these numbers, it's hard to notice that anything might be going wrong. Towards the end you definitely can tell something is up with sacks and rushing yards but even the passing numbers don't look too outrageous but when you start looking at it as a whole, it looks a lot worse.

Overall, it's hard to have much in way of feedback because I'm not entirely sure what causes certain things. The rushing seems to definitely be a sim engine issue though. I played an entire season using Play The Moment and my top RB Kareem Hunt had 317 carries and a long of 5 which falls in line with what I saw here. Other issues like passing, sacks, etc. that deteriorate over time seems to me like it's likely the incoming draft classes not matching up well with stock rosters. The way the league starts to turn over and the numbers start to drastically change once more and more of the stock players are gone makes me think that is the main culprit but without knowing the inner workings it's impossible to say.

Another theory I have is that it's not so much the players but how the AI progresses them and spends XP. Just looking at a few positions there were a ton of players that looked very, very similar in regards to their position specific attributes and it makes me wonder if the AI is spending XP in a way that actually works against the sim engine or atleast what the sim engine is using. Almost all RB's had really good Trucking and Stiff Arm ratings but I didn't see one player with a Spin or Juke move in the 90's. Same for the pass rushers.

I think one thing I might do is go through the stock roster after final cuts and see what the average OVR is for each position and then compare it with the current roster in season 15 and see what it looks like. I could even do that with certain attributes as well and I don't think it would take too long. But hopefully this information is useful in one way or another and it can lead to getting some of these issues identified and corrected.

[swanklax]

Have you looked at TDawg's XP slider thread? I would be curious about the differences in the metrics you discuss between base XP sliders and his final version. His sliders are designed to keep the talent dispersion in the league consistent with the original roster over years of simulation with CPU progression. Your findings suggest that the sim results get more unrealistical over time and adjusted XP sliders may fix that.

This may be difficult, but I'd be more interested in the aggregated and averaged stats from 15 simulations of the 2017 season than in just simply simming 15 years ahead because this method would provide far better control for the variables related to XP and player progression. The results of this would highlight any fundamental flaws in the sim engine without as many confounding variables.

[Prilosec]

very keen and good work.
you can actually create stats that are 95% to classic era football, with the older rosters.
If you understand the connections b/t ratings and their ceilings, with the application of generic offensive PB and use of particular scheme - you will see how great the madden sim engine can be.

I am not familiar with NBA 2K, but I have to say that when you understand the madden sim engine - it is the best of any console video game.
It is very cool to see guys like Terry Bradshaw and Dan Marino have near-exact sim stats in a video game in 2017.

[Steelers4190]

How do FGs and Punts compare?

[triplechin]

Quote: Originally Posted by Prilosec

very keen and good work. you can actually create stats that are 95% to classic era football, with the older rosters. If you understand the connections b/t ratings and their ceilings, with the application of generic offensive PB and use of particular scheme - you will see how great the madden sim engine can be. I am not familiar with NBA 2K, but I have to say that when you understand the madden sim engine - it is the best of any console video game. It is very cool to see guys like Terry Bradshaw and Dan Marino have near-exact sim stats in a video game in 2017.

Sorry if I am missing something obvious but where does it show Marino's and Bradshaw's stats?

[tdawg3782]

Deuce, last year the sack issue later on in CFM was caused by most all incoming talented QBs having the Ideal Sense Pressure trait. By later nearly every starting QB has that trait and it really affects sim stay sacks. If you still have the CFM saved I check to see if hats still the case.

As far as HB TDs the games base roster is stupid loaded with 80+ Ovr guys. I mean loaded. So that could be an issue.

QBs left at 100% XP Slider will progress way to good and you'll have anywhere from 30-50 players at 85+. Could be an issue there.

Just some thoughts off the top of my head. Good stuff man.

Sent from my SM-G950U using Operation Sports mobile app

[timhere1970]

Rushing td's are a result of the sim engine giving running backs the same yards every play. Goal line carries when being simmed are sure td's from inside the three. The explosion in offense is primarily created by way overpowered athletically gifted draft classes. It create way too many mis matches in the game. There are so many 97, 98 and 99 speed receivers that it is impossible to cover them.

[BleedGreen710]

Quote: Originally Posted by tdawg3782

Deuce, last year the sack issue later on in CFM was caused by most all incoming talented QBs having the Ideal Sense Pressure trait. By later nearly every starting QB has that trait and it really affects sim stay sacks. If you still have the CFM saved I check to see if hats still the case.

bingo, thats still what causes it. sadly they did not fix this in madden 18