Porzingod's In-Depth Sim Engine Analysis: Part I

[janmagn]

Great post, you can't imagine how much I have thought about these things myself. Will use these on my personal roster!

[ffaacc03]

Quote: Originally Posted by tetoleetd

here's something you need to consider though. when you go all out to achieve sim stats in simulated games, you can very easily effect the actual game play. a lot of people get so caught up in making tweaks to achieve sim stats, that they dont realize they are completely ruining the game play in the process. id rather have a game that plays well with a few inaccurate stats, then have a game that has the most realistic stats but plays like crap.

This is why Total Sim Control was such a great addition when it first appeared ... its approach seemed to treat on court and sim stats variables separatedly, there was a variable to control the on court pace and one to control the sim stats possesions per game (pace) for teams, etc, etc.

Independently of sliders, this should be the way to go, to have separate variables so we can address sim stat engine defficiencies with variables that dont affect/correlate the on court experience ... for the editors that do value sim stats, has been all about making compromises (less worthy road), wich can be avoided by retaking the approach that total sim control seemed to bring ... I remeber how much praise did the feature received back then and how hopefull the editors here were of the base/vision it set and its possible improvements.

At least, in this sole thing (thus I love the new related features, albeit some need tweaking and more consistency in its function), I found the new system presented on new/current gen more restrictive than that of the prior gen. With such approach fully implemented, I believe we could even account for exceptions to the rule without any worries.

Another thing I liked was the 1-100 approach on variables that you could set, talking more specifically about the variables that affect team stats, such as ppg & ppga, while the results were barely perceiveable from point to point it produced results a lot more broader than the very few that are currently achieveable, plus the correlation with the sim engine has now been reduced greatly, maybe to prevent the user from "braking" the on court experience as they are now more related than ever.

Hopefully Leftos/Simballer and company can see value in this (having more options for the user) and can have the time (within all the great stuff they add) to alter it and implement this so the course is retaken (have variables that separatedly affect each, on court and sim stats) and we can enjoy the best of both worlds: gameplay & sim stats.

[Hassan Darkside]

Quote: Originally Posted by ffaacc03

This is why Total Sim Control was such a great addition when it first appeared ... its approach seemed to treat on court and sim stats variables separatedly, there was a variable to control the on court pace and one to control the sim stats possesions per game (pace) for teams, etc, etc. Independently of sliders, this should be the way to go, to have separate variables so we can address sim stat engine defficiencies with variables that dont affect/correlate the on court experience ... for the editors that do value sim stats, has been all about making compromises (less worthy road), wich can be avoided by retaking the approach that total sim control seemed to bring ... I remeber how much praise did the feature received back then and how hopefull the editors here were of the base/vision it set and its possible improvements. At least, in this sole thing (thus I love the new related features, albeit some need tweaking and more consistency in its function), I found the new system presented on new/current gen more restrictive than that of the prior gen. With such approach fully implemented, I believe we could even account for exceptions to the rule without any worries. Another thing I liked was the 1-100 approach on variables that you could set, talking more specifically about the variables that affect team stats, such as ppg & ppga, while the results were barely perceiveable from point to point it produced results a lot more broader than the very few that are currently achieveable, plus the correlation with the sim engine has now been reduced greatly, maybe to prevent the user from "braking" the on court experience. Hopefully Leftos/Simballer and company can see value in this (having more options for the user) and can have the time (within all the great stuff they add) to alter it and implement this so the course is retaken (have variables that separatedly affect each, on court and sim stats) and we can enjoy the best of both worlds: gameplay & sim stats.

Or in this case, just a separate set of simulation sliders that only affect simulation stats. Roster editing and rating tweaks to strike that balance between good gameplay, realistic gameplay stats, and realistic sim stats is almost an impossible task. I'd like to think it would be easier to have sliders for both gameplay and sim stats and tweak accordingly than constantly trying to tweak ratings.

Sim sliders would also allow you to globally control the impact of rating changes accordingly. Think about the last update that 2k did dropping all shooting ratings by 8 points. Helped gameplay and gameplay stats but murdered simulation shooting percentages.

[psyberkayos]

Great post. 2k needs to bring you on the team. Excellent research and deduction.

BONUS: Now if you google "Porzingod" this is the second result! Cool

[ffaacc03]

Quote: Originally Posted by Hassan Darkside

Or in this case, just a separate set of simulation sliders that only affect simulation stats. Roster editing and rating tweaks to strike that balance between good gameplay, realistic gameplay stats, and realistic sim stats is almost an impossible task. I'd like to think it would be easier to have sliders for both gameplay and sim stats and tweak accordingly than constantly trying to tweak ratings. Sim sliders would also allow you to globally control the impact of rating changes accordingly. Think about the last update that 2k did dropping all shooting ratings by 8 points. Helped gameplay and gameplay stats but murdered simulation shooting percentages.

Agreed, I like having options, the more, the better ... In this case, implementing sliders only after they re implement the total sim control approach, wich was from team to team and to an extend it includded players (with the scoring 1-3 priorities) ... I mean, we need the tools to tinker/alter every variable on a per team/player base and then we need sliders ... think of it as how we approach roster editing (attributes/tendencies) and then, after everything is set to a scale, we use sliders to make global adjustments ... this way we can even account for exceptions, players such as Rodman, Bol, Chamberlain, Russell, etc.

[Porzingod]

Quote: Originally Posted by Rashidi

Drummond is a scale breaker. His production is worth a rating of 116 which is obviously not possible in 2K. Dennis Rodman is the career leader in ORB% with 17.2% and Drummond is at 17.4% through three seasons. Drummond is an exception and not the rule. 2K will not be able to replicate his production without breaking their system.

True, Drummond is an outlier, and he is on my scale as well, where his production corresponds to a 107 rating. I'd find it preferable if the scales used were broad enough that 99 ratings can capture GOAT-level performances, but that's a minor gripe in the scheme of things. The issue here is that while his production is off the charts by a good bit in both the scale used by the official roster makers and the scale I used, it's off the charts by an absolute ton in the scale that the sim engine uses.

It's possible to view a player's ORB% directly in the player scouting tab of MyLeague or MyGM, and with maxed ratings players seem to max out at around at an ORB% of around 10-11. With my scales in place Drummond gets an 11.6 ORB% , meaning that for the sim engine to get his ORB% right he would need a rating roughly in the 150's. Even in this weak era of offensive rebounding, at least a dozen players will be off the charts every season by that scale. With the official roster he gets around 9.4%, a bit over half his real life production.

So as you can see when it comes to the sim engine it's not just guys like Drummond or Rodman that are scale breakers- it's also guys like Robin Lopez and Zaza Pachulia! As a result, for someone in my position of trying to get sim stats as close to realistic as possible, there are only the following imperfect solutions:

1. Give everyone the OREB rating they ought to get on a linear basis, with anyone with an ORB% of over 11 or so getting a 99. This will yield accurate numbers for most poor offensive rebounders (though the worst ones will still be too high since a 25 rating will always yield about 1.0 ORB%). The problem here is that there are a lot of 99's and no separation between a guy like Pachulia and Drummond, who is literally about 50% better at getting offensive rebounds in real life. This also has the effect of distorting team offensive rebounding strength a bit- teams without good offensive rebounders will overperform because the worst rebounders get more than they should, while teams with elite offensive rebounders will relatively underperform because their best rebounders will be much weaker than they should be.

2. Use a scale that tries to preserve the relative strength of rebounders at the top end. Because the top rebounders will still not be able to manage close to their real life numbers, this means that everyone below them will have to be deflated somewhat as well. This has the advantage of coming closer to making the best rebounders stand out more than they otherwise would, and coming closer to preserving the relative offensive rebounding strength of teams.

I went with the second approach, imperfect though it is, but unfortunately the sim engine only leaves us imperfect options.

Quote: Originally Posted by Rashidi

The culprit is likely too many jumpers vs drives in the sim engine. Westbrook IRL: 654 FTA in 67 games (9.8 per) 2K: 459 FTA in 82 games (5.6 per) The stats scale fine but the opportunities aren't there for bigs to record them.

Not exactly. The excess of jumpers relative to inside shots/drives is not an inherent property of the sim engine. I will go into greater detail when I cover shot ratings and tendencies in Part II of my analysis, but in general the FTA issue is mostly fixable with corrected shot location tendencies, while the shot blocking issue is not.

FTA/game is not a good measure here because Westbrook mostly has a healthy Durant all year in a 2k15 simmed seasons, lowering his usage, so I use FTA/FGA.

Westbrook, last season IRL: .445 FTA/FGA
Westbrook, official roster: 0.370 FTA/FGA
Westbrook, Porzingod's: .416 FTA/FGA (8.5 FTA/game)

Note that I haven't tweaked anyone's Draw Foul tendency yet, so bringing Westbrook from his current 92 to 100 might just get his FT rate all the way there. On a side note, shouldn't Draw Foul be a rating? It has a huge effect on how good/efficient a player is, not just on his style of play.

All of this, however, has no effect on shot blocking numbers. Even with the entire league's shot location tendencies edited so that the distribution of inside shots vs. jumpers is correct, shot blocking numbers remain far too low. I think the explanation for this is that the sim engine isn't as robust as you would think, and it doesn't try to simulate which individual shots were blocked or anything like that- I think it just fudges in a certain (unrealistically low) number of blocks and calls it a day. Over 10 years of a franchise with my shot tendency edits in place for the entire league, the league leader has never averaged more than 2.5. The scale just simply does not go high enough.

Quote: Originally Posted by Rashidi

This is actually not true. 2K absolutely uses a different scale for PGs than it does other positions. I think you might be looking at it from an incorrect angle.

Ah, good catch. I didn't notice this because there are so few PG's with low enough AST% to make it apparent. It doesn't change any of my conclusions about the scales used by the sim engine to generate stats, as they are distinct from the scales used by the official roster makers to hand out ratings, but I suppose a truly complete guide would have detailed explanations and comparisons of both scales.

Quote: Originally Posted by Rashidi

FWIW I have Griffin at 73/77 A/V. He averaged 3.0 APG and 3.5 per36 in a test... though the Clippers also added a number of good passers this off-season - Lance Stephenson, Josh Smith, and Paul Pierce, so Griffin's numbers will almost assuredly drop.

Not necessarily. While having high-assist teammates would have the effect of reducing a player's assist numbers in past 2k iterations, that is no longer the case here. Players seem to put up the assist numbers you'd expect from their pass ratings pretty much regardless of who their teammates are. As with blocks, I suspect the sim engine doesn't try to figure out which shots were assisted, but instead just fudges in assist numbers for players based on their passing ratings. I have two rosters using my scales- one for the end of last season, and one incorporating this offseason's player movement. All the other results from my scale that I've posted are using the former roster, to ensure that comparisons to actual numbers from last season's NBA are valid. Here are the assist per 36 minutes numbers for next season's Clippers, in descending order:

10.2 Chris Paul
5.2 Blake Griffin
4.9 Lance Stephenson
3.5 Josh Smith
2.8 Paul Pierce

As you can see, everyone gets their numbers and Griffin is virtually the same. If there is any deflationary effect happening here it is extremely slight. With 73/77, Griffin will underperform by about 33% relative to last season's assist numbers in the sim engine, regardless of who his teammates are. I assume you went with that rating in your roster because you find it preferable for gameplay purposes, or perhaps because it seems wrong intuitively for Griffin to have similar passing ratings to CP3, but of course that is your prerogative.

If what you meant by your reference to Griffin's numbers dropping is that they are likely to drop in real life, then you may be right, but I'd suspect that it's more likely Stephenson and Smith will see most of the dropoff in real life and Griffin will stay pretty similar, since he probably won't be on the court with them too much. Pierce joining the starting lineup shouldn't be too significant, as he has done much less playmaking as he's aged (only 2.7 AST/36 last year).

Quote: Originally Posted by Rashidi

The reason the positions scale this way STATISTICALLY (SEPARATE from the rating) is because for years 2K had a problem getting PGs to post accurate assist totals. Steve Nash, Chris Paul, etc would never come close to their league-leading numbers. This was only recently fixed, and they clearly have a ways to go to make sure the rest of the league balances out.

Yup. I recall in maybe 2k12 or 2k13 Chris Paul would often be under 7 APG, and the only PG's that could get realistic assist numbers were low usage guys like Rondo. Yikes.

Quote: Originally Posted by Rashidi

I like the idea of the scales in theory, but role in the offense is what dictates assist rate, not position.

That's true in real life, of course, but as far as the sim engine understands, role in the offense and position are the exact same thing, and in the 2k sim engine position absolutely does dictate assist rate. If you take Chris Paul and force him to play C in the rotation, he'll still put up his 10 APG. If you keep him as a PG and change his natural position to C, his assist numbers are greatly reduced. It is imperfect of course, but my aim here is to figure out what works with the sim engine we have, not the one we wish we had.

Quote: Originally Posted by Rashidi

Griffin is obviously an exception and not a rule as far as big-men passing goes.

Great passing big men are uncommon, but they aren't unicorns. Some of the most important big men in the league are excellent passers, like Noah, Gasol, Cousins, etc. Using any other approach, they will always vastly underperform in assists, and it isn't just those guys. With the official roster, every non-guard will vastly underperform. All those bigs that get like 2 assists/36 will get 1. It isn't just the guys who are exceptions that is way off- it's also the guys who are the rule. And the effects here go to more than just stats- With stock rosters the league MVP will almost almost never be a big man, since bigs can't distinguish themselves in rebounds, assists, or blocks.

Quote: Originally Posted by Rashidi

I personally don't care much about sim stats and it's because of situations like this. Smart's assist rate IRL is kept in check because Turner is the primary ball handler and effectively the PG. Thomas comes off the bench IRL. In 2K the sim engine will start Thomas due to his OVR rating, which would completely change Turner's statistical output IRL.

That is of course your prerogative. I personally don't do much actual gameplay and am mostly interested in the season/franchise aspects, so sim stats are my main priority. I would argue, however, that if you think Blake Griffin is an exception among big men, so are the Celtics among teams. They are the only team in my roster with this issue of a player's backup being significantly better overall than the starter so that the CPU sets the wrong starters in comparison to real life.

Quote: Originally Posted by Rashidi

Rather than understanding "how" stats are generated, a scale that inflates players based on the position they play is more or less just gonna ignore that and hunt for an arbitrary number that fluctuates entirely based on the other four players sharing the court.

I don't quite get what you mean here. The scales I've posted here literally are "how" stats are generated. The sim engine doesn't delve into player roles or team dynamics or any of the stuff that leads to a player's assist rate in real life. All the available evidence I have says that it pretty much looks at a players pass ratings and position and throws out a number based on that. As a result, any ratings we give are essentially going to hunt for an arbitrary number, and all we can do is choose whether that arbitrary number is consistent with actual numbers we have from objective statistics, or whether it isn't. I aim for results that are descriptive rather than predictive, and it seems that you (to the limited extent that you do care about the sim engine) would rather go for predictive over descriptive. Neither approach is necessarily wrong, but simply tailored to our different priorities.

Quote: Originally Posted by Rashidi

I'm curious how this affects overall ratings, because there is already a notable discrepancy between rebounding bigs and stretch fours, not to mention PGs and the field.

Most bigs, as you know, are signifcantly overrated in rebounds, so they will take a bit of a hit. This is compensated, however, by a few factors:

-Because of the way the block scale works a lot of them will see an increase there. Blocks have a pretty high weight in the overall formula.
-Just about every non-guard had their pass ratings go way up, while many PGs were way overrated and had theirs go down (see Patrick Beverly)
-My edits to shot ratings to produce accurate results in the sim engine generally hit guards a lot harder than bigs- most guards are significantly overrated in Close and Medium shots. Great shooters IRL also stand out a lot more from poor ones now, so stretch bigs don't do too terribly.

I would say the overall results leave guards and bigs more balanced than in the stock roster. If you're curious, here are overalls for the top 5 at each position. Bear and mind that I have only touched ratings that affect the sim engine, so shots ratings (both standing and moving) for Close/Med/3pt, Layups, Pass Accuracy/Pass Vision, Steal, Block, and both rebound ratings.

Centers
87 Jordan
86 M.Gasol
86 Duncan
85 Cousins
85 Howard

Power Forwards
91 Davis
87 Griffin
87 Aldridge
85 P.Gasol
84 Nowitzki

Small Forwards
95 James
93 Durant
87 Leonard
87 Anthony
85 George

Shooting Guards
Harden seems low but drawing a million fouls is worth no rating points in 2k. His Close, Mid range, and at-rim FG% IRL is fairly poor, and stock Harden is overrated as a passer (usually puts up 8-9 APG). Jimmy Butler, another big FTA guy, suffers from this too.
89 Harden
87 Thompson
87 Wade
85 Bryant
82 Butler/DeRozan

Point Guards
CP3 seems high but his shooting stats by location IRL are amazing
93 Paul
92 Curry
90 Westbrook
88 Wall
83 Lillard/Lowry/Teague/Conley

Quote: Originally Posted by Rashidi

Overall good work. 2K can probably use this data to tweak their scales some..

A man can hope!

I've known you for a long time (remember the Tactics Ogre forum on GameFAQs about 10-15 years ago?), so I know how difficult any Rashidi praise is to come by. I'm glad you appreciate this stuff, and I'm sure you'll be interested in my findings on how the sim engine handles shot ratings and tendencies when I get around to posting Part II. For what it's worth, if I had a console I would definitely use your roster as a base (though applying my scales to prioritize accurate sim stats).

You seem to be more informed about the scales the roster makers use than I am. I do think that a truly complete guide would contained detailed information on both the sim engine stat scales and the ones used in the official rosters. Would you be interested in sharing your information on those scales, or in posting something similar to this describing them for posterity?

[Porzingod]

Quote: Originally Posted by tetoleetd

here's something you need to consider though. when you go all out to achieve sim stats in simulated games, you can very easily effect the actual game play. a lot of people get so caught up in making tweaks to achieve sim stats, that they dont realize they are completely ruining the game play in the process. id rather have a game that plays well with a few inaccurate stats, then have a game that has the most realistic stats but plays like crap.

In my view this is a false dichotomy. If anything I've found that making these tweaks to improve sim stats will improve gameplay rather than worsening it. In the stock roster, how often have you noticed, say, DeAndre Jordan completely dominating the glass? From what I've seen he doesn't really rebound noticeably better than someone like Brook Lopez, or his teammate Blake Griffin who averaged literally about half the rebounds he did last year IRL.

Likewise, how often in the stock roster have you seen a center like Marc Gasol or Cousins or Noah affect games with his passing? From what I've seen they don't seem to be noticeably better playmakers than guys like Mozgov. It's not like Gasol will put up 10 APG in played games if you use my scales. If anything he still seems to struggle to get to 4-5.

[Porzingod]

Quote: Originally Posted by tetoleetd

people adjust sliders to get realistic stats without considering the effect it also has on gameplay when a user is actually playing rather than simulating.

You seem to be misinformed on sliders. Sliders have no effect on simulated games, they only affect gameplay. If changes made to generate accurate sim stats do have bad effects on gameplay, (and I haven't seen any evidence that this is the case) that can generally be ameliorated with sliders. If a roster produces bad sim stats, there's no way to fix that whatsoever without editing the roster.