Porzingod's In-Depth Sim Engine Analysis: Part I

Collapse

Recommended Videos

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • Porzingod
    Rookie
    • Jul 2015
    • 17

    #1

    Porzingod's In-Depth Sim Engine Analysis: Part I

    This guide will explain in great detail how NBA 2k15's comes up with stats in simulated games in the MyGM and MyLeague modes. All information herein was gleaned from empirical research over many simulated seasons, using the PC version of 2k15 with the latest updates and official 2k roster. It will explain how the engine uses player ratings and tendencies to come up with their stats, and where possible, how you can modify these values so that the stats produced are realistic/accurate to actual NBA stats.

    Whenever I get a new 2k game the first thing I do is sim through a season a few times to see how close players' stats are to real-life stats. They are usually off pretty significantly in most areas, and 2k15 is unfortunately no exception. While the rosters seem to be greatly improved this year in terms of completeness, the player ratings in the areas that produce stats (i.e. rebounds, assists) either do not follow a consistent scale, or do not follow the same scale that the simulation engine uses.

    If you sim a season with the official 2k roster the following issues should jump out immediately:

    -Rebound stats are simply way off. Steph Curry averages around 7-8 RPG (was under 5 in real life) while the NBA's leading rebounder DeAndre Jordan averages under 12 (averaged 15 in real life). In general there is far too little separation between truly elite rebounders like Jordan and bigs like Brook Lopez or Marc Gasol who are just decent rebounders. Guards also generally get too many rebounds. This is mostly fixable with more accurate ratings.

    -Offensive rebounds too low for the best rebounders, too high for the worst. Almost no player will get less than 1 offensive rebound per 36 minutes, while in the NBA this is extremely common for guards. By the same token, the top offensive rebounders don't get close to what they should. Andre Drummond pulled down 6 offensive rebounds per 36 minutes last season. It's rare for any player in 2k to be above 4, even with maxed ratings. Likewise, defensive rebounds are slightly too abundant, but both of these flaws kind of compensate for each other so decently accurate total rebounding stats are achievable.

    -Assists are fairly accurate for guards, and increasingly inaccurate for the bigger positions. PF's and C's assist numbers will be too low, usually by 50% or so. This is fixable with ratings.

    -Three point percentage is too low across the entire league. This is mainly due to the global -8 point adjustment in shooting ratings that took place in one of the official roster updates. The simulation engine was not changed to reflect this, so pretty much every team shoots around 2-4% worse than they ought to. Also fixable with ratings.

    -Blocked shots are way too low for the best shot blockers. It seems completely impossible, generally, for anyone to average over 2.5 bpg or so in reasonable minutes. Often times a guy with maxed out block ratings will come out with like 1.8 bpg. In real life the league leader usually has 3 bpg or more. Unfortunately this cannot be fixed in any way. This issue has been present since at least NBA 2k12, so I wouldn't bet on it being fixed in next year's game.

    For part I of this guide I will cover assists, rebounds, blocks, and steals- the easier stuff. I also have comprehensive data for the way the engine handles shooting stats/tendencies, but that stuff gets much more complicated, so i'll post those later if enough people are interested. Now, on to the individual stats!

    ASSISTS:

    Assists use only two ratings: Passing Accuracy and Passing Vision. The player development menu says that Passing IQ affects assists, and you would think it does, but extensive testing shows it to have no effect. For convenience's sake you can average the two and call that a player's PASS rating. For point guards, ratings are fairly accurate- 99 PASS corresponds to around the top-level assist numbers they'll usually get in real life- around 10 assists per 36 minutes. The best stat to use to derive passing ratings is AST% from basketball-reference.com, as it is adjusted for pace. In general, for point guards:

    PASS = 1.6 * AST% + 25

    Steph Curry last season had 38.6 AST%, so to get accurate assist numbers (around 8.5 per 36 minutes) his Passing Accuracy and Passing Vision should average out to around 87.

    Most PG's ratings go roughly by this scale in the official roster (Curry himself is a bit overrated here, probably so they could pump his overall up), and so they are reasonably accurate.

    Problems arise with the other positions. 2k's official ratings rate all other players by the same scale as point guards. Blake Griffin for example has 63 PASS in the official roster, which by the PG scale would give him about 5 assists per 36 minutes, an accurate number for him. However, you'll notice that with the official roster he'll usually be under 2.5 APG. This is because the stat simulation engine uses different assist scales for different positions. While 99 PASS corresponds to about 10 assists per 36 minutes for PG's, it produces roughly the following numbers for different positions:

    SG: 7 assists per 36 minutes, or 37.5 AST%
    SF: 6 assists per 36 minutes, or 30 AST%
    PF: 5 assists per 36 minutes, or 25 AST%
    C: 4 assists per 35 minutes, or 20 AST%

    Now, these scales actually do make a bit of sense. The best passing PF's and C's in history historically have put up around those assist numbers, though the best passing SF's have been a bit higher. The problem is that the roster maker doesn't seem to have been aware of this at all, and so everyone is rated by the PG scale. The effect of this is that great passing bigs get nowhere near the assist numbers they ought to. If you want to get accurate assist numbers for non PG's, the formulas to use are as follows:

    Shooting Guards:

    PASS = 2 * AST% + 25

    Small Forwards:

    PASS = 2.5 * AST% + 25

    Power Forwards:

    PASS = 3 * AST% + 25

    Centers:

    PASS = 3.75 * AST% + 25

    It may seem a bit weird to have, say, Blake Griffin with 99 PASS, but it makes sense when you consider he is one of the best passing PF's of all time.


    REBOUNDS:

    Here, the issue is simply that the ratings don't follow a consistent scale. Some players are rated accurately, but most starting bigs are rated too high, which makes truly great rebounders like Jordan, Drummond, or Cousins fail to stand out. Guards will tend to get significantly more rebounds than they ought to as well. The only ratings that matter here are OREB and DREB, Box Out has no effect on simulated stats.

    It seems the core issue here is that the minimum ratings do not correspond to 0 rebounds in the simulation engine, but 2k seems to rate players as if it does. The best stats to use for rebounds are ORB% and DRB% from basketball-reference.com, as they are adjusted for both pace and minutes. As far as I can tell, a rating of 25 will produce an ORB% of about 1.0 and a DRB% of about 6.0. After extensive testing I've determined the following scales will produce quite accurate rebounding numbers:

    OREB = (ORB% - 1) * 4.75 + 25

    DREB = (DRB% - 6) * 2.75 + 25

    Please note that you will only get accurate rebound stats for individuals players if you adjust their whole team, as there are only so many rebounds to go around.


    STEALS:

    These aren't all that bad in the official roster, just inconsistent in a lot of cases. One interesting thing to note is that the bottom 25 points of the ratings scale aren't really used here. That is, a 25 rating will produce the same amount of steals as 50. The only thing that matters here is the STEAL rating, Pass Perception does nothing. Best stat to use is STL% from basketball-reference.com.

    STEAL = 16.25 * (STL% - 0.5) + 50


    BLOCKS

    These are fine up until you reach the top level shot blockers, who will have way too few blocks. The league leader in blocks in the NBA is usually around 7-7.5 in BLK% - last season it was Gobert with 7%. The problem is that maxed BLOCK rating produces only about 5.0 BLK%, so the best shot blockers will all underperform. The only rating that matters here is BLOCK, Shot Contest and the various tendencies that sound relevant all do nothing.

    BLOCK = BLK% * 15 + 25

    So the top 8 shot blockers in the league last year should all be rated 99, and unfortunately the best ones are just going to significantly underperform. Here's hoping 2k16 doesn't have this same issue for the 5th year in a row.


    SOME EXAMPLES OF MY SCALES IN ACTION:

    To give you an idea of how much these ideas can improve the verisimilitude of stats, here's a comparison of the stats covered in this guide in the official 2k roster vs. a roster using my scales. All stats are on a per-36 minute basis.



    ASSISTS

    PLAYER REAL NBA Official 2k Porzingod's
    S.Curry 8.5 8.9 8.6
    B.Griffin 5.4 2.2 5.7
    M.Gasol 4.1 2.2 4.6
    D.Jordan 0.8 0.3 0.7
    D.Cousins 3.8 2.3 3.7

    REBOUNDS

    PLAYER REAL NBA Official 2k Porzingod's
    S.Curry 4.7 6.5 4.5
    B.Griffin 7.8 9.1 8.1
    M.Gasol 8.4 10.7 9.8
    D.Jordan 15.7 12.7 14.4
    D.Cousins 13.4 10.5 12.1



    Not perfect by any means, but I'd say it's a substantial improvement!

    Now this will all probably be just a fart in the wind, but there are any other stat nerds out there who care about these issues, I urge you to make the community's voice heard. Please like this or upvote it or whatever you can do here, and maybe tweet this at some of the folks who could make a difference if you want to see simulated stats be a priority in future versions!
  • BluFu
    MVP
    • May 2012
    • 3596

    #2
    Re: Porzingod's In-Depth Sim Engine Analysis: Part I

    Originally posted by Porzingod
    Please note that you will only get accurate rebound stats for individuals players if you adjust their whole team, as there are only so many rebounds to go around.
    This is also true for assists per game as well. In order for the higher apg players to get their numbers, the lower apg guys need to be tweaked accordingly.

    Great post.

    Comment

    • LorenzoDC
      MVP
      • Sep 2010
      • 1857

      #3
      Re: Porzingod's In-Depth Sim Engine Analysis: Part I

      Great job, lots of work you've done.

      I would be interested in your tendencies analysis as well, if you ever get to writing it up.

      You get the Porzingod champion gif:

      Comment

      • Phreezy P
        MVP
        • Jan 2010
        • 3219

        #4
        Re: Porzingod's In-Depth Sim Engine Analysis: Part I

        Originally posted by LorenzoDC
        Great job, lots of work you've done.

        I would be interested in your tendencies analysis as well, if you ever get to writing it up.

        You get the Porzingod champion gif:

        <snip>
        Spoiler
        Minnesota Timberwolves and Toronto Raptors

        CHECK OUT MY YOUTUBE CHANNEL

        Comment

        • Carboncoms
          Rookie
          • Jan 2015
          • 72

          #5
          Re: Porzingod's In-Depth Sim Engine Analysis: Part I

          Just wanted to let you know that this is awesome research OP.

          Comment

          • dwayne12345
            MVP
            • Dec 2010
            • 1407

            #6
            Re: Porzingod's In-Depth Sim Engine Analysis: Part I

            With your block tests did you also try adjusting coaching strategies to see if that increases averages?

            For example defensive strategy (physical), rotations (making the big man play 40 minutes a night), etc

            Comment

            • Cleve07103
              Rookie
              • Sep 2011
              • 206

              #7
              Re: Porzingod's In-Depth Sim Engine Analysis: Part I

              This is awesome. I would love to see the tendancies. I asked about this previously but no one responded.
              'Hard work beats talent when talent doesn't work hard&quot; - Kevin Durant.......................... Los Angeles Clippers: San Francisco 49ers: New York Yankees: New Jersey Devils: New Jersey Nets (until they moved to Brooklyn): Temple University:

              Comment

              • Hassan Darkside
                We Here
                • Sep 2003
                • 7561

                #8
                Re: Porzingod's In-Depth Sim Engine Analysis: Part I

                OP, I just want you to know that I haven't read your post yet but I liked it because of your username.
                [NYK|DAL|VT]
                A true MC, y'all doing them regular degular dance songs
                You losin' your teeth, moving like using Kevin Durant comb
                Royce da 5'9"


                Originally posted by DCAllAmerican
                How many brothers fell victim to the skeet.........

                Comment

                • jfsolo
                  Live Action, please?
                  • May 2003
                  • 12965

                  #9
                  Re: Porzingod's In-Depth Sim Engine Analysis: Part I

                  I hope the 2K devs have seen this, or have already came to this same conclusion through their own research, because this analysis is on point.
                  Jordan Mychal Lemos
                  @crypticjordan

                  Do this today: Instead of $%*#!@& on a game you're not going to play or movie you're not going to watch, say something good about a piece of media you're excited about.

                  Do the same thing tomorrow. And the next. Now do it forever.

                  Comment

                  • Real2KInsider
                    MVP
                    • Dec 2003
                    • 4660

                    #10
                    Re: Porzingod's In-Depth Sim Engine Analysis: Part I

                    Originally posted by Porzingod
                    Andre Drummond pulled down 6 offensive rebounds per 36 minutes last season.
                    Drummond is a scale breaker. His production is worth a rating of 116 which is obviously not possible in 2K.

                    Dennis Rodman is the career leader in ORB% with 17.2% and Drummond is at 17.4% through three seasons.

                    Drummond is an exception and not the rule. 2K will not be able to replicate his production without breaking their system.

                    It's rare for any player in 2k to be above 4, even with maxed ratings. Likewise, defensive rebounds are slightly too abundant, but both of these flaws kind of compensate for each other so decently accurate total rebounding stats are achievable.
                    This season was actually the worst offensive rebounding season in history, with only 25% of rebounds going to the offense (though one could also say defensive rebounding has never been valued higher). 2K's sim engine hasn't changed much but they're actually much closer to reality than they were 5 years ago.

                    -Assists are fairly accurate for guards, and increasingly inaccurate for the bigger positions. PF's and C's assist numbers will be too low, usually by 50% or so. This is fixable with ratings.
                    PGs are operating on a different (elevated) scale than the rest of the positions. The numbers are much better w/ everyone on an equal playing field. Every other position is like 20 points underrated. Never makes sense to me when I see a SG with a 32 pass rating. There are plenty.


                    -Three point percentage is too low across the entire league. This is mainly due to the global -8 point adjustment in shooting ratings that took place in one of the official roster updates. The simulation engine was not changed to reflect this, so pretty much every team shoots around 2-4% worse than they ought to. Also fixable with ratings.
                    Yes, and this also might lend itself towards the ORB/DRB skew.

                    -Blocked shots are way too low for the best shot blockers. It seems completely impossible, generally, for anyone to average over 2.5 bpg or so in reasonable minutes. Often times a guy with maxed out block ratings will come out with like 1.8 bpg. In real life the league leader usually has 3 bpg or more. Unfortunately this cannot be fixed in any way. This issue has been present since at least NBA 2k12, so I wouldn't bet on it being fixed in next year's game.
                    The culprit is likely too many jumpers vs drives in the sim engine.

                    Westbrook
                    IRL: 654 FTA in 67 games (9.8 per)
                    2K: 459 FTA in 82 games (5.6 per)

                    The stats scale fine but the opportunities aren't there for bigs to record them.


                    ASSISTS:

                    Assists use only two ratings: Passing Accuracy and Passing Vision. The player development menu says that Passing IQ affects assists, and you would think it does, but extensive testing shows it to have no effect.
                    I did not test but figured as much. I doubt Pass Perception and Shot Contest have any impact on the sim engine either.

                    For convenience's sake you can average the two and call that a player's PASS rating. For point guards, ratings are fairly accurate- 99 PASS corresponds to around the top-level assist numbers they'll usually get in real life- around 10 assists per 36 minutes. The best stat to use to derive passing ratings is AST% from basketball-reference.com, as it is adjusted for pace. In general, for point guards:

                    PASS = 1.6 * AST% + 25
                    It should be noted this is not the official formula for Vision or Accuracy, though it's probably close enough.


                    Steph Curry last season had 38.6 AST%, so to get accurate assist numbers (around 8.5 per 36 minutes) his Passing Accuracy and Passing Vision should average out to around 87.
                    FWIW I have Curry 89/97 V/A (he averaged 8.5 per game and 8.8 per 36 in a test)

                    Most PG's ratings go roughly by this scale in the official roster (Curry himself is a bit overrated here, probably so they could pump his overall up), and so they are reasonably accurate.

                    Problems arise with the other positions. 2k's official ratings rate all other players by the same scale as point guards.
                    This is actually not true. 2K absolutely uses a different scale for PGs than it does other positions. I think you might be looking at it from an incorrect angle.

                    AST% and Pass Rating
                    35.2* Isaiah Thomas (PG) = 80
                    30.0 Evan Turner (SF) = 70
                    16.3 Marcus Smart (PG) = 58
                    14.2 Jared Sullinger (PF) = 60
                    12.2 Kelly Olynyk (C) = 42
                    10.8 Tyler Zeller (C) = 33
                    8.9 Avery Bradley (SG) = 42
                    8.6 Brandon Bass (PF) = 44
                    8.6 Jae Crowder (SF) = 40

                    *BOS numbers; 27.2 for season due to 23.6 in PHX for Thomas

                    I noticed a couple years ago that very few players have pass ratings between 50-60 relative to any other range and realized there were two separate scales. 2K has PGs on a scale where it is impossible for them to register a pass rating lower than 40 (Literally an AST% of 0 would = a rating of 40). Continuing along...

                    Blake Griffin for example has 63 PASS in the official roster, which by the PG scale would give him about 5 assists per 36 minutes, an accurate number for him
                    26.2 Blake Griffin = 62

                    He is clearly not rated to scale (In fact, his rating appears to be FOUR YEARS out of date).

                    FWIW I have Griffin at 73/77 A/V. He averaged 3.0 APG and 3.5 per36 in a test... though the Clippers also added a number of good passers this off-season - Lance Stephenson, Josh Smith, and Paul Pierce, so Griffin's numbers will almost assuredly drop.

                    SG: 7 assists per 36 minutes, or 37.5 AST%
                    SF: 6 assists per 36 minutes, or 30 AST%
                    PF: 5 assists per 36 minutes, or 25 AST%
                    C: 4 assists per 35 minutes, or 20 AST%

                    Now, these scales actually do make a bit of sense. The best passing PF's and C's in history historically have put up around those assist numbers, though the best passing SF's have been a bit higher. The problem is that the roster maker doesn't seem to have been aware of this at all, and so everyone is rated by the PG scale.
                    The reason the positions scale this way STATISTICALLY (SEPARATE from the rating) is because for years 2K had a problem getting PGs to post accurate assist totals. Steve Nash, Chris Paul, etc would never come close to their league-leading numbers. This was only recently fixed, and they clearly have a ways to go to make sure the rest of the league balances out.

                    The effect of this is that great passing bigs get nowhere near the assist numbers they ought to. If you want to get accurate assist numbers for non PG's, the formulas to use are as follows
                    I like the idea of the scales in theory, but role in the offense is what dictates assist rate, not position. Griffin is obviously an exception and not a rule as far as big-men passing goes. Using the Celtics as an example again

                    IRL AST per36
                    Thomas: 7.5
                    Turner: 7.2
                    Smart: 4.1

                    2K test AST per36
                    Thomas: 7.7
                    Smart: 6.0
                    Turner: 5.4

                    I personally don't care much about sim stats and it's because of situations like this. Smart's assist rate IRL is kept in check because Turner is the primary ball handler and effectively the PG. Thomas comes off the bench IRL.

                    In 2K the sim engine will start Thomas due to his OVR rating, which would completely change Turner's statistical output IRL.

                    Rather than understanding "how" stats are generated, a scale that inflates players based on the position they play is more or less just gonna ignore that and hunt for an arbitrary number that fluctuates entirely based on the other four players sharing the court.


                    REBOUNDS:
                    Here, the issue is simply that the ratings don't follow a consistent scale. Some players are rated accurately, but most starting bigs are rated too high, which makes truly great rebounders like Jordan, Drummond, or Cousins fail to stand out. Guards will tend to get significantly more rebounds than they ought to as well. The only ratings that matter here are OREB and DREB, Box Out has no effect on simulated stats.

                    It seems the core issue here is that the minimum ratings do not correspond to 0 rebounds in the simulation engine, but 2k seems to rate players as if it does. The best stats to use for rebounds are ORB% and DRB% from basketball-reference.com, as they are adjusted for both pace and minutes. As far as I can tell, a rating of 25 will produce an ORB% of about 1.0 and a DRB% of about 6.0. After extensive testing I've determined the following scales will produce quite accurate rebounding numbers:

                    OREB = (ORB% - 1) * 4.75 + 25

                    DREB = (DRB% - 6) * 2.75 + 25

                    Please note that you will only get accurate rebound stats for individuals players if you adjust their whole team, as there are only so many rebounds to go around.
                    ORB is generally overrated across the board.

                    Melo
                    ORB 5.7% = 54 (1.3 per36 in test, 1.8 per36 IRL)
                    DRB 16.0% = 66 (6.2 per36 in test, 4.8 per36 IRL)

                    By your scale Melo would have 47/52 (-7 and -14), which would probably generate an improvement. Though it is worth noting that teammates are a factor and Melo playing next to a pathetic defensive rebounder (Robin Lopez is even worse than his brother) may also have inflated his DRB total.

                    STEALS:
                    These aren't all that bad in the official roster, just inconsistent in a lot of cases. One interesting thing to note is that the bottom 25 points of the ratings scale aren't really used here. That is, a 25 rating will produce the same amount of steals as 50. The only thing that matters here is the STEAL rating, Pass Perception does nothing. Best stat to use is STL% from basketball-reference.com.
                    Yep, which is why the bottom of the scale doesn't get used. Except with rookies, who predominantly make up the sub 50 ratings. I speculated that someone besides Mike may have done those ratings, or more likely, the college stat conversion scale they have isn't creating an accurate replication.

                    Now this will all probably be just a fart in the wind, but there are any other stat nerds out there who care about these issues, I urge you to make the community's voice heard. Please like this or upvote it or whatever you can do here, and maybe tweet this at some of the folks who could make a difference if you want to see simulated stats be a priority in future versions!
                    Overall good work. 2K can probably use this data to tweak their scales some.
                    I'm curious how this affects overall ratings, because there is already a notable discrepancy between rebounding bigs and stretch fours, not to mention PGs and the field.
                    Last edited by Real2KInsider; 07-29-2015, 11:33 PM.
                    NBA 2K25 Roster: Real 2K Rosters - Modern Era
                    PSN: Real2kinsider
                    http://patreon.com/real2krosters
                    http://twitter.com/real2kinsider
                    http://youtube.com/real2krosters

                    Comment

                    • tetoleetd
                      MVP
                      • Jul 2011
                      • 1151

                      #11
                      Re: Porzingod's In-Depth Sim Engine Analysis: Part I

                      here's something you need to consider though.

                      when you go all out to achieve sim stats in simulated games, you can very easily effect the actual game play.

                      a lot of people get so caught up in making tweaks to achieve sim stats, that they dont realize they are completely ruining the game play in the process.

                      id rather have a game that plays well with a few inaccurate stats, then have a game that has the most realistic stats but plays like crap.

                      Comment

                      • YungGun
                        Pro
                        • Aug 2011
                        • 536

                        #12
                        Re: Porzingod's In-Depth Sim Engine Analysis: Part I

                        Originally posted by tetoleetd
                        here's something you need to consider though.

                        when you go all out to achieve sim stats in simulated games, you can very easily effect the actual game play.

                        a lot of people get so caught up in making tweaks to achieve sim stats, that they dont realize they are completely ruining the game play in the process.

                        id rather have a game that plays well with a few inaccurate stats, then have a game that has the most realistic stats but plays like crap.
                        Isn't that what sliders are for?

                        Comment

                        • Porzingod
                          Rookie
                          • Jul 2015
                          • 17

                          #13
                          Re: Porzingod's In-Depth Sim Engine Analysis: Part I

                          Originally posted by BluFu
                          This is also true for assists per game as well. In order for the higher apg players to get their numbers, the lower apg guys need to be tweaked accordingly.

                          Great post.
                          While I did indeed note this to be true in previous versions, it appears that in 2k15 it is no longer the case. If you go by my scales Chris Paul and Blake Griffin both deserve around max PASS ratings, and they have no problem getting the amount of assists they ought to.

                          A team with all 99/99 rated passers will all average the full amount of assists predicted in the guide.

                          Comment

                          • Porzingod
                            Rookie
                            • Jul 2015
                            • 17

                            #14
                            Re: Porzingod's In-Depth Sim Engine Analysis: Part I

                            Originally posted by dwayne12345
                            With your block tests did you also try adjusting coaching strategies to see if that increases averages?

                            For example defensive strategy (physical), rotations (making the big man play 40 minutes a night), etc
                            Yes, believe me I have tried everything. Making the big man play 40 minutes is no solution, I'm interested in getting the right rate of stats per minute instead of totals. Jacking up minutes would just make other stats too high unless I nerfed those ratings to compensate.

                            Comment

                            • tetoleetd
                              MVP
                              • Jul 2011
                              • 1151

                              #15
                              Re: Porzingod's In-Depth Sim Engine Analysis: Part I

                              Originally posted by YungGun
                              Isn't that what sliders are for?
                              people adjust sliders to get realistic stats without considering the effect it also has on gameplay when a user is actually playing rather than simulating.

                              Comment

                              Working...