MLB 15 Sim Engine Testing

Collapse

Recommended Videos

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • WaitTilNextYear
    Go Cubs Go
    • Mar 2013
    • 16830

    #1

    MLB 15 Sim Engine Testing

    As some of you know, I am a big proponent of using results from in game sims to inform us as to what player ratings should be. Whereas the stats from played games are heavily dependent on sliders and user skill, the sim engine relies almost entirely on ratings to produce stats for players/teams around the league. None of us like it when you play through an epic franchise only to see that, at the end of the season, there are no guys that hit .300, there are 5 pitchers with over 300 Ks and several starters with ERAs under 2. We rely heavily on this type of testing for the Hybrid roster ratings.

    Here is some testing I did recently in MLB '14.

    http://www.operationsports.com/forum...zips-osfm.html

    I have been running sims since I got MLB '15 and it appears that a few small tweaks were made to the sim engine. Preliminary data tells me that SCEA ratcheted up the strikeouts. Another way to say this is that an 85 K/9 rating in MLB '15 will result in more Ks than an 85 K/9 rating in MLB '14. It looks like an 85 K/9 rating in MLB '14 is roughly equivalent to an 81 K/9 rating in MLB '15.

    There also appears to be a slight difference in homers, with homers being a bit more abundant in the new game. This has been observed using my own ratings as well as out-of-the-box SCEA ratings.

    Some of the changes are good things. It looks like very high BB rates for batters and high K rates for RPs are a bit more obtainable with this new engine although we still can't quite do Kimbrel and Chapman justice with a mere 99 K/9 rating.

    Anyhow, I will be posting some data in this thread that highlights how ratings for specific players interact with the sim engine to produce stats. It won't be a full-on derivation or anything, but the data will be used as a basis for the ratings that go into the Hybrid roster. During all of my tests, I will be using default sliders and with injuries/trades off (cpu roster control on). Feel free to post your own data in this thread as well.
    Last edited by WaitTilNextYear; 04-04-2015, 05:49 PM.
    Chicago Cubs | Chicago Bulls | Green Bay Packers | Michigan Wolverines
  • JTommy67
    Pro
    • Jul 2012
    • 598

    #2
    Re: MLB 15 Sim Engine Testing

    Excellent work, and kudos for the efforts you're making.

    Strikeout rates continue to go up, and once again, last year broke the record again: 7.7 K/game compared to 7.55 in 2014.

    It's fairly evident to me that statistical calculations are the predominate factor for batter and pitcher ratings. Thus we should expect slightly higher K/9 ratings and slightly lower vision ratings in the aggregate, which should produce more strikeouts.

    Your observations about the 99 ceiling are spot on, IMO. Players with a 99 K/9 can only improve if vision ratings drop across the board. It'd be nice if SCEA could rescale the effects of ratings to accommodate greater discrepancies and produce higher statistics.

    More home runs is an oddity, since they are down from .96/game to .86/game. If anything we should expect to see them drop slightly in the game. That is, if they were accurate last year.

    Comment

    • johntait1
      Rookie
      • Feb 2015
      • 116

      #3
      Re: MLB 15 Sim Engine Testing

      Thanks for the hard work of testing everything. Glad to see that K/9 ratings have been tweaked slightly, but it would be nice if a 99 rating would work for Chapman. I feel like a greater range of possible stats would be really nice, but oh well.

      Comment

      • WaitTilNextYear
        Go Cubs Go
        • Mar 2013
        • 16830

        #4
        Re: MLB 15 Sim Engine Testing

        Here's an early look at some data.

        The methodology

        I have already done a lot of work with the '14 sim engine and the stats-->ratings conversions have come a long way. I've posted more detail about this in another thread.

        So, I took my ratings (carried over from the Hybrid roster and taking 2015 ZiPS projections into account) and plugged them into '15. I went through 4 or 5 iterations of simming/comparing sim stats to irl last year stats/adjusting to hone the ratings to a pretty fine blade.

        I then inserted the updated ratings (as found out from this testing in the '15 sim engine) for all MLB players and simmed one year 5 different times to collect data. I collected a mixture of data for teams, players, and even awards. I did not thoroughly look at every player's stats (though I could because everything is saved), so I am reporting on a handful of key players to help us draw conclusions about whether these ratings (which will feature in the Hybrid roster) are good enough or not. I tried to monitor different types of players (contact hitters, power hitters, SPs, CPs).

        The data

        Individual Player Stats - Average Data

        Spoiler



        Team Stats - Average Data

        Spoiler



        Yearly Awards - Sim 1

        Spoiler



        Yearly Awards - Sim 2

        Spoiler



        Yearly Awards - Sim 3

        Spoiler



        Yearly Awards - Sim 4

        Spoiler



        Yearly Awards - Sim 5

        Spoiler



        At some point, I will post a spreadsheet with my ratings in it, but for now just be assured that the forthcoming Hybrid roster will be amazing.
        Last edited by WaitTilNextYear; 04-05-2015, 10:44 PM.
        Chicago Cubs | Chicago Bulls | Green Bay Packers | Michigan Wolverines

        Comment

        • HopiDesertPriest
          Banned
          • Nov 2013
          • 597

          #5
          Re: MLB 15 Sim Engine Testing

          Sliders play a big role. I got a set in the slider vault; PSN: SaintPadrePio.

          Cpu Starter Stamina -- 3
          Cpu Reliever Stamina -- 2
          Manager Hook -- 6
          Injury -- 8


          With Scea ratings:


          Cpu Starter Stamina -- 2
          Cpu Reliever Stamina -- 3
          Manager Hook -- 6
          Injury -- 8

          Comment

          • WaitTilNextYear
            Go Cubs Go
            • Mar 2013
            • 16830

            #6
            Re: MLB 15 Sim Engine Testing

            Originally posted by ghostofsparta15
            What sliders are you using to get these results?

            I've been really struggling to get results like these, appreciate all the work you do.
            Originally posted by WaitTilNextYear
            During all of my tests, I will be using default sliders and with injuries/trades off (cpu roster control on).
            Originally posted by HopiDesertPriest
            Sliders play a big role. I got a set in the slider vault; PSN: SaintPadrePio.

            Cpu Starter Stamina -- 3
            Cpu Reliever Stamina -- 2
            Manager Hook -- 6
            Injury -- 8


            With Scea ratings:


            Cpu Starter Stamina -- 2
            Cpu Reliever Stamina -- 3
            Manager Hook -- 6
            Injury -- 8
            Let's clarify this a bit. Some sliders (injury and SP/RP stamina etc..) play a role in how much playing time guys get. But sliders don't have an impact on their sim stats in the playing time they do get. In other words, changing sliders does not influence how effectively the players play. What's more important to look at is the rate stats (/9, HR% etc..) which sliders have zero effect on.
            Chicago Cubs | Chicago Bulls | Green Bay Packers | Michigan Wolverines

            Comment

            • HopiDesertPriest
              Banned
              • Nov 2013
              • 597

              #7
              Re: MLB 15 Sim Engine Testing

              BEWARE:

              BA RISP cpu vs cpu Sim Stats BROKEN AGAIN !!!!


              terrible .... most teams hitting .225 AT BEST .... sim a season and check ... terrible.

              Comment

              Working...