New Sim Engine-Based Ratings Calculator Available (Download attached)

Collapse

Recommended Videos

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • JTommy67
    Pro
    • Jul 2012
    • 598

    #1

    New Sim Engine-Based Ratings Calculator Available (Download attached)

    UPDATE (v 1.2)

    - Added full splits. Splits for every year can now be added. The lack of this feature was throwing contact ratings off a bit. Since the appropriate number of plate appearances and at-bats must be calculated for vsRHP and vsLHP, you have to break down walks, sacrifices, and HBP as well. You only need to enter information for vsRHP and it will automatically calculate the vs.LHP numbers.

    - Added an adjusted overall rating calculation. Player can accrue overall points well over 99, but since the game truncates the total at that point, you can't tell the differences between players who have totals higher. The adjusted overall rating lets you see this value as a percentage of the maximum value possible for that position. This keeps the rating on a scale of 0-100, but allows you to see more of an absolute value and a better spread between players.

    - Made some adjustments to the layout. Had to make everything fit, so made the font a tad smaller and moved some things around. The middle portion I plan to use for vitals when I can figure out how to export the fields properly. This will take some time, though.

    ---------------------------------------------------------------------------------

    UPDATE: I've attached a new archive. Keep in mind the instructions/analysis do not reflect the following changes I've made:

    The "Stat Input" section has been significantly changed for pitchers. Instead of reworking the statistics through the sim engine calculations, ratings are now assigned using scales that closely resemble what the devs are probably using. This allows pitcher ratings to be appropriately scaled.

    The "Ratings Input" section for pitchers remains unchanged, and still reflects sim engine output. Other than one minor change (see below), calculations for position players is the same.

    Some details on changes:

    -Stamina Calculation:

    You now have a column for IP(R) which denotes the number of innings pitched in relief. The calculator then differentiates between IP as a starter and IP as a reliever. If the position is set to "starter", it uses only starter innings in calculating stamina and ignores innings pitched as a reliever. When set to "long relief", it rates starter stamina using starter innings, reliever stamina using reliever innings, and then creates an average weighted according to the percentage of appearances as a starter vs. reliever. For all other relief pitchers, starter innings are ignored. This is in accordance with what it looks like they're doing.

    -Because of the new rating calculation method, the "staff strength" option is removed from stat input.

    -Fixed a few incorrect references that I saw.

    -Durability calculation for position players now uses a scale close to what appears to be used by the devs.

    -Fixed a number of /DIV 0 errors when certain divisor statistics are at zero. This eliminates a number of the ### when cells are not filled out.

    Note: Remember that I crafted the tables for pitcher ratings based on correlations between weighted averages for pitchers and the rating assigned to them in the default roster. They are not a product of my own subjectivity, but are derived from the actual correlations observed from the game's own rating system to real-life statistics. If you disagree with the scales, you're not really disagreeing with me, but someone else!

    Also, the tables reflect the general trendlines I observed, but I came across a number of instances where ratings for certain players were obviously boosted. All things considered, it's highly probable they're calculating ratings based on similar charts, but are then making minor adjustments to individual players based on their own subjective projections, taking into account recent trends in player performance, changes in teams and ballparks, aging, and any number of other factors.

    ----------------------------------------------------------------------------------

    While I'm not calling this a beta version, I may still do additional work on this, especially after the 2014 version comes out. For now, I'm posting so people can have a look at it.

    This is a sim-engine based calculator (requires Microsoft Excel) which attempts to reproduce ratings based on analysis of the simulation engine. It is not an attempt to pinpoint exactly how ratings are assigned by the devs.

    There are two portions for both positional players and pitchers. The top half allows a number of season statistics to be input and weighted, returning ratings. The bottom portion allows for the input of ratings and returns a stat line of what would likely be seen in the game as an average over the course of many seasons. The formulas and tables contained in the file were derived from as many as 500 to 900 seasons.

    There are some limitations. Initially I started this only for myself, but got drawn into it more and more as time went on. So it lacks the ability to grab statistics from the web, and doesn't contain a full split stats entry function (though there is still splits calculation). I'm hoping that either myself or others can work on this for the future. My knowledge of doing Visual Basic editing in Excel is about zero, and I'd like to add some command button functionality at some point, so if you're good with that, maybe we can join forces and make this a little more user-friendly.

    Instructions are attached. There is a simple .txt version and a very detailed .pdf file which also contains my thoughts, analysis, and explanations of methods.

    Please read through these before drawing too many conclusions. I have put months and months into this and while there are improvements to be made, I'm happy with how it is currently working.

    If you have some quick questions, I'll respond in this thread. If you have some constructive feedback, this would be the place for it. I think it's pretty tight as it now stands but there may be some incorrect references in it in places.
    Last edited by JTommy67; 07-02-2014, 06:44 AM.
  • the_future420
    MVP
    • Jul 2002
    • 3086

    #2
    Re: New Sim Engine-Based Ratings Calculator Available (Download attached)

    I'll be the first to say it. Holy s*** this is in-depth and hats off to you for all your hard work and analysis.


    Now to read through this again and try to help my little brain make sense of it.
    PSN ID: thefuture420
    Twitch
    Now Playing: MLB The Show 16, Fifa 16, Fallout 4

    Comment

    • chaz526
      Rookie
      • Jan 2012
      • 218

      #3
      Re: New Sim Engine-Based Ratings Calculator Available (Download attached)

      I didn't get a chance to plug in stats, but this looks solid and well done. I bet it's possible to import stats somehow if there is a way to export data from baseball-reference as a .csv file - I'd have to look into it.

      Comment

      • JTommy67
        Pro
        • Jul 2012
        • 598

        #4
        Re: New Sim Engine-Based Ratings Calculator Available (Download attached)

        Thanks to you both for the compliments.

        Going back through I've already noticed a couple of incorrect references that I've made corrections to, and I've decided to scrap my table for the games played/durability calculation. I'm going to insert the formula they are apparently using (GP/1.64 = durability rating for everyday players). Since other statistics and ratings aren't affected by this, seems a harmless change.

        Originally posted by chaz526
        I bet it's possible to import stats somehow if there is a way to export data from baseball-reference as a .csv file - I'd have to look into it.
        Yeah, there is. It's how I developed my tables for the defensive ratings and statistics. I believe right above the stat chart for a given player is .CSV link that converts the table for export.
        Last edited by JTommy67; 03-22-2014, 02:20 PM.

        Comment

        • tbbucsfan001
          Pro
          • Aug 2009
          • 642

          #5
          Re: New Sim Engine-Based Ratings Calculator Available (Download attached)

          Hey, just to let you know. Your OBP formula for the ratings input is off. Hits shouldn't be in the denominator.

          Comment

          • puppies_on_acid
            French Fried Taters, mhmm
            • Jan 2010
            • 343

            #6
            Re: New Sim Engine-Based Ratings Calculator Available (Download attached)

            This is fantastic. I think it may be time for me to make another classic roster using this. It's been a few years since I have done one. I keep intending on making another but every year I end up not having enough time.
            Streaming MLB The Show and other games on Twitch

            Comment

            • JTommy67
              Pro
              • Jul 2012
              • 598

              #7
              Re: New Sim Engine-Based Ratings Calculator Available (Download attached)

              Originally posted by tbbucsfan001
              Hey, just to let you know. Your OBP formula for the ratings input is off. Hits shouldn't be in the denominator.
              Hmm...just checked this while it is wrong, I do have hits in the numerator. I forgot to add HBP (because I added that statistic in later and forgot to go back and insert it into OBP.)

              I think it may be time for me to make another classic roster using this. It's been a few years since I have done one. I keep intending on making another but every year I end up not having enough time.
              I would advise against that at this point. I'm going to be making some additional changes. As it stands now, this utility is not ready for use in making rosters. Here is why:

              It's predicting sim engine output with an excellent degree of accuracy, but the problem is that the sim engine does not reflect the statistics from which the ratings were initially based too well for pitchers. In particular, the hits are off (strikeouts and innings pitched, too, it looks like.)

              Put another way: The devs go to player stats to generate their ratings. They use more simplistic formulas (beyond clutch, that is) than what I'm doing in the excel file, and consequently the engine doesn't take that input and return statistics with the same degree of accuracy.

              For example, let's say they have a pitcher who has a real-life weighted average of 7.50 H/9. Based on my estimations so far, they would rate that about 75. However, in the sim engine, when all other factors are considered, a H/9 rating of about 82-83 would be required to see statistics that matched the source figures. I'm consistently finding that my H/9 ratings are between 10 and 15 percent higher because of this. Their stamina ratings also appear to be too low for the sim engine.

              So even though my calculator is making good predictions, using them (for pitchers), will produce a huge jump in H/9 ratings, and consequently a roster made from it will have batting averages far below the norm (I'm guessing top averages would be about .290 - .300).

              I have two choices: First, I can rescale the hitting side so as to compensate which I most definitely don't want to do. It's working great so far and this move would just correct an imbalance with another imbalance. Instead, I'm going to rework the pitching side and use the formulas and scales they're using. That way the pitching scale stays the same and lessens the "drift" by whole roster edits, if that makes sense. I'm working on this as we speak.

              Down the road I'm hoping that (if they continue using a similar scale) we may develop a utility that takes average H/9, K/9, and BB/9 ratings from an entire roster of pitchers and based on those figures, adjusts the tables for contact, discipline, and vision. Then historical roster makers could make a pitcher set and then the hitters they face would be scaled properly to produce correct statistical ranges. It's certainly within the realm of possibility, though would be a lot of work. If they had a global ratings editor function in the game, it would save a HUGE amount of time. lol

              This weekend I spent some time looking at a roster I made that has every single attribute set to zero for every player (except for 99 durability.). I then added in an all-99 player on each team. Definitely learning more about how these ratings interact. Very interesting...!
              Last edited by JTommy67; 03-23-2014, 06:13 PM.

              Comment

              • Ghost Of The Year
                Turn Left. Repeat.
                • Mar 2014
                • 6387

                #8
                Re: New Sim Engine-Based Ratings Calculator Available (Download attached)

                Originally posted by JTommy67

                For example, let's say they have a pitcher who has a real-life weighted average of 7.50 H/9. Based on my estimations so far, they would rate that about 75. However, in the sim engine, when all other factors are considered, a H/9 rating of about 82-83 would be required to see statistics that matched the source figures. I'm consistently finding that my H/9 ratings are between 10 and 15 percent higher because of this. Their stamina ratings also appear to be too low for the sim engine.
                A 7.5 h/9 should probably have a rating of at least 90, less than 10 pitchers hit that number in 2013 IRL.
                Will tweaking the Human Contact Slider & CPU Contact Slider help any?
                T-BONE.

                Talking about things nobody cares.

                Screw Discord. Make OS Great Again.

                Comment

                • JTommy67
                  Pro
                  • Jul 2012
                  • 598

                  #9
                  Re: New Sim Engine-Based Ratings Calculator Available (Download attached)

                  Well, we're talking about that H/9 rating and its effect in the sim engine, not gameplay. If the game plays fine for you with ratings as they are, then there would be no need to make changes outside of your normal adjustments and tweaks.

                  They have only twenty pitchers rated 75 or higher, so they're on target there (remember they take into account the three years prior to the rated season). What I'm saying is that, in the sim engine, a 75 H/9 rating doesn't seem to (re)-generate the statistics it's derived from in the first place...on average. (That last part is important. You'll have to run a number of seasons and tabulate to see the difference, but that's what I'm observing.)

                  As sensitive as the sim engine is, it's possible I'm seeing the effects of some tweaking to get things right. They have to think both about gameplay and the sim engine, and that's got to be a formidable task when it comes to something like this.
                  Last edited by JTommy67; 03-23-2014, 07:56 PM.

                  Comment

                  • tbbucsfan001
                    Pro
                    • Aug 2009
                    • 642

                    #10
                    Re: New Sim Engine-Based Ratings Calculator Available (Download attached)

                    Hey man if you could head over to this thread down here we have some questions and comments about how this tool and our project.

                    Thanks a bunch

                    http://www.operationsports.com/forum...ial-setup.html

                    Comment

                    • tessl
                      All Star
                      • Apr 2007
                      • 5685

                      #11
                      Re: New Sim Engine-Based Ratings Calculator Available (Download attached)

                      What program do you use to download your program?

                      If I understand this correctly your program allows a person to plug in stats and get attribute ratings for players in mlbts, is that correct?

                      Comment

                      • JTommy67
                        Pro
                        • Jul 2012
                        • 598

                        #12
                        Re: New Sim Engine-Based Ratings Calculator Available (Download attached)

                        Originally posted by tessl
                        What program do you use to download your program?

                        If I understand this correctly your program allows a person to plug in stats and get attribute ratings for players in mlbts, is that correct?
                        The file itself is a .rar file that contains and excel file and some instruction files. You'll need WinRAR to unpack it.

                        Read the instructions carefully. The cells which contain formulas are not locked, so if you overwrite them you'll screw things up.

                        But you're correct, it allows you to input real life statistics and get ratings in return. But make sure you read the discussion in this thread because there are problems on the pitching side which I'm currently working on.

                        Comment

                        • tessl
                          All Star
                          • Apr 2007
                          • 5685

                          #13
                          Re: New Sim Engine-Based Ratings Calculator Available (Download attached)

                          Originally posted by JTommy67
                          The file itself is a .rar file that contains and excel file and some instruction files. You'll need WinRAR to unpack it.

                          Read the instructions carefully. The cells which contain formulas are not locked, so if you overwrite them you'll screw things up.

                          But you're correct, it allows you to input real life statistics and get ratings in return. But make sure you read the discussion in this thread because there are problems on the pitching side which I'm currently working on.
                          Thanks for the reply. Have you looked at the thread I started about changes to plate discipline which Brian, lead AI programmer replied to? There has been some adjustment to various categories with a goal of producing hitters to chase more bad pitches. That may impact what you are attempting to do.
                          http://www.operationsports.com/forum...e-written.html


                          I'm guessing you are working with mlbts 13.
                          Last edited by tessl; 03-25-2014, 12:59 PM.

                          Comment

                          • JTommy67
                            Pro
                            • Jul 2012
                            • 598

                            #14
                            Re: New Sim Engine-Based Ratings Calculator Available (Download attached)

                            Yeah I did see that thread, but unless the changes impact the sim engine they won't really have an effect on what I'm doing. Thanks for the heads up, though.

                            Comment

                            • JTommy67
                              Pro
                              • Jul 2012
                              • 598

                              #15
                              Re: New Sim Engine-Based Ratings Calculator Available (Download attached)

                              See the top for an explanation of changes.

                              Comment

                              Working...