Groundhog learns Python - Road to a Basketball Text Sim

[muns]

Ya, still looking good!

[Groundhog]

Thanks guys!

[CraigSca]

LOVE!

Incredible job!

[Groundhog]

Originally Posted by CraigSca

LOVE!

Incredible job!

Thanks, appreciate it!

And now for a boring post that might be of interest to anyone else that tries something like this...

When I started this project a little over a year ago, I felt the hardest thing would be coming up with a way to accurately simulate a game of basketball. Looking at it now, that was probably the easiest part. After 25 or so years of playing and watching the sport, I know how basketball works. Once I found out how Python syntax 'works' and learnt how to control the flow of my game loop it was just really a matter of filling in the gaps and coming up with ways to calculate different results.

Since I started working on a GUI it's been a whole new ballpark, and I've had to dig deep into documentation and other people's code to come up with solutions to a lot of difficult problems that pop up, usually relating to concepts that are completely new to me.

A good example is threading. The isn't an entirely new concept to me because it's something I'm familiar with at a high-level through my day job, but I did not expect that, for something as seemingly simple as pausing play-by-play text between results, I would need to read through a ton of documentation and spend a good part of the last two weeks trouble shooting and re-writing large sections of code.

On the plus side, it's made me a better programmer, and it's definitely improved the structure of the sim!

[SirBlurton]

Totally agree with all the compliments, it does look amazing.

I've been struggling through the first couple of chapter of the PyQt book you referenced earlier in the thread, so I fully appreciate the amount of work it took you to get this far!

I haven't run into the threading issue yet, but my goal project is somewhat similar to yours - wondering if you could give any more detail on what resources you used to figure out what you needed to re-write?

[Groundhog]

The PyQT book is super dry, I agree, mostly due to it's age. It's good starting point, but in general I've seen people do the same things in better ways based on posts to stackoverflow, and have opted to follow those. I'm not using PyQt, I opted for PySide for licensing reasons, but they are almost identical so it hasn't been an issue.

Keep in mind that I'm far from an expert on any of this, but I'll just detail my experience for you in case it helps - I managed to solve my issues, but there are probably better ways to do it.

The problem I encountered was just a problem with controlling how data is passed between classes... basically, OOP. My program's structure is one .py file with the main UI logic - navigation, configuring widgets, displaying data, etc. and then many separate .py files containing classes for different aspects of the game - one for handling the saving/loading of data, one for controlling the scoreboard, generating players, the actual simulator engine, etc.

The problem you quickly encounter is blocking - I actually ran into this first working on a BBCF->FOF utility that scrapes thousands of web pages to build player profiles. Your Qt UI is one big loop and any process that blocks the loop from updating the display makes your UI unresponsive, as if it's hung. The best way around this is to take these processes outside of your UI loop and have them run in another thread. Qt has it's QThread class which is exactly for this purpose. You inherit the QThread class*, put all your blocking code in it's run() method, then kick it off via the start() method:

def run_blocking_process(self)
    athread = MyInheritedQThreadClass()
    athread.start()

This worked fine for nearly everything. The ticker at the top of the simulator screen for example - it's a 'While True:' loop that displays text, sleeps, displays text, sleeps, etc. until the sim finishes. It uses signals and slots to connect to the simulator engine and retrieve stats, then connect to the main UI class and update the widget with a signal, because you should never update a widget directly from another class, even a QThread, as it will cause blocking.

Pausing the play-by-play of the simulator class would also block the UI, so obviously I needed to convert the (huge) class to a QThread and follow this same logic. I changed the simulator class to inherit from QThread, and put all the 'sim possession' logic under the QThread's run() method, to prevent the time.sleep() function after each line of PbP from blocking the UI. This was no small task but was also a good opportunity to clean up my 8 or so months worth of simulation code (that's still ongoing). The PbP text for the GUI as well as scoreboard data etc. was all sent via signals rather than by directly updating widgets, etc.

To my frustration, this did not work. The UI still blocked and I had no idea why. I made a stripped down test class, and it did work, so what was the problem.....

The problem was that my simulator class's __init__ method was loading all the player and team information from my main program class, which was making it a child of this process, rather than a true threaded process. The simulator class also directly accessed some of the variables loaded into memory by the main class - some player/team attributes, etc. - which led me to read a little bit about the difference between 'stack' and 'heap'.

So the solution was to build the simulator class, THEN load ALL the values for teams/players/etc. into that class so it doesn't reference back to the main program class - this is the OOP part I mentioned above.

This is a simplified example of what I ended up with:

def run_game(self, home_team_id, away_team_id):
    simulator = Simulator()
    values = self.data_handler.build_simulator_values(home_team_id, away_team_id) # Load everything needed by the simulator into a dictionary.
    simulator.load_values(**values) # Unpack these values into the simulator class.
    simulator.start() # Magic happens!

Hopefully some of this makes sense... Obviously feel free to pm me if you have any Qs. I can't guarantee I'll know the answers, but it's likely you'll run into the same problems along the way that I have!

*One small note about inheriting the QThread class... If you google QThreads, it seems to be popular opinion that this is not the recommended way to do this. There's another way to use QThreads, but I found this way easier to work and it seems to work fine for me.

[SirBlurton]

Wow, thank you so much for the detailed write up - I've saved it for future reference. I was just starting to bump into some blocking issues but couldn't figure out what was happening....you've saved me a great deal of time.

I really appreciate you taking the time to give this advice and share your experience! Also wanted to say that there've been a few times where I've wondered if I should even bother continuing down the Python road because I couldn't find similar projects - your project has helped me move past that internal debate so wanted to thank you for that too!

Also appreciate the offer around pm's...will likely take you up on that - like you I've spent time on my core simulation loops and now am trying to figure out how to tie all my spaghetti together to make a cohesive program.

[Groundhog]

Originally Posted by SirBlurton

Also wanted to say that there've been a few times where I've wondered if I should even bother continuing down the Python road because I couldn't find similar projects - your project has helped me move past that internal debate so wanted to thank you for that too!

To be honest with you, this is something I still think about. After getting a little better with Qt I knew Python could do what I wanted it do, functionally and graphically.

The biggest 'problem for another day' I have though is if I want to actually sell this simulator at some point. Compiling/obfuscating Python code is problematic, and just simply not the goal of the language.... There are options like Nuitka, Cython, and learning a little C or C++ to build extensions, but I haven't toyed with any of them as yet. I suspect this is the major reason why there aren't more projects like these out there...

The trade off of course is that I don't think there's another language where I could have gone from zero to where I'm at now in the time it's taken. If I'd tried learning C or C++ I'd probably still be trying to get my head around if/else statements.

[SirBlurton]

I totally hear you on the if/else statements...I've dabbled with C# for Unity and I find it takes me so much more time to accomplish the same things because I'm tripping over syntax. I just like Python so much more...

I've only done a little bit of reading about compiling but you're right that it doesn't seem to be the focus...hopefully a clearer path will emerge when you need it! I have a bookmark saved for cx_Freeze as something to look into but I haven't explored it in any depth at all.

[Groundhog]

cx_Freeze and Py2exe both work as far as freezing the python code to an .exe and including all the different modules, but they are just as easy to unfreeze. With Py2exe you can use PyInstaller to retrieve the .py files (or .pyc - I can't recall which) and the same is also true with cx_Freeze, so it works as far as packaging your program as a standalone exe, but does not protect the source code.

[Groundhog]

I've done some work with the UI around player stats, which I'll show some screens of once I'm happy with it. Some more boring stuff now...

After spending time planning how to handle the saving/loading of data - mainly related to database queries and converting the output of these queries into the format I use within Python - dictionaries - I had an idea that seems stupidly simple in hindsight... why am I even using databases? I mean, I understand the benefits of relational databases, but why spend the time converting data between SQL and my Python dictionaries when I can just serialize the data using Python's pickle module or something similar?

There are some drawbacks to going with this method, but if I maintain my own 'foreign keys' correctly, I think the pros outweigh the cons. It also makes putting together a pure-Python editor for data very, very simple.

[Groundhog]

Originally Posted by Groundhog

cx_Freeze and Py2exe both work as far as freezing the python code to an .exe and including all the different modules, but they are just as easy to unfreeze. With Py2exe you can use PyInstaller to retrieve the .py files (or .pyc - I can't recall which) and the same is also true with cx_Freeze, so it works as far as packaging your program as a standalone exe, but does not protect the source code.

A little more on this - it sounds like Nuitka is the solution. Nuitka freezes to .exe (and some .dll files) and is not nearly as simple to decompile back to the python source. I'm not positive on how well (if at all) it compiles PySide/PyQt though.

[SirBlurton]

Thanks for posting about Nuitka, will definitely check it out!

[Groundhog]

Stats... I figured this would be easy, but it took me a good 3 weeks to come up with a good way to display the stats (all of which are just test data right now), calculate career totals, and restrict by leagues and regular/post season on the fly...

Added hover text to the league acronym to give a summary of what nations take part in the league, because I think this will come in handy when scouting guys for foreign leagues.

[SirBlurton]

This looks awesome...I love seeing this thread every time it's updated.

Understand if it's a trade secret - but would love to know how you got it to update on the fly...

[Groundhog]

Definitely not a trade secret! This will be a lot of info, hopefully it's helpful.

First I came up with a structure to store the stats. It's a dictionary structured like:

Player_ID > Year > League_ID > Team_ID > 'season'/'postseason' > Stats

The league selection combobox is filled each time a player is opened, the program loops through all the leagues stored in the player's stats similar to the below:

# Initialize a blank list.
leagues_played = []

for year in PLAYERSTATS[player_id]:
    for league in PLAYERSTATS[year]:
        leagues_played.append(league)

the 'league' variable here is the league_id value, a unique number that ties it to a key in another dictionary that stores all the leagues. I then create a 'set' structure of this list (one of those things you never think you'll need when you learn it!) of leagues to eliminate duplicates, and use it to populate the combobox options, including an 'All' option as well if the player has played in more than one league.

I have another dictionary that looks similar to this:

stats_settings = {'league': id, 'season_or_post': 'season', 'format': 'totals'}

Changing the league selections through any of the comboboxes or radiobuttons changes these settings here, and then refreshes the stats. So if the league was set to a particular league_id with 'season' stats I would query all the player's stats like:

raw_stats = []

for year in PLAYERSTATS[player_id]:
    for league in PLAYERSTATS[year]:
        if league == league_id_ive_selected:
            for team in PLAYERSTATS[player_id][year][league]:
                raw_stats.append(PLAYERSTATS[player_id][year][league]['season'])

This gives me a list of all the raw stats for a player as per the criteria set by the different selection widgets. The stats object is a list that contains the dictionary of season stats, as well as the other info needed for the row of stats that looks like [year, league_id, team_id, {'G': 82, 'GS': 82, 'FGA': 2, etc.}].

The next step is what I call 'pretty stats', which is to format the stats correctly - totals, averages, per36, advanced (not done yet). Each stat object is passed to the 'pretty stats' function to generate a list that is formatted for display. So for example if 'averages' has been selected, the pretty stats function will return something like:

[year, league_id, team_id, 82, 82, 1.5, 3.2, .401, etc.]

This is appended to a new list, and is ready to add to my datamodel for display via the QTableView widget. Last step is to come up with the totals for the selected league. This was the hardest part. A copy was made of the original raw_stats list (before any other changes were made), and I'm going to post my solution for this in full, changing a few function/variable names so it makes more sense.

I'm not necessarily happy with this solution, but it works. It's a bit messy and I'm sure there's a more efficient way to do it.

    def stats_career_totals(self, raw_stats, format_):
        # Take all the stats in the raw_stats object, and determine the career average or total based on format_
        # attribute. Exclude the attributes we don't want to calculate first.
        # TODO: Not happy with this method at all. Find a better way to do it.
        calc_fields = []
        return_list = []

        for i in raw_stats:
            i = i[4] # i[4] is the dictionary of stats - i[0]-i[3] is the info columns for each row (year/league/etc)
 
            # x variable is a 'throw-away' list that makes up all the elements to be added together to determine the
            # total.
            x = []
 
            # A list of the different categories - G/GS/FGM/etc. This is used to make sure the stats are ordered in the correct order - and ordered list - 
            # so I can use the index of the list to determine which stat is which.
            for cat in self.STAT_CATEGORIES: 
                if cat not in ['FG%', 'FT%', '3P%']:
                    x.append(i[cat])
 
            calc_fields.append(x)
 
        # Add all individual elements in the lists together to get the total.
        # This 'magic' line of code was the part that took me (with help from Google) a long time to come up with.
        TOTALS = [sum(i) for i in zip(*calc_fields)]
 
        # Iterate through the TOTALS, using the index to identify the FG%/3P%/FT% fields.
        for index, stat in enumerate(TOTALS):
 
            if index in [0, 1, 5, 7, 9]:
                # GP and GS
                if index in [0, 1]:
                    return_list.append(stat)
                # FG%
                elif index == 5:
                    return_list.append(self.player_profile_stats_format_shot_percentage(TOTALS[3], TOTALS[4]))
                # 3P%
                elif index == 7:
                    return_list.append(self.player_profile_stats_format_shot_percentage(TOTALS[5], TOTALS[6]))
                # FTs
                elif index == 9:
                    return_list.append(self.player_profile_stats_format_shot_percentage(TOTALS[7], TOTALS[8]))
 
            # Don't want to calculate anything for the 'G' or 'GS' stats - just use the raw figures.
            if index not in [0, 1]:
                if format_ == 'average':
                    # Fetch the average of all games played / tallied stats.
                    return_list.append(self.player_profile_stats_format_average(TOTALS[0], stat)) # TOTALS[0] = 'G' stat.
                elif format_ == 'per36':
                    # Fetch the per36 average of the tallied stats.
                    return_list.append(self.player_profile_stats_format_per36(stat, TOTALS[2])) # TOTALS[2] = 'MINS' stat.
                elif format_ == 'totals':
                    # Just return the totals.
                    return_list.append(stat) # Append the raw stat.
 
        return return_list

The total row is appended to the other 'pretty stats' and ready for display. These pretty stats and a list containing the headers are then built into a QAbstractTableModel, which is then assigned to the QTableView widget to be displayed. I make this part sound trivial, but the 'model view' style of programming with qtableviews and item models is in itself a few weeks of work to get a semi-decent grasp on... Definitely worth the time though, this guy's series of tutorials was invaluable:

PyQt4 Model View Tutorial Part 01_0 - YouTube

[SirBlurton]

That's awesome - thanks for sharing!
Love the 'magic line'...list comprehensions were definitely an "ah-ha" moment for me.

Did you end up going with pickle for storing your stats?

I've been playing with JSON for my project.

I started by using .csv files to read and write my data but it was taking a full minute in some cases (which could have been due to extremely poor implementation on my part). I decided to try JSON, thinking it wouldn't make much difference, but it went from a minute to a second to complete the operation without changing anything else about the approach.

Thanks for the model view link! And for being so open about your process in general, learning a lot! Always been shy about posting but considering posting some status updates of my project as a way to keep myself honest.

[Radii]

Couple thoughts if you are interested (and if you are not, let me know, what you're doing and sharing here is awesome, I don't want to potentially bog you down over anything that you're satisifed with)

If you are using the raw_stats dictionary in a lot of places othern than this one display screen, did you consider creating a class for it instead of using the string dictionary? Being able to refer to raw_stats.GP instead of remebering that games played is in index[0] could make the data a lot esier to work with.

Alternatively, if the dictionary is the best way to go, what about an enumeration so that you don't have to refer to the index numbers in hard coded fashion every time you touch the stats. Something that sets Stats.GP = 0, Stats.GS = 1, Stats.MINS = 2, etc. If you decide you want to change the layout of things down the road, or change the order, or alter the data based on the addition of some new advanced stats calculation, then you only have to change it in one place instead of many.

I should point out I don't use Python, just generic programming thoughts. May or may not be relevant to the exact way you're doing things


Keep it up, this is an awesome read and its great to see the progress you've made.

[Groundhog]

Originally Posted by SirBlurton

That's awesome - thanks for sharing!
Love the 'magic line'...list comprehensions were definitely an "ah-ha" moment for me.

Did you end up going with pickle for storing your stats?

I've been playing with JSON for my project.

I started by using .csv files to read and write my data but it was taking a full minute in some cases (which could have been due to extremely poor implementation on my part). I decided to try JSON, thinking it wouldn't make much difference, but it went from a minute to a second to complete the operation without changing anything else about the approach.

Thanks for the model view link! And for being so open about your process in general, learning a lot! Always been shy about posting but considering posting some status updates of my project as a way to keep myself honest.

Please do post, I would definitely be interested in reading your progress too!

I'm using pickle now, but I'm not sure how well it's going to scale up. A pickled object of historical stats for example could end up being quite a big object and take awhile to load when needed, so I've made it easy to change this down the line if I feel I need to go a different way down the line.

I use JSON for storing all my 'settings' data. Originally I used it for everything but having to convert the type of loaded JSON from strings to other types was a bit of a pain... There are ways to handle this, but for now I've opted for pickle. I see the advantages to SQL databases and did spend a bit of time reading up on them - managing my own 'foreign keys' to link dictionaries together in particular feels like it might get out of hand pretty easily.

[Groundhog]

Originally Posted by Radii

Couple thoughts if you are interested (and if you are not, let me know, what you're doing and sharing here is awesome, I don't want to potentially bog you down over anything that you're satisifed with)

If you are using the raw_stats dictionary in a lot of places othern than this one display screen, did you consider creating a class for it instead of using the string dictionary? Being able to refer to raw_stats.GP instead of remebering that games played is in index[0] could make the data a lot esier to work with.

Alternatively, if the dictionary is the best way to go, what about an enumeration so that you don't have to refer to the index numbers in hard coded fashion every time you touch the stats. Something that sets Stats.GP = 0, Stats.GS = 1, Stats.MINS = 2, etc. If you decide you want to change the layout of things down the road, or change the order, or alter the data based on the addition of some new advanced stats calculation, then you only have to change it in one place instead of many.

I should point out I don't use Python, just generic programming thoughts. May or may not be relevant to the exact way you're doing things


Keep it up, this is an awesome read and its great to see the progress you've made.

Appreciate the tip, and I agree that that's a really good idea!

My current coding process is:

1) Try to think how something could work.
2) type away madly.
3) test through trial-and-error until I get it to work.
4) Completely rewrite/refine it from scratch to match the structure of everything else.

Right now I'm on step #4 with stats, and I'm going to do exactly what you propose.

[Radii]

Originally Posted by Groundhog

1) Try to think how something could work.
2) type away madly.
3) test through trial-and-error until I get it to work.
4) Completely rewrite/refine it from scratch to match the structure of everything else.

Yup! The more experience you get doing steps 2-4, the better you'll get at step 1. I am not a great designer, its not uncommon for me to start planning out class structures and to reach a point where I get stuck and can't visualize everything as well as I wish I could and just start writing something. You could easily go look at a project I finished and pick out the parts that I slammed together vs the parts that were well thought out. Always something to learn I suppose

[Groundhog]

I ended up creating two new classes, one specific for working with the individual rawstats - it takes the dictionary entry of a specific season's stats in the dictionary format that it's stored, and returns it in a format ready for display - and a class that takes a player's whole statistical history and will return it (using the rawstats class per entry) depending on the restriction applied - all stats for all leagues, all stats for one league, all stats for one league in one year, all post-season stats for one league etc. as well as generate the 'career totals' row on the bottom.

The end result is much the same as what I had before these two classes, but the amount of code needed was much, much less and is much cleaner. Thanks Radii.

[Radii]

Originally Posted by Groundhog

The end result is much the same as what I had before these two classes, but the amount of code needed was much, much less and is much cleaner.

Awesome stuff man! Can't wait to see what's next!

[Young Drachma]

Glad you're still trucking!

[Groundhog]

I put some work into cleaning up the code around styling the Qt forms, as this was some of the first coding I did and it lacks consistency, especially the first forms I put together. To be less polite, it works, but it's a real mess.

Qt Designer is one of the best reasons to use the Qt framework, but I really only use it to lay out my forms, and do all the actual styling from within Python. I chose to go this route (you could do it through Qt - you can set StyleSheets from within the Designer and see the results instantly), but if you want to be flexible and allow custom colour schemes you will be using the .setStyleSheet method for your widgets, and using the string formatted stylesheets, which are similar to .css in html, with variables.

As an example, a simple QPushButton widget's style sheet will look like:

standard_button = '''QPushButton {{
                                       color: rgb{};
                                       background-color: rgb{};
                                       border-bottom-left-radius: {}px;
                                       border-bottom-right-radius: {}px;
                                       border-top-left-radius: {}px;
                                       border-top-right-radius: {}px; }}
                     QPushButton:hover {{
                                             background-color: rgb{};
                                       }}
                     QPushButton:disabled {{
                                                background-color: rgb{};
                                                color: rgb{};
                                          }}'''

One of the downsides to Qt is that changing the original style is not enough to then go on and update all the widgets with it assigned - it's obviously just a stored string, so no signals are emitted to let all the widgets know to reapply it. I figured there would be a quick and easy way to do this, but after some google searches it seems not. To allow a user to change the colour of a push button, you will need to update the above string with the new rgb values, and then re-apply the stylesheet to all the widgets.

To handle this, I created a new class. I won't post the whole thing as it's huge, but this is a quick snippet of how it is structured for the settings screen:

class Style:

    def __init__(self, app):
        # The 'app' variable is my application itself, which gives the Style class access to directly modify all the widgets.
        # So when I create an instance of this class from within my application, I create it as self.style = Style(self).
        self.app = app
        # Class attributes - this will store all the stylesheets strings.
        self.attrs = {'css': {}}
        self.store_ss_strings()
        self.setup_shortcuts()
        self.init_all_widgets()

    def setup_shortcuts(self):
        # The quick command to retrieve the stylesheet and current variables stored in settings.
        self.mainwindow_panel = self.attrs['css']['mainwindow_panel'].format(self.retrieve_styling('mainwindow_panel'))

    def store_ss_strings(self):
        # Define all the stylesheets strings and store them in the self.attrs dictionary.
        self.attrs['css']['mainwindow_panel'] = mainwindow_panel = '''QLabel {{
                                                                                 color: rgb{};
                                                                                 background-color: rgba{};
                                                                                 border-bottom-left-radius: {}px;
                                                                                 border-bottom-right-radius: {}px;
                                                                                 border-top-left-radius: {}px;
                                                                                 border-top-right-radius: {}px;
                                                                           }}'''
      
    def init_all_widgets(self):
        # This method is called upon creation of the class instance and applies all stylesheets to the different widgets,
        # via the individual method per Qt form. It uses the colours and other settings related to styling in the main
        # application's 'settings' class, so if changes are made to the colour scheme, this method is run again to re-style
        # everything.
        self.init_settings_form()

    def retrieve_styling(pointer):
        # Fetch the colours from settings class.
        if pointer == 'mainwindow_panel':
            r, g, b, a = self.app.settings.panel_background_color[0], self.app.settings.panel_background_color[1], \
                         self.app.settings.panel_background_color[2], self.app.settings.panel_transparency[0]

        # This dictionary supplies all the different settings needed by the stylesheet string.
        lookups = {'mainwindow_panel': (self.app.settings.panel_text_color, (r, g, b, a), self.app.settings.corner_radius,
                                       self.app.settings.corner_radius, self.app.settings.corner_radius,
                                       self.app.settings.corner_radius)}

        return lookups[pointer]

    def init_settings_form(self):
        # Apply the style sheets to all widgets.
        self.app.ui.settingsPanel.setStyleSheet(self.mainwindow_panel)

[Groundhog]

Initial work on the nation screens. I want to make it real easy to add/remove nations, so it took me some time to work out how to best generate menus and screens 'on the fly' based on nation information stored in a csv file. The sim will read the file, populate the menus, and colour the scheme based on the foreground/background colours defined for the nation.

Leagues work in a similar way, with the menu system grouping by Region > Nation > League(s).

[SirBlurton]

Sooooooooo pretty....Guessing it's too early to tell you to shut up and take my money?

[LastWhiteSoxFanStanding]

Looking great. Hope you can include Singapore too

[Groundhog]

Ask and you shall receive.

...although still need to put the first names/last names and city files together.

[LastWhiteSoxFanStanding]

Talk about great customer service!

Can't wait to see the next update.

[Groundhog]

No updates in a while, but it hasn’t been for lack of progress (relatively speaking). This post will more be for those of you who are interested in the programming side of things… ie. lots of text, no pics.

After 100,000+ lines of code, much more if you count deleted/restarted lines of code, it was time to take a step back and look at the structure of my program. Things were getting a bit messy and, it seemed to me, overly complicated. I had been doing some reading lately on software design patterns (half because of this project, half because I think it’s something I might be interested in doing for money one day) and while reading through the different patterns, I was interested to see that without really knowing it I had drifted towards what is called the Model-View-Controller pattern. I won’t add to many details about what an MVC pattern is (there’s lots of info on the web), other than to say that it became pretty clear to me why things were getting messy, and what I needed to do to fix it. It requires re-writing a bit of code, but not excessively so.

The other major thing I’d been looking at was the interface. As monitor resolutions grow, the fixed-window style of GDS and Wolverine Studios sims began to bug me. Early on I realised why they opt to go this way, and why it's not as straight forward to make a program that will 'stretch' to fit your selected resolution, like with any graphical video game you play.

I spent some time the last couple of weeks thinking of ways around it - Qt (the UI framework I'm using) has layouts which are made for exactly the purpose of resizing components to fit a screen. The problem is they don't really suit a program like this IMO, and objects on the screen stretch and shrink and look ugly. For general purpose desktop apps I think it works great, but not for this. So I've decided to stick with what I've got so far, which is a a fixed-window, though in the standard 'widescreen' size of 1280x720 pixels. I'm going to push on with this with the idea that if I want to improve on this later, all of my presentation logic can be replaced without touching anything else (this relates directly back to the MVC stuff above).

As far as the actual simulator goes, I haven't done a great deal of work for a while because I know how it's going to work. There's no problem to solve (unlike quite a few other areas), so I haven't been focusing on it as much. I have been looking at schedules, trying to make it as flexible as I can. Initially there will be two types of schedules - round-robin, and your NBA-styled 82 game schedules. I have a good idea of how to generate both, just ironing out a few wrinkles right now.

[CraigSca]

Always good to hear updates, Groundhog - been missing these!

[Groundhog]

Until this point I've imported all of my data - teams, coaches, leagues, nations, etc. - in a database (sqlite) that I manually populated from csvs full of info I've gathered over the last couple of years, so I spent some time the past couple of days putting together the 'new game' process - main menu/league selections, etc.

Here's a couple of screens:

Title Screen
A quick (5 minute) title screen just to give the standard new/continue/load options.


New Game Menu
This one took a lot longer - the fully functional league selection screen. All the data here is taken from csv files that can be easily modified to add/subtract/modify the leagues included - currently 44 fields per league (some repetitive, like 7 fields for each day of the week that games will be scheduled). Leagues where I've manually created teams (also done via csvs) default to 'import' teams, other leagues will be randomly generated. Active leagues will be bolded (not yet done), and I'll ditch the '#' in the Teams column header.

The Olympics/Tournaments options on the left will be changed to a table like the regional leagues, because it gives me more flexibility with how many I can include without running out of room - there's international tournaments in the Americas, Asia and Africa that I don't plan on looking at now but would probably like to look at down the track:



Nav Bar
Lastly, the top-of-screen navigation bar - not completely functional yet, because I don't have all the screens completed. This image is at 100% actual width, to give you an idea of how wide the UI will be:

[SirBlurton]

Super, super awesome....if you get to the point you want any testing help, I'd love to volunteer! No worries if not though.

Really impressed with your GUI skills, I just got to the point of making my first (very remedial) GUI for one of my sims the other day. Your posts are a reminder that I still have a long way to go!

[Groundhog]

Thanks for the kind words! When I get to the point of testing, this would be the first place I would come. That point is still some ways away unfortunately... inching closer, but lots of work that needs to take place before that point. No idea of timeframes.

UI design is as foreign to me as coding was at the beginning. I just took a look at some of the programs I find I really like, tried to figure out what it was about them that I liked, and then try to figure out how replicate it, or tweak it to fit what I'm doing. Although I didn't go with a web-based program, Flask and the Bootstrap css stuff was a big influence on what I've tried to put together so far, too.

[Groundhog]

I'm revisiting the whole player generation part of the sim now, as the rest of the logic is now all in place to create a new universe.

When I started to learn Python one of the first scripts I came up with was a random name generator, to add more flavour to the foreign recruits in my FBCB game. When putting together the player generation code it was also one of the first things I coded. I set out to make it smarter over the weekend, and allow for different settings for each nation - things like hyphenated names, no surnames, middle names, 'I, II, III, IV, Jnr' names, etc. Although names are purely cosmetic, in the world of text sims I personally find little details like this are important because it gives a bit more uniqueness to individual players which for me adds another important layer of non-technical abstraction between the user and the random number generator that's at the core of all simulators.

Each nation has 2 or 3 name files built mostly by text scraping national soccer league team pages on wiki where possible (and a much more manual process where not). In addition there is a name rules csv file in the same folder that determines % of players with no surnames (Portuguese styled sportsmen - Nene, etc.), % of players with ', II' appended at the end or hyphenated names etc., most of which I determined by data scraping every Div I college basketballer and coming up with %s. You can also specify substitute nations and a % chance of the name being chosen from there. For example, I didn't bother scraping Canada names, instead assigning USA as 100% to be chosen instead. For the USA itself there's a slight chance of a 'Puerto Rican' name, etc.

When I run the name generator now, I specify a nation, and it will produce a name, taking into account the different rules, and spit out a first name and a last name.

Player hometowns work in a similar fashion, except are weighted selections, with more populated cities in each nation a better chance of being selected. USA is slightly different as I included state distribution as well. Again using Div 1 basketball rosters I calculated the % of players from different states, and USA hometowns will select the state first, and then a weighted selection of cities from that state.

I also put together a list of US colleges, breaking them down into 5 levels of talent from top tier schools down to division 2. When a US player is generated, there is a greater chance that higher ability players will have attended higher tier schools.

I'm trying to think of a good way to handle non-US players with college experience. Some nations (like Australia) send a lot of kids to US colleges, and the pro league is largely national players who attended US colleges. This is not the norm internationally though, as most European countries in particular have stronger youth systems and there is not as much drive to play in the US college system when they can earn decent money as teens.

[Young Drachma]

This is looking great!

[Groundhog]

I mentioned that I was trying to make the influx of new talent each year as realistic as possible, and I've spent a couple of nights this week designing a system called 'Youth Settings'. These are a series of rules that determine the age ranges and probability of each age for incoming rookies, as well as the chance of a rookie having attended a USA college. If a generated rookie is determined to have attended college, a calculation is made based on his potential ('scouted' as opposed to 'actual') of how many years he attended college (weighted towards higher potential means less years), which will be used to determine his age instead of the range of ages mentioned earlier - so a higher rated player from the US is more likely to have spent 1 or 2 years at college and enter the league at a younger age than, say, a 3-star type player, who might be 22 or 23 as a rookie after graduation. As mentioned above, the quality of the college selected will be based on the player's scouted ability, too.

Right now I have setup 10 different categories of youth settings to cover the NBA (99.9% chance of college, to allow for the Mudiay/Jennings exceptions in theory - ie. they wouldn't actually play overseas for a year at this point, and would just be generated at 17 or 18 with 'None' for college), high level European nations (younger rookies, less chance of having attended college), mid-tier European nations (slightly higher chance of college), Latin America (above-average chance of college), etc. These are mapped out in a csv file with a reference number (or 'primary key'). I added an additional column to the csv containing all the countries information to point to the different youth settings.

It's working as expected, but further down the line I want to add more logic to it - ie. a one-star Nigerian rookie with 4 years college experience is probably not going to want to play professional basketball in Nigeria for peanuts. One way to handle this would be outside of the Youth Settings, making anyone with a college other than 'None' a lot less willing to sign in low salary leagues, or rather than just having a single % to determine how many rookies have college experience, weight it towards only the stronger players generated from that nation. I'm tabling it for now, but it's something I'll have to come back to when it comes time to think about free agency.

[tarcone]

I have been watching from afar. I really like reading this and watching your progress. Very impressive.

Keep up the good work and I am looking forward to watching this project come to its full potential.

[Groundhog]

Thanks!

It's been a slow couple of weeks - really struggling to come up with a player progression system that I'm happy with. Physical progression is in - players will develop in terms of height and weight (and speed/strength/jumping ability). Players will have growth spurts between 16-18, and potentially up through 20. Strength develops mostly around the 20-24 years.

What I'm struggling with now is 'basketball' attribute progression - whether to generate 'potential' per rating (which would be hidden), or have players develop semi-randomly.

My initial stance was semi-random, but I don't like how chaotic it makes the player progression system. Now I'm leaning the other way - players will be generated with a 1-5 potential rating in each attribute to indicate their ceiling, and an overall 'potential' rating that is partially masked until a certain age. I continue to tweak it.

Once I'm happy with this, I'll be working on the player generation logic per nation/league, and then initial work on the financial side of things, which is a little daunting to even think about to be honest!

[Groundhog]

Been awhile since an update. The previous post outlines the biggest reason for that, and I basically had to shelve it for now and just push on with other aspects of the sim. Hopefully I'll come up with a solution I'm happy with further down the line.

I put the finishing touches on the play-by-play "engine" today. It's completely customisable, with all text and the chance of that text appearing being stored in an .ini file as per the following quick example:

[introduction]
option1 = "Welcome to $home_team_arena, $home_team_display_city, where the $home_team_name are up against the visiting $away_team_name.|100"

[pre_opening_tip]
option1 = "$home_team_jumper and $away_team_jumper are in the middle for the opening tip, and we're ready to go.|50"
option2 = The referees blow their whistles, $home_team_jumper and $away_team_jumper are lined up for the opening tip, and we're ready to get things underway.|50"

Every bit of text in the PbP is in the ini file (outside of misc. stat reports, information re: the scores/time left, etc.), so it allows for some added flavour to the pbp, especially for things like dunks, passes, etc.

[SirBlurton]

Hah, I'd thought twice this week about bumping the thread for an update but tried to stay patient!

How do you end up processing the text file? That's something I haven't seen, I've been using massive dictionaries....what are the $ tags and |100 things? Codes for replacing when processed?

[Groundhog]

It's not Python at all, just tried to come up with something that is easy to edit by someone who doesn't know Python.

The '|integer' thing is the chance of that line appearing, and the $ tags are words that will be replaced by the Python equivalent, and then I use simple string operations (split on "|" to get the chance, and then a string replace method to swap out the $ tag).

The $ tags seemed like a good idea at the time but I might just replace them with the Python syntax - "{home_team_jumper}" instead of "$home_team_jumper".

I read this in with the configparser module rather than just importing the strings, but I do then build a dictionary that puts them in an easier format to work with.

[nilodor]

This is really awesome, I've been working on something similar, although I'm at about midway down the first page in figuring out how to get it done. I'm very inspired!

[Groundhog]

Originally Posted by nilodor

This is really awesome, I've been working on something similar, although I'm at about midway down the first page in figuring out how to get it done. I'm very inspired!

Thanks! What language are you working in?

[GoldenCrest Games]

Just wanted to join the rest in saying this is an awesome dynasty topic!

Love all the little details you put in. I hope you keep it going!

[Balldog]

I've started learning Python due to this thread. What you have so far looks amazing.


Sent from my iPhone using Tapatalk

[nilodor]

Originally Posted by Groundhog

Thanks! What language are you working in?

I had started off in unity, then went to Java and now am in Python. You are way ahead of where I am but you've certainly inspired me to keep going.

[GoldenCrest Games]

It just dawned on me that this has been a 2 year project! That's impressive.

Kudos for grinding away at it. I have such a hard time staying with projects for that long!

[Groundhog]

Yikes, thanks, I didn't even realize. Thought it was about 18 months. Most interesting thing has probably been how much I've adopted Python at work off the back of trying to make this thing - some of which has made my team's lives a lot easier (or at least, a lot less monotonous).

Only update I have lately is I've revisited the player generation/progression logic that I put aside because I thought of a way to do it. During player generation, I determine a 'ceiling' for each player, as well as their age, athletic profile and 'personality' attributes. The players are then "grown" to their current age with all progression being applied as it would during a normal offseason.

Currently the plan is to have players only progress in the offseason, but I like the idea of having a chance of 'bonuses' applied to player's attributes over the course of a season based on court time. The bonuses would take into account the player's potential in each rating, and be lost in the offseason, but playing time would also be a factor in offseason progression. I think it kind of models what happens in reality to some players, but I'm not sure if it will make its way in initially.