Sub Out is the setting for when a player comes out of the game. So if you set "sub out" at 75, players will come out of the game when their stamina is 75 or lower. Sub In describes when a player is ready to return. So setting Sub In at 90 says players shouldn't return until stamina is 90.
You may want to enter a game and look at a player who you think is ready (yellow) for subbing. Go into player management while pausing the game and see what their current stamina is. Take note of it and make that your "Sub Out."