Ben E Lou
10-09-2007, 08:01 AM
This article is designed to help those who are newer to FOF with regards to how to evaluate players. This is taken mainly from a couple of posts I made a few minutes ago in the OSFL.
PRESEASON: WHY NOT?
I cannot stress strongly enough that you're shooting yourself in the foot if you rely on such a small sample size as the preseason (especially in MP, where teams are using their talent and game plans in *widely* differing manners--I'll address this later) to make *ANY* determination.
You cannot, cannot, cannot, cannot judge players in FOF by preseason performance. And, quite honestly, NFL teams don't do it that way, either. All those practices (simulated in FOF by ratings changes) have MUCH more of a bearing than the preseason games.
FOF STATISTICAL FLUCTUATION
One reason I make such a blanket statement above is because of the level of statistical fluctuation in FOF (and in real football). For an example of what I mean, check out the test results in this post (http://www.operationsports.com/fofc/showpost.php?p=1564091&postcount=82). In those test runs, everything was kept *IDENTICAL* for 16 games over 15 seasons in each of the charts, with injuries off. So, that's as little fluctuation as possible. You'll get much more in a real game situation, especially against human GMs. Notice a few things...
In chart one, yards per carry ranged from 4.07 to 5.07. That's over 16 game trials--roughly 500 carries in each sample. Imagine how much it can vary from the norm over just 2-4 games played and maybe 30-40 carries for any particular player. You might could easily play four games at 3.0 ypc, and decide your guy or your gameplan sucks, and cut him or scrap the plan, and be getting rid of a 4.5ypc back/plan based on a tiny sample size.
Also in chart 1, QB rating varied by 26 points--again over sample sizes of roughly 500 passes. It's pretty reasonable to think, then, that your QB/game plan that should really produce an average rating of 85 guy could *easily* put up anywhere from, oh, 50 to 120 in a four-game stretch with such variable receiving talent, variable opposition talent, and variable opposition game plans.
In chart one, points per game varied by 10. It's quite possible that you could score, oh, 17ppg with a game plan in the preseason, and be scrapping a 25ppg game plan.THE OPPOSITION'S USE OF TALENT AND GAME PLANS
I'd contend that more accurate evaluates can be made in SP than MP based on the preseason (and when I say "more accurate", I mean like, say, with 20% certainty as opposed to 10% certainty...still nothing worth making decisions on). I say this because of two issues that exist in MP that do not exist in SP:
1. MP GMs use their game plans very differently in the preseason. A few examples...
Some are trying out specific stuff with specific players here. For example, in the first FOF2K7 preseason of IHOF, I used the formation screens to put my RB and TE in the wideout slots to see how'd they do there. They sucked, and probably shouldn't be used in those roles. (I've since learned that it's a rare RB/TE who should be used in that role unless the WRs just absolutely suck.) If someone who played against me used those games to evaluate their DBs, they would have thought their guys were much better than they were.
Some are going very vanilla, so their early-season opponents don't have a read on what they might try in the first few weeks of the regular season. I know I use rex, or something that I'd never do in the regular season.
Some are trying to evaluate new ideas that might really, really, really suck in FOF. If someone without a good FB were to try to use a preseason game to see if they could operate out of the single-back exclusively (hint: doing so really, really, really, really sucks in FOF ;) ), you might *think* that whatever defensive players were lucky enough to be on the field for most of that game are better than they are.By contrast, you know that in SP, you're never going to face a game plan that sucks by nature.
2. MP GMs use their PLAYERS very differently in the preseason. Some play scrubs the first couple of games, then starters later on. Some play a mix of starters and scrubs. Others play their starters the first couple of games, and scrubs late. So you'd need to take into account opposition player usage. And even if they're going vanilla, other than the two or three players mentioned in the log, you don't know who you were facing on a particular play.
SO, HOW DO I KNOW WHAT TO DO WITH MY GAME PLAN AND WITH THESE NEW ROOKIES AND FREE AGENTS?
I personally use the following to evaluate players:
Veteran players: RATINGS--Yes, there's the scout error issue (which has been reduced for players generated by 6.1), but it still appears to be the best overall indicator for players with maybe 3 years of experience or more.
Young players: RATINGS MOVEMENT--For young players, this is probably more important than their scouted ratings. My theory is that young players who are moving rapidly have not just higher future potential than we see, but higher current ratings, too. For example, my 34 current-rated RB has had a pretty good rookie year (http://www.thefofl.com/players/player.php?player=13181) in the FOFL, and we would probably do well to pay attention to Chuck Milnes (http://www.fof-ihof.com/player/player.php?playerid=11567) in the IHOF as his season/career unfolds. He's rated a rousing 31/37 by my scout at the moment. (For further discussion of unmasking, go here (http://www.operationsports.com/fofc/showthread.php?t=61252).) Ratings movement isn't the greatest "realistic" thing, but it is a pretty decent proxy for guys performaing well in practice, which we don't get to see in FOF. Maybe down the road we'll get some sort of practice report, but for now, this is the primary mechanism for seeing if a guy is going to be better than he looks on the surface, so it's important to watch.As far as game plans, to evaluate what works and doesn't work in game planning, SP is the way to go. There are some things (like the fact that you don't ever, ever, ever, ever want to try using only the single back formations :p ) that can be picked up by just simming one season. Other subtleties of FOF game planning take seasons of playing to understand.
I wouldn't base ANYTHING on a game plan on one or two games, though. As a small example as I close, the very first quick test season I ran to illustrated this produced exactly the point I wanted to make. I ran a season with injuries off, and with the same game plan every week. It produced a total 2,034 yards rushing, 4.74 yards per carry and 17 TDs for the primary RB. However, he had a game against a mediocre defense in which he had 26 carries for only 67 yards, and the next game he had 31 carries for 103 yards against a bad rushing defense. If those had come back-to-back at the beginning of the season or in the preseason, and I'd evaluated the game plan on those two games with a total of 2.98 yards per carry, I would have scrapped it and gone in another direction-which clearly would have been a horrible move.
PRESEASON: WHY NOT?
I cannot stress strongly enough that you're shooting yourself in the foot if you rely on such a small sample size as the preseason (especially in MP, where teams are using their talent and game plans in *widely* differing manners--I'll address this later) to make *ANY* determination.
You cannot, cannot, cannot, cannot judge players in FOF by preseason performance. And, quite honestly, NFL teams don't do it that way, either. All those practices (simulated in FOF by ratings changes) have MUCH more of a bearing than the preseason games.
FOF STATISTICAL FLUCTUATION
One reason I make such a blanket statement above is because of the level of statistical fluctuation in FOF (and in real football). For an example of what I mean, check out the test results in this post (http://www.operationsports.com/fofc/showpost.php?p=1564091&postcount=82). In those test runs, everything was kept *IDENTICAL* for 16 games over 15 seasons in each of the charts, with injuries off. So, that's as little fluctuation as possible. You'll get much more in a real game situation, especially against human GMs. Notice a few things...
In chart one, yards per carry ranged from 4.07 to 5.07. That's over 16 game trials--roughly 500 carries in each sample. Imagine how much it can vary from the norm over just 2-4 games played and maybe 30-40 carries for any particular player. You might could easily play four games at 3.0 ypc, and decide your guy or your gameplan sucks, and cut him or scrap the plan, and be getting rid of a 4.5ypc back/plan based on a tiny sample size.
Also in chart 1, QB rating varied by 26 points--again over sample sizes of roughly 500 passes. It's pretty reasonable to think, then, that your QB/game plan that should really produce an average rating of 85 guy could *easily* put up anywhere from, oh, 50 to 120 in a four-game stretch with such variable receiving talent, variable opposition talent, and variable opposition game plans.
In chart one, points per game varied by 10. It's quite possible that you could score, oh, 17ppg with a game plan in the preseason, and be scrapping a 25ppg game plan.THE OPPOSITION'S USE OF TALENT AND GAME PLANS
I'd contend that more accurate evaluates can be made in SP than MP based on the preseason (and when I say "more accurate", I mean like, say, with 20% certainty as opposed to 10% certainty...still nothing worth making decisions on). I say this because of two issues that exist in MP that do not exist in SP:
1. MP GMs use their game plans very differently in the preseason. A few examples...
Some are trying out specific stuff with specific players here. For example, in the first FOF2K7 preseason of IHOF, I used the formation screens to put my RB and TE in the wideout slots to see how'd they do there. They sucked, and probably shouldn't be used in those roles. (I've since learned that it's a rare RB/TE who should be used in that role unless the WRs just absolutely suck.) If someone who played against me used those games to evaluate their DBs, they would have thought their guys were much better than they were.
Some are going very vanilla, so their early-season opponents don't have a read on what they might try in the first few weeks of the regular season. I know I use rex, or something that I'd never do in the regular season.
Some are trying to evaluate new ideas that might really, really, really suck in FOF. If someone without a good FB were to try to use a preseason game to see if they could operate out of the single-back exclusively (hint: doing so really, really, really, really sucks in FOF ;) ), you might *think* that whatever defensive players were lucky enough to be on the field for most of that game are better than they are.By contrast, you know that in SP, you're never going to face a game plan that sucks by nature.
2. MP GMs use their PLAYERS very differently in the preseason. Some play scrubs the first couple of games, then starters later on. Some play a mix of starters and scrubs. Others play their starters the first couple of games, and scrubs late. So you'd need to take into account opposition player usage. And even if they're going vanilla, other than the two or three players mentioned in the log, you don't know who you were facing on a particular play.
SO, HOW DO I KNOW WHAT TO DO WITH MY GAME PLAN AND WITH THESE NEW ROOKIES AND FREE AGENTS?
I personally use the following to evaluate players:
Veteran players: RATINGS--Yes, there's the scout error issue (which has been reduced for players generated by 6.1), but it still appears to be the best overall indicator for players with maybe 3 years of experience or more.
Young players: RATINGS MOVEMENT--For young players, this is probably more important than their scouted ratings. My theory is that young players who are moving rapidly have not just higher future potential than we see, but higher current ratings, too. For example, my 34 current-rated RB has had a pretty good rookie year (http://www.thefofl.com/players/player.php?player=13181) in the FOFL, and we would probably do well to pay attention to Chuck Milnes (http://www.fof-ihof.com/player/player.php?playerid=11567) in the IHOF as his season/career unfolds. He's rated a rousing 31/37 by my scout at the moment. (For further discussion of unmasking, go here (http://www.operationsports.com/fofc/showthread.php?t=61252).) Ratings movement isn't the greatest "realistic" thing, but it is a pretty decent proxy for guys performaing well in practice, which we don't get to see in FOF. Maybe down the road we'll get some sort of practice report, but for now, this is the primary mechanism for seeing if a guy is going to be better than he looks on the surface, so it's important to watch.As far as game plans, to evaluate what works and doesn't work in game planning, SP is the way to go. There are some things (like the fact that you don't ever, ever, ever, ever want to try using only the single back formations :p ) that can be picked up by just simming one season. Other subtleties of FOF game planning take seasons of playing to understand.
I wouldn't base ANYTHING on a game plan on one or two games, though. As a small example as I close, the very first quick test season I ran to illustrated this produced exactly the point I wanted to make. I ran a season with injuries off, and with the same game plan every week. It produced a total 2,034 yards rushing, 4.74 yards per carry and 17 TDs for the primary RB. However, he had a game against a mediocre defense in which he had 26 carries for only 67 yards, and the next game he had 31 carries for 103 yards against a bad rushing defense. If those had come back-to-back at the beginning of the season or in the preseason, and I'd evaluated the game plan on those two games with a total of 2.98 yards per carry, I would have scrapped it and gone in another direction-which clearly would have been a horrible move.