The Importance of Volatility

[agrimsley]

Can anyone explains their thoughts to me on volatility please?

Does a high volatility make it more likely a player's ratings will change at all? Does it make it more likely that any change is a positive one?

[redfox000]

Originally Posted by agrimsley

Can anyone explains their thoughts to me on volatility please?

Does a high volatility make it more likely a player's ratings will change at all? Does it make it more likely that any change is a positive one?

A high volatility means the chance is very great that they could be either a large increase in ratings or a large decrease in ratings in training camps. Usually the move is smaller in size, but with volatility in the mix I have seen players go from 60/80 to around 15/20 in one training camp. I have also seen players go from 20/25 to 50/55, but it seems it's more likely to bomb the player than the other way around. Thus the term VSOD was born.

[agrimsley]

And does experience play any part in the large ratings changes? Are rookies more likely to experience VSOD than a veteran?

[agrimsley]

I was asking in order to try and understand the current belief in the role that volatility plays in VSOD etc. Having reviewed 25 draft classes of just over 20,000 players, the two images below show some of my findings:

• Volatility does NOT effect the likelihood of a change occuring
• Volatility does NOT effect the likelihood of a change being positive or negative
• Volatility does NOT effect the size of a change if one does happen

In image 1, we can see a player's volatility shown across the bottom of the chart and above each one the % likelihood of a change in ratings occurring. The red section of the bar indicates the number of players who experienced a downgrade in ratings, the yellow an upturn and the green indicates no change at all. While the trend is not exact, there is certainly no indication that the number of changes increases and volatility does or that the number of positive changes increases as volatility does.


Image 1

In image 2, the graph depicts the average size of the change for each volatility. The general trend shows that a loss of ratings is generally greater than a gain in ratings and also that volatility appears to play no part in the size of changes.


Image 2

Notes: This test was performed in FOF7 and used data for rookies who were drafted and any changes that occurred in those rookies between the post-camp ratings and the start of the regular season thus covering changes seen during exhibition week 2.

[garion333]

Volatility hits happen in a very small number of players.

How do your graphs take that into account?

[agrimsley]

Originally Posted by garion333

Volatility hits happen in a very small number of players.

How do your graphs take that into account?

The graphs show changes in ratings between camp ratings and EX2 ratings for rookies across 25 years. How would you distinguish which of these rating changes are caused by volatility and which are caused by something else?

What the graphs do show is that 91.4% of all rookies experience a ratings change between camp and EX2.

[Ben E Lou]

Originally Posted by redfox000

A high volatility means the chance is very great that they could be either a large increase in ratings or a large decrease in ratings in training camps.

This is incorrect. High volatility means that the chance is INCREASED that they'll have a significant ratings change, and maybe even significantly higher odds than low volatility. However, it does not mean that the chance is very great. My observation is that it's more along the lines of...

100 volatility = 2% chance of big change
0 volatility = .5% chance of big change

EDIT: The numbers above aren't exact or the results of any study. The point is this:

1. Volatility makes a difference.
2. The difference between 0 volatility and 100 volatility--while significant when comparing one to the other--isn't enough to make it worth even looking at the number if you know what you're doing otherwise.

[Ben E Lou]

Originally Posted by agrimsley

How would you distinguish which of these rating changes are caused by volatility and which are caused by something else?

That's impossible to do by a study of game-generated players.

What the graphs do show is that 91.4% of all rookies experience a ratings change between camp and EX2.

More accurately, 91.4% of rookies experience a change in the scout's view.

The HUGE majority of them do *not* experience a ratings change.

[agrimsley]

Originally Posted by Ben E Lou

This is incorrect. High volatility means that the chance is INCREASED that they'll have a significant ratings change, and maybe even significantly higher odds than low volatility. However, it does not mean that the chance is very great. My observation is that it's more along the lines of...

100 volatility = 2% chance of big change
0 volatility = .5% chance of big change

Interesting point. How do you come to that conclusion?

If that is indeed the case then how is this not reflected in the numbers? Do you believe that the scouts opinion is masking the changes? Is the sample size too small to highlight the actual trend?

[Ben E Lou]

Originally Posted by agrimsley

Interesting point. How do you come to that conclusion?

Volatility was added to the game 12+ years ago. I've played a lot since then.

If that is indeed the case then how is this not reflected in the numbers?

Because the scout view changes SO much that the volatility-related changes are lost in your numbers.

Jim spoke to some of this years ago: apart from volatility-related random changes and age-related drops, players' actual future potential rating does not change, not even by one point. You should read the Unmasking 101 thread.


(By the way, if you're using the in-game league scouting numbers,--player_ratings_yyyy.csv--it's completely useless. The league scout changes frequently, and is usually very bad at scouting. You might see a guy "change" 20-30 points in future potential in one year simply because the first awful league scout overvalued him and the next one undervalued him.)

[agrimsley]

Originally Posted by Ben E Lou

Volatility was added to the game 12+ years ago. I've played a lot since then.

Because the scout view changes SO much that the volatility-related changes are lost in your numbers.

Jim spoke to some of this years ago: apart from volatility-related random changes and age-related drops, players' actual future potential rating does not change, not even by one point. You should read the Unmasking 101 thread.


(By the way, if you're using the in-game league scouting numbers,--player_ratings_yyyy.csv--it's completely useless. The league scout changes frequently, and is usually very bad at scouting. You might see a guy "change" 20-30 points in future potential in one year simply because the first awful league scout overvalued him and the next one undervalued him.)

I am indeed using the league scout ratings but as this study was only covering the period between the creation of the draft class through to the end of the preseason, the league scout remained constant.

I think the conclusion can still be summarised to something along the lines of "volatility plays such a negligible part in a player's development that it should play a small part in an evaluation of a player's worth".

[Ben E Lou]

Originally Posted by agrimsley

I am indeed using the league scout ratings but as this study was only covering the period between the creation of the draft class through to the end of the preseason, the league scout remained constant.

I think the conclusion can still be summarised to something along the lines of "volatility plays such a negligible part in a player's development that it should play a small part in an evaluation of a player's worth".

Heh. OK. To be honest, I didn't read the details of what you did, because the quoted bit has been fairly well known for years. Combines, bar profile (if you have good scouts), and interview results (if your staff is highly rated at that) will virtually always push one player above the other to the point where volatility is to be ignored.

[QuikSand]

volatility plays such a negligible part in a player's development that it should play a small part in an evaluation of a player's worth

[Ben E Lou]

Dola:

And to be clear, even though the league scout remained constant, he's likely still terrible. Quite often they will miss the mark significantly in one direction pre-camp, then miss it in the other post-camp. (In other words, a guy is really a 45 potential, and pre-camp he's 20/30, then post-camp he's 25/60. Or 20/60-->25/30. It happens in either direction.)

[agrimsley]

Ok then, let me just clear up exactly why I was interested in this. I read somewhere (and I've read a lot lately so can't recall exactly) that volatile players were good to focus on during later rounds of the draft. I wanted to challenge that theory.

[Ben E Lou]

Originally Posted by agrimsley

Ok then, let me just clear up exactly why I was interested in this. I read somewhere (and I've read a lot lately so can't recall exactly) that volatile players were good to focus on during later rounds of the draft. I wanted to challenge that theory.

Got it.

So....here's the unfortunate reality: you'll read some things on these forums-- and with greater frequency, I've observed so far, in the Steam forums--that are stated as fact but that are just plain incorrect. Over time, you'll learn that different users have different thresholds for when they'll state absolutes like redfox and I did in this thread. Speaking for myself, I can assure you that you will virtually never hear the level of absoluteness coming from me in several posts in this thread unless it's based on one or more of the following:

1. I've run controlled tests on it in the past and feel confident that JG hasn't changed it in a recent version/patch.
2. I've picked it up from the thousands of hours I've put into FOF in the last 16+ years.
3. It's directly from the Help File, a post from Jim, or a response from Solecismic Support.

[agrimsley]

Originally Posted by Ben E Lou

Got it.

So....here's the unfortunate reality: you'll read some things on these forums-- and with greater frequency, I've observed so far, in the Steam forums--that are stated as fact but that are just plain incorrect. Over time, you'll learn that different users have different thresholds for when they'll state absolutes like redfox and I did in this thread. Speaking for myself, I can assure you that you will virtually never hear the level of absoluteness coming from me in several posts in this thread unless it's based on one or more of the following:

1. I've run controlled tests on it in the past and feel confident that JG hasn't changed it in a recent version/patch.
2. I've picked it up from the thousands of hours I've put into FOF in the last 16+ years.
3. It's directly from the Help File, a post from Jim, or a response from Solecismic Support.

So to add one final twist to the thread before we put it to bed. I feel like the post I read about looking for volatile players came from Jim himself but therein lies the issue that I cannot find said post or any quote of it.

Unless I find that post again, I will ignore the nagging feeling in the back of my mind.

[Ben E Lou]

Originally Posted by agrimsley

So to add one final twist to the thread before we put it to bed. I feel like the post I read about looking for volatile players came from Jim himself but therein lies the issue that I cannot find said post or any quote of it.

Unless I find that post again, I will ignore the nagging feeling in the back of my mind.

Heh.

I'm pretty certain that he's never said it. Actually, I'm pretty certain that he's repeated more than once his same party line about volatility:

""Volatility simply indicates how likely the player is to boom or bust. That's all there is to it."--Jim Gindin, 11/14/2003"

Now that said, TECHNICALLY, if you aren't very good at drafting and are just stabbing in the dark in the late rounds, sure, go for high volatility. However, you seem like a reasonably smart type. You should be able to draft well enough in a short time to ignore volatility.

[aston217]

I'm quite interested in how you compiled your data into one place here. Pretty slick graphs, my friend!

I realize the numbers Ben threw out there were for illustration purposes only, so this continues only in that vein. Largely you'll easily find other reasons to distinguish between two players.

So for illustration:

SUPPOSING 0.5% annual odds, the 10-year odds that a 0 volatility guy never experiences a big change are 95.1%, whereas

SUPPOSING 2.0% annual odds, the 10-year odds that a 100 volatility guy never experiences a big change are only 81.7%.

So, if you're looking at a franchise QB or something with a 1.1 overall, maybe you feel just a touch less comfortable if you think there could be a 20% chance something awful happens to him sometime in his useful career, vs maybe another guy with only a 5% chance of the same. I mean, if he looks great, the odds are pretty in your favor either way, right? But you know, gotta take the plunge. And anyway, the assumption that volatility offers constant odds every year of a player's career may be an inaccurate one, so perhaps it's never as drastic a difference as even this calculation suggests.

In late rounds, I don't think it's too sensible to search for high vol guys personally. You aren't keeping these guys around for 10 years. I suppose it can be one consideration, out of many.

For what it's worth I do recall reading that Jim himself saying something like that about the late rounds. But I don't really agree, and maybe it's something more like Ben's "if you're stabbing in the dark, sure, go for it" that's been misinterpreted over the years.
.