QB Sim Stats: Only AWR Matters

[BaylorBearBryant]

Quote: Originally Posted by Palo20

I agree, there's no question that AWR is extremely important for QBs. Being below an arbitrary AWR number (81) shouldn't be the difference between a Heisman trophy winner and borderline incompetence.

Yes, I would agree with that.

[Pogo27]

Quote: Originally Posted by Palo20

My premise isn't that there is a linear progression for success based on AWR. I'm saying that 81 AWR is the line right now. Anyone at 81+ will have success, and anyone below that, it will basically be a crapshoot. Last year in the NCAA there were 43 QBs at 60% or better. Out of the top 100 QBs in Yards/Game, not one of them was below 50%. Now, as for how much data we have so far. I've simmed 2010 twice and 2011 once. In those 3 years, there have been 53 total players who have had an AWR at 81+. There were 19 players for both 2010s and 15 players for 2011. During every sim, the ONLY players who have had a comp% over 60% are these 53 players. Every other player with an AWR under 81 also had a comp% under 60%. So, just using the starters, there are 120 starting QBs per season, so over 3 seasons, that's 360 guys. This may not be pure science, but this is enough data for me. There has not been one QB in 3 years of simming to fail at this rule. 81 AWR is the number. QBs Tested AWR 81+ AWR <81 Comp% >60% Comp%<60% 320 53 267 53 267 These numbers do not include Martez Wilson who was >60% twice. It also doesn't include backup QBs who also have fallen into the same pattern. In EVERY season I've simmed where Texas Tech's Taylor Potts and Steven Sheffield both play, Potts is >60% and Sheffield is <60%. My premise is that 81 AWR is the minimum threshold.

Okay, so when I called you out after just your original post for making your assumption based on not enough data, and you said you'd been doing lots of simming, that was a lie, right? Because in this post, you said you simmed 2010 twice and 2011 once, and at least one of those three sims came after I called you out (and I'd venture to guess two of them).

And three sims isn't 360 samples just because there are 120 different started quarterbacks. Three sims is 3 samples.


Additionally, I think the point I've been trying to make from the very beginning is absolutely right: AWR isn't the only thing that matters. Sure, it is an interesting phenomenon that through 3 samples, all the players with 81+ awareness have 60% completion percentage or better, and everyone with 80 AWR or lower have less than 60% completions. But that's all that you think you've proven (which, 3 samples doesn't prove anything). It certainly isn't the type of data that says "Only AWR Matters."



Another test that could be done... make a handful of players...

Edit all of one team's quarterbacks to 50 arm strength and accuracy, 99 awareness (make sure they're still the highest rated QB on team by making sure to lower everyone else's THP/THA to 40) and see if they end up with 60%+ on a regular basis. Do the same with a team who has 81 AWR to see if your theory is right on that being some sort of magic number.


And then edit a team to 99 THP/THA QBs with 80 and lower AWR, but actually try one at 80, try one at 75, try one at 50, etc.

[youALREADYknow]

Quote: Originally Posted by Pogo27

And three sims isn't 360 samples just because there are 120 different started quarterbacks. Three sims is 3 samples.

If you're going to bust his balls about sample size, then at least acknowledge that the correlation factor is spot on to back up his claim. I can't see that correlation going down too much given a larger sample size.

Here's an idea: YOU do the tests which you're suggesting since you obviously don't respect the time the OP has put into this.

[sallingm]

You guys are making this WAYYYY to complicated. Just run 5 simulations of season 1 (2010 season) w/ a team, Ohio State for example. Take the average of Pryor's stats. Then edit the rosters and change Pryors AWR rating, and again simulate 5 years worth of season 1. Compare the stats. If you run at least 5 simulations, you should get a decent average that will take into account any issues w/ injuries to the rest of the team.

[Palo20]

Quote: Originally Posted by Pogo27

Okay, so when I called you out after just your original post for making your assumption based on not enough data, and you said you'd been doing lots of simming, that was a lie, right? Because in this post, you said you simmed 2010 twice and 2011 once, and at least one of those three sims came after I called you out (and I'd venture to guess two of them). And three sims isn't 360 samples just because there are 120 different started quarterbacks. Three sims is 3 samples. Additionally, I think the point I've been trying to make from the very beginning is absolutely right: AWR isn't the only thing that matters. Sure, it is an interesting phenomenon that through 3 samples, all the players with 81+ awareness have 60% completion percentage or better, and everyone with 80 AWR or lower have less than 60% completions. But that's all that you think you've proven (which, 3 samples doesn't prove anything). It certainly isn't the type of data that says "Only AWR Matters." Another test that could be done... make a handful of players... Edit all of one team's quarterbacks to 50 arm strength and accuracy, 99 awareness (make sure they're still the highest rated QB on team by making sure to lower everyone else's THP/THA to 40) and see if they end up with 60%+ on a regular basis. Do the same with a team who has 81 AWR to see if your theory is right on that being some sort of magic number. And then edit a team to 99 THP/THA QBs with 80 and lower AWR, but actually try one at 80, try one at 75, try one at 50, etc.

3 sims is 360 QBs that have been tested (actually more with injuries factored in). Out of those 360+, not one of them has been an outlier when it comes to 81 AWR being the key number.

I've simmed a ton of seasons since purchasing the game, I've only recorded the data for a few seasons. The first times I simmed, I knew something was wrong, I just didn't quantify it. That lead me to look at it further.

So I'm sticking by my conclusion: If you want a QB to have better than average real life QB stats, their AWR needs to be 81 or higher. For over 360 QBs that I've recorded, this has been the case. That's enough evidence for me.

[12]

Quote: Originally Posted by Palo20

3 sims is 360 QBs that have been tested (actually more with injuries factored in). Out of those 360+, not one of them has been an outlier when it comes to 81 AWR being the key number. I've simmed a ton of seasons since purchasing the game, I've only recorded the data for a few seasons. The first times I simmed, I knew something was wrong, I just didn't quantify it. That lead me to look at it further. So I'm sticking by my conclusion: If you want a QB to have better than average real life QB stats, their AWR needs to be 81 or higher. For over 360 QBs that I've recorded, this has been the case. That's enough evidence for me.

I'm with you. Thanks for the research. In my season, all the QB's with an AWR of under 80 are having horrible completion rates. John Brantley is only completing 37% of his passes. No QB's with an AWR under 81 are completing even 50 percent. I've only played three games but your data shows that this will continue over the course of the season.

[Pogo27]

Quote: Originally Posted by Pogo27

Okay, so when I called you out after just your original post for making your assumption based on not enough data, and you said you'd been doing lots of simming, that was a lie, right? Because in this post, you said you simmed 2010 twice and 2011 once, and at least one of those three sims came after I called you out (and I'd venture to guess two of them). And three sims isn't 360 samples just because there are 120 different started quarterbacks. Three sims is 3 samples. Additionally, I think the point I've been trying to make from the very beginning is absolutely right: AWR isn't the only thing that matters. Sure, it is an interesting phenomenon that through 3 samples, all the players with 81+ awareness have 60% completion percentage or better, and everyone with 80 AWR or lower have less than 60% completions. But that's all that you think you've proven (which, 3 samples doesn't prove anything). It certainly isn't the type of data that says "Only AWR Matters." Another test that could be done... make a handful of players... Edit all of one team's quarterbacks to 50 arm strength and accuracy, 99 awareness (make sure they're still the highest rated QB on team by making sure to lower everyone else's THP/THA to 40) and see if they end up with 60%+ on a regular basis. Do the same with a team who has 81 AWR to see if your theory is right on that being some sort of magic number. And then edit a team to 99 THP/THA QBs with 80 and lower AWR, but actually try one at 80, try one at 75, try one at 50, etc.

Quote: Originally Posted by youALREADYknow

If you're going to bust his balls about sample size, then at least acknowledge that the correlation factor is spot on to back up his claim. I can't see that correlation going down too much given a larger sample size. Here's an idea: YOU do the tests which you're suggesting since you obviously don't respect the time the OP has put into this.

Thanks for reading.



I have a feeling there are a lot of people on this board that struggled in science class when it came to the scientific method and actually performing your own experiments.

I'm not trying to pick on the original poster, but there are a LOT of threads the crop up here that are just bad science.

The Scientific Method

1. Define the question - In this case, "Is awareness the only factor that matters when it comes to QB sim stats?" But I think as the thread has progress, the more accurate question needs to be stated as "Does 81+ awareness directly correlate to 60%+ completion percentage?"

2. Gather information and resources (observe) - In this case, the observation should be the simulation results. But doing three simulations and the same holding true is the equivalent of being dealt two-pair in a game of 5-card draw three times in a row and making the assumption that you will always be dealt two-pair.

3. Form hypothesis - In this case, the hypothesis was stated in the subject title: QB sim stats are determined only by awareness. However, as with the question, the original post isn't quite accurate to where the thread has progress. The hypothesis should be "Quarterbacks with at least 81 awareness will always throw for at least 60 completion percentage, while quarterbacks with 80 and below will never throw for 60 completion percentage or better."

4. Perform experiment and analyze data - To this point, that really hasn't been done. There have been suggestions in this thread as to how to do it. Hundreds of simmed seasons would serve as a possible experiment, to see whether the default quarterbacks always hold true to the rule. Also, creating non-default quarterbacks using constants and variables to determine whether awareness TRULY is the only factor would be a great experiment.

5. Analyze Data - This basically means assort it into graphs, which has been done, quite nicely I might add, though it would be slightly nicer to see it sorted by a player's AWR or sorted by Completion percentage, as these are the two main factors we are looking at.

6. Interpret Data and draw conclusions - We've done some of this, on the little amount of data we have. But again, your original hypothesis, that only awareness matters, doesn't match the data. The hypothesis that you came to later about 81 AWR and 60% accuracy does seem to hold true to the little amount of data that we have, but again, no REAL experiment has been done to prove this.

7. Publish results - This has been done, essentially. Sure, it's not a formal publication, but you have put the results on the forum. This step is important because an important part of the scientific method is letting others see the testing methods and the results gathered for the express purpose of looking for potential flaws in order to better the conclusion.

8. Retest - If it is a hard and fast rule, no amount of testing could ever prove it wrong.





Anyway, back to the poker hand example which I'm sure you skimmed over, despite me organizing this long-ish post into easy, numbered paragraphs...

I mentioned your results in comparison to being dealt two-pair in a game of five card draw. Now let's say that not only was I dealt two pair, but the other five players at the table were dealt nothing better than a pair of fives. And this happened three hands in a row. Now this is more equivalent to your test. Granted, for this to happen in poker is extremely rare, but it's not impossible. Yet if you were observing this, it would make you think that it's a hard and fast rule that player A gets two pair, while players B, C, D, E, F all get nothing better than a pair of fives.

But we're all intelligent. We all know that it's extremely unlikely that player A will get dealt that two-pair in the forth hand. And even if he does, over the course of thousands of hands, we know that these first three hands in which we saw player A get two-pair will turn into a minority of his hands overall. Likewise, the scenarios in which five players all hold nothing better than a pair of fives will also turn into a minority.


And I'm not trying to say that this is exactly like simmed quarterback stats. Obviously, poker is a matter of purely luck and the cards are dealt randomly to random players, whereas in a football video game, the virtual players do have skill attributes which have some effect on how their final stats turn up.

But if we look at all the results, we see examples of QBs with 88 AWR (7 above your threshold) complete just 61% (just 1 above your threshold), and QBs with 80 AWR (below your threshold) complete 59%. That's QBs with a difference of 8 awareness points only differentiated by 2% completion percentage.

And I will still point out GJ Kinne, who despite only 67 awareness is completing around 50% in the sims you've put results up for. He's right up there with players with significantly higher awareness ratings.

Moreover, all the guys who do hit your over 81 AWR and over 60% rule are relatively scattered.


With enough sims, you might be able to prove to me that 81+ AWR = 60% + completion percentage, but your results don't even begin to approach proof that awareness is the only skill that matters in quarterback sim stats.

[Palo20]

While I review the Scientifc Process, someone needs to rename the thread:

"Only QBs with 81+ AWR will complete >=60% Passes"


I don't really care about the scientifc process, I'm not a scientist.

But, only AWR matters. The part of AWR that matters: 81.

GJ Kinne is not a good example for you. Once again, I never said it was linear, just either/or. Kinne is at 67, therefore has no chance of completing a high % of passes.

Back to my science homework.....