When will Madden get the ratings right?

[KBLover]

Quote: Originally Posted by Big FN Deal

http://www.operationsports.com/forum...hand-hand.html

Wow.

I missed this thread so I'm glad you linked it here.

Great breakdown and explanation of the general points.

[Big FN Deal]

Quote: Originally Posted by KBLover

Wow. I missed this thread so I'm glad you linked it here. Great breakdown and explanation of the general points.

Yeah man, I bookmarked it when GMS first posted it and just makes me shake my head at the notion that Tiburon "reads these boards". I have asked before, "what threads exactly?" because they seem oblivious to the quality in-depth things that people post and discuss on OS. Yet a "purple name" will comment on some lesser thread like virtual Twitter additions, articles about Tebow at TE or engage the community about meaningless votes for ratings bumps.

The precision modifier seems like a step towards animation based ratings differential but who the hell knows if that's the case or not. Maybe I pay too much attention but I still remember when Tiburon acted as if this article was something that didn't cross their minds. http://www.operationsports.com/forum...-controls.html

I hope that was them just being coy though and they actually have some vision for building on this.

[splff3000]

Am I the only one that thinks that production AND scouting data should be used to determine ratings? Watching DCEBB and Kushmir go back and forth, I was waiting for someone to say, let's combine the 2 and make one great rating system. For the most part, Madden tried to do this in CC last year. One of the best things that Madden did for 13 was add production into the formula used to figure out a player's overall rating. It, among other ratings, is probably not weighted correctly, but at least it's there and that's a start.

You have to use both. Like DCEBB was saying, you could have a WR that all of the scouts are raving about, but if he doesn't have the production numbers in the NFL, you can't rightfully give him a 90 rating. Joe Blow, the WR, might have the eye of every scout in the league, but until he actually does something, he's just Joe Blow, the WR, and nothing else. This could easily be depicted in the game with good "physical" ratings, but low "production" ratings, so if his physical ratings averaged out to a 90 and his production rating came in at about 60, he would average out to about a 75 overall. That's a fair rating for someone that scouts think has great talent, but hasn't done anything with it.

A real life example of someone like this is Charlie Whitehurst. There was so much buzz about him that the Chargers received a 3rd round pick from Seattle and he received a 2yr 8mil contract after having done pretty much nothing. The Seahawks said they had picked up a potential franchise QB with the move. This is the scouting department making moves based on scouting data and not production. As everyone knows, he still hasn't done much of anything and this is why you need both production and scouting data for your ratings. If your system is based solely on scouting data, Charlie Whitehurst is probably a high 80's or low 90's overall QB. If it's based solely on production, he's probably a low 70's or high 60's guy at best. You need both to acurately rate the player.

Note: I know there is the "intangible" rating in CC as well, but I left that out of my example to prove how you need both production and scouting data since that is what the back and forth was about. Besides, I have no idea in hell how EA determines their "intangible" ratings.

I think the problem with ratings in Madden is the scouting data part. It seems like there is none for a lot of players and players are grossly overrated in that area at times so that leads to inflated overalls. Another problem is the formula that is used to determine the overalls. You have a formula where "awareness" is weighted heavily for a defensive player even though, according to EA, "play recognition" is more important in actual gameplay. Route running and catching are not weighted enough for WR's and speed is rated too heavily for just about all positions. I could go on for days about how the formula for the overall doesn't seem to be weighted correctly. I know there are some that say EA should just get rid of overall ratings. The "overall" rating itself isn't the problem tho. If it was weighted properly, it would give you a good idea of how good a player is at a glance. That's the problem, the weight of each attribute in the formula to determine the overall rating, not the overall rating itself.

In conclusion, until we stop calling Donny Moore the ratings czar, the ratings are gonna be messed up in Madden. I follow him on twitter and it's painful to see his tweets during football season. It seems like after every fumble, TD, or interception, someone is getting ratings changed. Someone had tracked the ratings update changes for each week during the season and there was an amazing number of ratings increases/decreases that got changed right back in the next update. That's the next week!! So to answer the OP's question, as long as they use ratings as marketing and things like that, you can rest assured they will never get the ratings right.

[Big FN Deal]

I get what you are saying Spliff but off the top of my head, I don't think production should have any effect on MOST ratings. I think the exception is AWR, which doesn't directly represent a skill/talent but the others, nah. I think production should effect DPP, contract demands, team financial interest, etc but not ratings. A key problem in Madden is that it's so easy to produce with lesser ratings except speed, it doesn't make much difference anyway and also the obvious issue with the way player ratings are determined in the first place.

Also we need to be clear what production we are referring to because I am mostly referring to in-game production, not real life production. An issue that seems to happen frequently is that some people think that because player production in real life equates to a ratings change in Madden, that player production in Madden Franchise should equate to a ratings change too. I'm fine with ratings changes in Madden based on real life production over a period of time, say ratings adjustments every 4-8 weeks/games but not every week or in-game production equating to ratings.

I hoped that DPP would be the answer to weekly ratings adjustments and Moore would adjust that for players weekly instead of base ratings but that didn't happen.

[DCEBB2001]

Quote: Originally Posted by Big FN Deal

I post less and less about Madden because it's frustrating to see potentially solid ideas that could improve the game, go repeatedly unnoticed by Tiburon. However after reading over the discussion ITT I decided to throw in my two cents. I will admit I am a fan of DCEBB's data based approach to ratings, so that makes my perspective bias. I try to see things from the POV of others because I understand that I don't know everything, nor am I always right. That said, I fail to see how anyone could turn what was a discussion about the best real world data to use to standardize ratings in Madden, scouting, production, some mixture, etc, into some off the cuff litmus test for DCEBB to use scouting data to predict real world player performance. WTF? lol I will try to sum up my opinion on this as short as possible because it won't matter for Madden much anyway until/if Tiburon actually does something. DCEBB has repeatedly stated that that scouts are accessing talent/skill, not production and that's exactly what player ratings are supposed represent, talent/skill. Just because a player has been scouted to and actually possesses great hands/catch ability, does NOT mean they will be productive with that skill/talent in the NFL. If that player has demonstrated great catch ability from HS, to college, in practice, combine workouts, just goofing around, etc, there is no dispute that they have the talent/skill to catch but production is about how they utilize that talent/skill within other given factors. In a nutshell, each player rating in Madden should be a representation of each players skill/talent of something but production/stats are a byproduct of utilizing those talents/skills together with other factors like system, environment stress, etc. So just because Terrell Owens has x amount of drops of easily catchable passes during the season, it should NOT equate to a knee jerk drop in his CTH rating in Madden, instead that should be a factor on his concentration, ie AWR and relevant DPP traits. In Madden, production/stats should effect contract demands and team financial interests like jersey sales and positive media attention, not ratings. Bill Belicheat scouted Wes Welker for his talent/skill, not his production at Miami, whereas Larry Brown landed a big payday with the Raiders essentially for timely ints in the Super Bowl and good production on a historically talented Cowboys team. I'll stop there because in lies a fundamental flaw in the way ratings are done in Madden, individual ratings don't correlate to varying skill/talent or at least not using a consistent tiered based system, so that's why ratings are a hot mess and likely why some have difficulty embracing what DCEBB is saying. Although DCEBB's approach would be an improvement for Madden ratings currently because it would at least standardize them, ultimately here is a good breakdown of the fundamental issue with ratings: http://www.operationsports.com/forum...hand-hand.html

Thanks for that link. That was an EPIC post by SKILLZ. Due to the fact that none of us here are programmers for EA/Tiburon, we can only control so much...which is terrible. My small slice of this pie was to simply look at some relative data that I trust and see how it could be used in Madden.

I have so much information from an actual scouting department for an actual NFL team, that I didn't even know where to begin. They had guys on here who dropped out of high school football back in 2001 on there, but I digress...

All I am trying to do is take their evaluation and turn it into Madden ratings. That's it. I do not control the data, the scout's opinions, etc. I just take the data and interpolate it based on the distribution of the data provided. If I was working for EA, I would be doing a LOT more, and would probably alienate my girlfriend even further than she already is during the season when updating the site becomes a part-time job with little pay.

However, since I do not work for EA and likely squandered any opportunity for their employ as I held to my integrity for realism over simply getting a job where you are ordered to rate players according to the demands of a public relations department, I can only do so much.

Using the primary source material of actual scouting information seems to make the most sense to me. Why? Because they have already done the hard work by turning what they see (qualitative) into numbers (quantitative). All I have to do is filter it and introduce it into the "Madden system" and PRESTO, you have players rated as if an NFL scouting department had rated them. I love my Packers and as much as I would love to see them all rated well in the scouts eyes, they are not all rated 90+. I have kept an unbiased approach to this because I want to see what the scouts think.

Want to know a few things I have found from this data? Here's some conclusions I have drawn from working with this stuff for nearly 4 years now:

1. The upper level of talent in the NFL is EXTREMELY RARE. Including street free agents, college kids, high school kids, and professionals (NFL/CFL/Arena/IFL/etc) the list is currently at 57851 and growing for players that this team's department has data on. 57851! That is the money pit for football data, which is why I had to sign several NDAs to ensure that my contacts would not let this stuff out into the public domain (I manage to get around this with the website by turning those ratings into Madden ratings, but because that is not PRIMARY source material at that point, I have the green light to publish so long as I do not release the primary source material).

All players are rated on a scale from 0 to 1000. Some players actually get over 1000 and some get under 0 due to injury or rare playing level (like a HOF season...Tomlinson 2006? and Brady 2007?) The current scale including all of these athletes is 985 to -198. In this scale, you can take an unbiased approach by using equal intervals. That means that each point is worth the same. When you do this, you calibrate 985 to your maximum and -198 to your minimum. The result is that the distance between each point is the same. So the difference between a player rated 999 and 998 is the same as the amount of difference between a player rated 1 and 0.

When you do this, you truly find how rare the best talent in the NFL is. Only 8 players have over 900 points. 8! In all of football (remember, this includes everyone that this department has even looked at in over the last decade). Meanwhile there are only 18 over 800 points. 85 are over 700 points, 414 are over 600 points, 1413 are over 500 points, 4054 are over 400 points, 16188 are over 300 points, 36085 are over 200 points, 53721 are over 100 points, and 57848 are 0 and above. Only 3 are below 0 (due to career injuries).

Here is a histogram showing this distribution with a rational function applied (in red). The "dip" in the data circa the 200 point marker is due to the gap between high school and college-level athletes. It is statistically significant as you can see. The point of inflection is at 251 points. That means that the number of players at 251 points is most according to the function, not the actual data. The mode for the data is 294 points.



This graph illustrates how rare NFL talent is. If you were to exclude all high school and college kids and only use professional players, the histogram would look like this (below). Here the maximum for the function is at 345 points. The mode for the data is 386.




This data (according to this scouting department), tells us that your run-of-the-mill player even worth a look is around 251 points, while the average professional player even worth a scouting report is around 345 points. Since the NFL is always evaluating talent and new talent is always being brought in (mostly through the NCAA and college football), using the first number as the "base" for players even getting a look makes sense. That way every player no matter who they are or where they play is on the same scale and you don't have to re-scale should you want to rate college players someday (Yes, this is being considered with FBG. I get about a dozen emails every month asking for college teams to be rated too.)

I still prefer the unbiased approach of using equidistant values (1 point being worth the same no matter where it falls on the scale) because it is unbiased and not skewed (not altered around a mean). The trick is to use this to rate players in Madden. If every 10 points was one point in Madden/NCAA Football, you would have 8 90s, 10 80s, 67 70s, 329 60s, 999 50s, 2641 40s, 12133 30s, 19897 20s, 17637 in the teens, and 4130 rated below a 10 OVR.

If Madden players were rated that way, there would be outrage! Only 18 players rated over 80+? 85 over 70+? People's heads would explode if that were the case. However, that is what the data says, right? Well the other thing you must consider is that the scouts' formula for the "OVR" is different from that of Madden. Well, what happens if you scale all of the attributes like you do in Madden (the individual attributes: THP, SAC, ELU, etc) and only rate the players who are professionals. Well then you throw those into Madden, you still have to use Madden's OVR calculator, which will give you only one player rated over 90 (Revis, amazingly at 91), and an average player rating of 52. Only 1 90!? More outrage!!!

So when we use the scouting data to rate the OVR, people will not be happy if we were to do it without bias and scaling. If we just rate players based on their attributes (and choose to simply ignore the OVR) we will get a better picture of how players should be rated. But for the people who only look at the OVR, they will be outraged as well.

So there really is not a win-win with this data, either. The fact that Madden's system of calculating the OVR rating is so messed up (really?...the CTH attribute doesn't affect a CBs OVR rating? Please...), only makes things worse. You can either alter the distribution for the OVR, alter the attributes to make them fit the overall, or do nothing, which I suppose makes this conversation moot anyhow.

The point is that no data source is going to be perfect, especially when you are working within a framework that lacks common, logical, sense.

[DCEBB2001]

Quote: Originally Posted by splff3000

Am I the only one that thinks that production AND scouting data should be used to determine ratings? Watching DCEBB and Kushmir go back and forth, I was waiting for someone to say, let's combine the 2 and make one great rating system. For the most part, Madden tried to do this in CC last year. One of the best things that Madden did for 13 was add production into the formula used to figure out a player's overall rating. It, among other ratings, is probably not weighted correctly, but at least it's there and that's a start. You have to use both. Like DCEBB was saying, you could have a WR that all of the scouts are raving about, but if he doesn't have the production numbers in the NFL, you can't rightfully give him a 90 rating. Joe Blow, the WR, might have the eye of every scout in the league, but until he actually does something, he's just Joe Blow, the WR, and nothing else. This could easily be depicted in the game with good "physical" ratings, but low "production" ratings, so if his physical ratings averaged out to a 90 and his production rating came in at about 60, he would average out to about a 75 overall. That's a fair rating for someone that scouts think has great talent, but hasn't done anything with it. A real life example of someone like this is Charlie Whitehurst. There was so much buzz about him that the Chargers received a 3rd round pick from Seattle and he received a 2yr 8mil contract after having done pretty much nothing. The Seahawks said they had picked up a potential franchise QB with the move. This is the scouting department making moves based on scouting data and not production. As everyone knows, he still hasn't done much of anything and this is why you need both production and scouting data for your ratings. If your system is based solely on scouting data, Charlie Whitehurst is probably a high 80's or low 90's overall QB. If it's based solely on production, he's probably a low 70's or high 60's guy at best. You need both to acurately rate the player. Note: I know there is the "intangible" rating in CC as well, but I left that out of my example to prove how you need both production and scouting data since that is what the back and forth was about. Besides, I have no idea in hell how EA determines their "intangible" ratings. I think the problem with ratings in Madden is the scouting data part. It seems like there is none for a lot of players and players are grossly overrated in that area at times so that leads to inflated overalls. Another problem is the formula that is used to determine the overalls. You have a formula where "awareness" is weighted heavily for a defensive player even though, according to EA, "play recognition" is more important in actual gameplay. Route running and catching are not weighted enough for WR's and speed is rated too heavily for just about all positions. I could go on for days about how the formula for the overall doesn't seem to be weighted correctly. I know there are some that say EA should just get rid of overall ratings. The "overall" rating itself isn't the problem tho. If it was weighted properly, it would give you a good idea of how good a player is at a glance. That's the problem, the weight of each attribute in the formula to determine the overall rating, not the overall rating itself. In conclusion, until we stop calling Donny Moore the ratings czar, the ratings are gonna be messed up in Madden. I follow him on twitter and it's painful to see his tweets during football season. It seems like after every fumble, TD, or interception, someone is getting ratings changed. Someone had tracked the ratings update changes for each week during the season and there was an amazing number of ratings increases/decreases that got changed right back in the next update. That's the next week!! So to answer the OP's question, as long as they use ratings as marketing and things like that, you can rest assured they will never get the ratings right.

For the record the current rating I have on Whitehurst is 528 points...same number as Rahim Moore, Pat Sims, Vonnie Holliday, and LaMichael James among others. That's about midway on the scale of 985-(-)198. Puts him at the 990th best player overall according to the scouts.

Ultimately, there needs to be a standardized formula for the scouting data. I'm so tired of seeing speed points added to players who didn't get literally get faster...just had a good game. That stuff drives me up the wall as it makes no logical sense.

[KBLover]

Quote: Originally Posted by splff3000

Am I the only one that thinks that production AND scouting data should be used to determine ratings? Watching DCEBB and Kushmir go back and forth, I was waiting for someone to say, let's combine the 2 and make one great rating system. For the most part, Madden tried to do this in CC last year. One of the best things that Madden did for 13 was add production into the formula used to figure out a player's overall rating. It, among other ratings, is probably not weighted correctly, but at least it's there and that's a start.

The problem is that production being used goes back to the "he had a great game so he must have gotten better" issue.

Production is a "place to look" as production changes can be flags (both pos. and neg.) to changes in skill/mechanics/mindset/health, etc.

But those things actually change first, then the production tends to follow. To use a baseball example, a guy doesn't stop hitting 40 HR a year, THEN lose power - the power/bat speed/approach drop, THEN the HR fall off. It's just that we humans tending to be backward-looking, pattern-seeking critters need the result to clue us to the cause, but the reality is often the exact opposite. When building a model (which is, ultimately, what a video game is if it calls itself realistic), you can build in simulated changes as it goes, leading to the result.

Quote: Originally Posted by splff3000

You have to use both. Like DCEBB was saying, you could have a WR that all of the scouts are raving about, but if he doesn't have the production numbers in the NFL, you can't rightfully give him a 90 rating. Joe Blow, the WR, might have the eye of every scout in the league, but until he actually does something, he's just Joe Blow, the WR, and nothing else. This could easily be depicted in the game with good "physical" ratings, but low "production" ratings, so if his physical ratings averaged out to a 90 and his production rating came in at about 60, he would average out to about a 75 overall. That's a fair rating for someone that scouts think has great talent, but hasn't done anything with it.

Thing is, if the guy ain't producing, he probably either:

-doesn't have the skills and is overhyped and the scouts are wrong (which can't happen in Madden because there's no scouts and emulation of human scouting error/bias/drinking the kool-aid or whatever else)

-in the wrong scheme/system

-and/or playing hurt/distracted

If it's the case of scenarios 2 and 3 - then his ability should not be touched. His production is not down because of his ability, but other issues. Production score should NOT be included in OVR, imo, but a SEPARATE grade altogether. That would provide the most information.

If you see a guy who's skills are top 10 in the league, but he's got a 30/100 production grade, that could tip you off if you were thinking of trading for his guy or signing him. Do you take the risk? Do you try to find out why he's underachieving so badly? Was he hurt all last year? Etc.

Also, the production rating itself has to be logical. Kendle Maeweather has done nothing BUT produce since he hit the field. He got over 180 tackles, 10 sacks, 2 defensive TDs, a pick, forced fumbles, won Defensive MVP, won Defensive ROY...and has a garbage PRO rating. WHY? The man has literally never had a bad game. (Which is another thing, production doesn't always equal tackles, but doing your job on the field, but Madden can't judge that either - being a successful force player on an outside run is doing your job, even if you don't get the tackle. Pushing the pocket in the face of the QB is doing your job even if you don't get the sack, etc.)

[KBLover]

Quote: Originally Posted by Big FN Deal

I hoped that DPP would be the answer to weekly ratings adjustments and Moore would adjust that for players weekly instead of base ratings but that didn't happen.

Which is a damn shame.

DPP has the potential to be a very robust system capturing a lot of stuff we're discussing here about the number ratings and what they can/can't/don't/won't capture as well as introducing more realistic vagaries in performance to truly give a bad year/good year feel to guys.

*sigh*