Donny Moore, the 'Madden Ratings Czar', Leaving EA

[briz1046]

Quote: Originally Posted by DCEBB2001

All you need to do is read their FAQ page. Lot's of stuff that should raise red flags in there. Here are some quotes: "All of the data is collected in order to build a detailed picture of each player’s performance and production over the course of a season." "We offer a different type of scouting, strictly based on performance and not technique or upside" "We are looking for the result of that poor technique, not the poor technique itself. If poor technique results in a positive play, that is graded at the same level as good technique yielding a positive play." "Essentially, we’ve created a new type of scouting that strictly looks at performance, not necessarily the process that gets there. In our dealings with NFL clients, we’ve referred to this as supplying the “what” as they supply the “why.” We can tell a team that an offensive tackle gives up an inordinate amount of bullrush pressure and they can determine if it’s a lack of technique, functional strength, or perhaps a combination of the two." To me, PFF doesn't tell you how good a player is. Instead it tells us how well a player produces. There is a big difference between the two. They only look at the result of the play. Anytime someone attempts to justify rating players in Madden with PFF stats, I just roll my eyes and ignore it.

Whilst I would agree that certain aspects of how PFF rates players can be worrisome to discount it's value entirely would be to throw out the baby with the bath water
It's oviously production based but in real life you rarely achieve consistent success in anything football included without the necessary talent. Over a short period you may get by with a combination of luck and/or help but in time the cream mostly rises to the top
Problems with PFF grades most often arise when sample sizes are small as they do not take any account of such factors as strength of opposition, minor injury etc which will affect smaller samples
Pure scouting data itself is far from infallible, it is after all at its heart opinion based , the opinions of experienced qualified professionals but still opinion and thus subjective
With all due respect to dans sources I'm sure you could find other equally qualified individuals who would produce markedly different grades
After all professional scouts are largely responsible for providing the data teams use in the draft and free agency where busts are frequent and potential stars go overlooked every year
Even the hard data such a 40 and split times and drills from the combine and pro days can be misleading . If you have ever followed the times run by sprinters over a year or career you would see peaks troughs trends as they have good/ bad days and the such as well as a career trajectory which often peaks in their late 20s . Basing a players speed /acceleration on a few runs in his early 20s is far from infallible
To sum up I'm sure a more methodical and spread out approach would be a benefit but let's have no illusions that this is more accurate just that's it's more likely to be so

[charter04]

Quote: Originally Posted by khaliib

What gets lost in the ratings discussion is "Animations" and that's the core of the systems problem no matter what data method is used. For those ratings that are needed to trigger certain animations during gameplay, what is the best method/scale to be used as the mechanism to determine the triggering points? If there are only "5" tackle animations the game will utilize, then any system that is used that incorporates data that goes beyond 5 is pointless because data after 5 isn't used. Thus, is the issue with utilizing a 100 based scale for a particular rating, when there aren't 100 animations to be utilized for that rating. Or any scale that's not in-line with number of possible triggering animations. They should just render a particular rating according to the number of available animations to be the used. As developement injects more animations for a particular rating, the scale should increase. 1) if there's currently 20 catch animations, then that's the scale for that rating. 2) if there's 5 tackle breaking animations, then that should be the scale. We don't need to know the formula that dictates the win/loss of a triggered animation, just that said RB has/utilizes 3 tackle breaking animations out of 5, that he uses. - not every back stiff arms, jukes, spin etc... This is mainly dealing with those interaction ratings. Of course ratings such Throw Power/Accuracy would require more under-the-hood structure even though they're animations also. Not that this is the end all, be all answer to the issue, but it would provide a clean and applicable foundation that can be utilized within every mode of play or play style. Just thinking out loud!!!

Do you still have one of those rosters you did using the editor for NCAA FB 14?


Sent from my iPhone using Tapatalk

[DCEBB2001]

Quote: Originally Posted by ggsimmonds

We would have to agree to disagree then. In most things, this included, limiting or restricting yourself to one thing is unwise. One should always consider all the evidence, utilize all resources, and consider different views or angles. PFF's guys are not scouts no, nor do they claim to be. We both agree, and PFF acknowledges, that they are not in the business of scouting. So why discredit them because they are not professional scouts? Why would they be?

I have considered using PFF in the past and found, in my opinion, that the data they provide is less reliable than other sources. Like I said before, they would be good for tracking tendencies, but I do not believe they would be good for assigning values to the actual ratings.

My reasoning for discrediting them is because actual scouts are trained observers. Viewing and recording is what they are paid to do. Like police testimony, their opinions hold a more weight. If I have 6 police officers tell me that they saw two planes fly into WTC 1 and 2 on 9/11/2001 and three hot dog stand operators say that they saw giant pterodactyls hit those buildings instead, I would be more apt to believe the trained observers in their version of the events.

In rating these players, if you really want validity and accuracy, you have to go with sources you trust. I use scouting data at FBG because I have been on that side of the business before and I believe that it offers a certain level of reliability. You could just as well start your own ratings site using PFF as your source material and you may reach different conclusions (ratings) or you may reach the same ones. What matters is that as the author you trust the data you are using. So long as you use data that you believe is valid, you really can't go wrong. In this case, I am highly skeptical of the stuff PFF publishes because of things like the Brad Jones = Secret Superstar article.

[DCEBB2001]

Quote: Originally Posted by briz1046

Whilst I would agree that certain aspects of how PFF rates players can be worrisome to discount it's value entirely would be to throw out the baby with the bath water It's oviously production based but in real life you rarely achieve consistent success in anything football included without the necessary talent. Over a short period you may get by with a combination of luck and/or help but in time the cream mostly rises to the top Problems with PFF grades most often arise when sample sizes are small as they do not take any account of such factors as strength of opposition, minor injury etc which will affect smaller samples Pure scouting data itself is far from infallible, it is after all at its heart opinion based , the opinions of experienced qualified professionals but still opinion and thus subjective With all due respect to dans sources I'm sure you could find other equally qualified individuals who would produce markedly different grades After all professional scouts are largely responsible for providing the data teams use in the draft and free agency where busts are frequent and potential stars go overlooked every year Even the hard data such a 40 and split times and drills from the combine and pro days can be misleading . If you have ever followed the times run by sprinters over a year or career you would see peaks troughs trends as they have good/ bad days and the such as well as a career trajectory which often peaks in their late 20s . Basing a players speed /acceleration on a few runs in his early 20s is far from infallible To sum up I'm sure a more methodical and spread out approach would be a benefit but let's have no illusions that this is more accurate just that's it's more likely to be so

Back up the train here. I never said that PFF wasn't valuable for something. PFF would be great for assigning traits to players in the game as they would likely follow tendencies - something PFF does quite well. I just don't believe that they are as valid for deriving the ratings themselves. The scouts do provide the data, but they don't make the picks. If I had a dollar for every time a GM went against the data to make a player selection...some of these guys (especially some of the more meddlesome owners) really just pick whoever the heck they want without using any data. It's happened before and will happen again. We are talking about billionaires who treat their teams as their hobbies, not the source of their wealth.

As for the 40s and their splits, I do follow the data from sources like speedendurance.com. They track particular sprints in the 100m dash for players throughout their careers. Keep in mind that many of these players still run 40s throughout their careers while training in the offseason. I know for a fact that under Holmgren, all players (including vets) still had to run timed 40s prior to training camps. It was part of their offseason schedule to evaluate everyone on their roster to see who was already in playing shape before camps began (they did other conditioning tests too, but that it besides the point).

The data that I get on NFL players' 40 times shows undoubtedly that players tend to lose about .005s on a 40 for every year in the league after being drafted. That is an average that includes guys who get hurt and guys who don't. It includes all positions (yes, some are more prone to losing more speed, likely through injury). Some players stay about the same after 10 seasons, while some drop off tremendously. It all depends on each individual player. I think I posted some of these in-house times on a forum here, but I don't remember which one.

The point is that I do have data for older players who recent timed in for their teams, and am building a model as we speak to include such data.

[briz1046]

Quote: Originally Posted by DCEBB2001

I have considered using PFF in the past and found, in my opinion, that the data they provide is less reliable than other sources. Like I said before, they would be good for tracking tendencies, but I do not believe they would be good for assigning values to the actual ratings. My reasoning for discrediting them is because actual scouts are trained observers. Viewing and recording is what they are paid to do. Like police testimony, their opinions hold a more weight. If I have 6 police officers tell me that they saw two planes fly into WTC 1 and 2 on 9/11/2001 and three hot dog stand operators say that they saw giant pterodactyls hit those buildings instead, I would be more apt to believe the trained observers in their version of the events. In rating these players, if you really want validity and accuracy, you have to go with sources you trust. I use scouting data at FBG because I have been on that side of the business before and I believe that it offers a certain level of reliability. You could just as well start your own ratings site using PFF as your source material and you may reach different conclusions (ratings) or you may reach the same ones. What matters is that as the author you trust the data you are using. So long as you use data that you believe is valid, you really can't go wrong. In this case, I am highly skeptical of the stuff PFF publishes because of things like the Brad Jones = Secret Superstar article.

The point at least in theory of using multiple sources for your input data would be to see where consensus is reached and point out where individual sources would vary from this so that any abberatins could be double checked
It may be that sources genuinely differ in opinion in which case only time will tell who's right but copying errors and the like can thus be identified and corrected
Individual scouts like PFF or anyone else will make mistakes and misjudgement occur no matter how reliable and highly regarded as the draft busts and free agent failures attest

[briz1046]

Quote: Originally Posted by DCEBB2001

Back up the train here. I never said that PFF wasn't valuable for something. PFF would be great for assigning traits to players in the game as they would likely follow tendencies - something PFF does quite well. I just don't believe that they are as valid for deriving the ratings themselves. The scouts do provide the data, but they don't make the picks. If I had a dollar for every time a GM went against the data to make a player selection...some of these guys (especially some of the more meddlesome owners) really just pick whoever the heck they want without using any data. It's happened before and will happen again. We are talking about billionaires who treat their teams as their hobbies, not the source of their wealth. As for the 40s and their splits, I do follow the data from sources like speedendurance.com. They track particular sprints in the 100m dash for players throughout their careers. Keep in mind that many of these players still run 40s throughout their careers while training in the offseason. I know for a fact that under Holmgren, all players (including vets) still had to run timed 40s prior to training camps. It was part of their offseason schedule to evaluate everyone on their roster to see who was already in playing shape before camps began (they did other conditioning tests too, but that it besides the point). The data that I get on NFL players' 40 times shows undoubtedly that players tend to lose about .005s on a 40 for every year in the league after being drafted. That is an average that includes guys who get hurt and guys who don't. It includes all positions (yes, some are more prone to losing more speed, likely through injury). Some players stay about the same after 10 seasons, while some drop off tremendously. It all depends on each individual player. I think I posted some of these in-house times on a forum here, but I don't remember which one. The point is that I do have data for older players who recent timed in for their teams, and am building a model as we speak to include such data.

Good to hear
I'm sure your correct in many cases as to scouting data being over - ruled or ignored but nevertheless no scout or source is infallible
As for PFF data I agree it's not designed as a source for rating individual player attributes but valid for comparing how different players should acredit themselves
Glad to hear you update your 40 times etc where possible and have a model for speed regression at hand . As I stated I believe track athletes often peak in their late 20s but on average would not be subject to the same wear and tear as their NFL counterparts who indeed may peak in this respect as rookies

[DeuceDouglas]

Quote: Originally Posted by DCEBB2001

I have considered using PFF in the past and found, in my opinion, that the data they provide is less reliable than other sources. Like I said before, they would be good for tracking tendencies, but I do not believe they would be good for assigning values to the actual ratings. My reasoning for discrediting them is because actual scouts are trained observers. Viewing and recording is what they are paid to do. Like police testimony, their opinions hold a more weight. If I have 6 police officers tell me that they saw two planes fly into WTC 1 and 2 on 9/11/2001 and three hot dog stand operators say that they saw giant pterodactyls hit those buildings instead, I would be more apt to believe the trained observers in their version of the events. In rating these players, if you really want validity and accuracy, you have to go with sources you trust. I use scouting data at FBG because I have been on that side of the business before and I believe that it offers a certain level of reliability. You could just as well start your own ratings site using PFF as your source material and you may reach different conclusions (ratings) or you may reach the same ones. What matters is that as the author you trust the data you are using. So long as you use data that you believe is valid, you really can't go wrong. In this case, I am highly skeptical of the stuff PFF publishes because of things like the Brad Jones = Secret Superstar article.

Never thought I'd see 9/11 used in an analogy when talking about ratings. Nice...

But an interesting point that comes up is whether or not Madden ratings represent the same things that a scouting rating would represent or if the ratings in-game are more a representation of the tendencies that you talk about and from my experience with Madden it's the latter when it comes to a lot of ratings. And I think khaliib brings up a good point when it comes to the animations as well. When you look at something like pass blocking, there's nothing that really represents good technique vs. bad technique when it comes to players and how they're rated. The rating represents how often they're likely to "win" and I think that's kind of what PFF does as well. It doesn't mean that it's right or that it should be the sole piece of data when it comes to rating players but I think when you look at Madden and how it and the ratings work, it's understandable, or at least makes sense, that information like that would be used. Again, that doesn't make it right or the most accurate or valid but until they overhaul or significantly tweak how things work in terms of animations and how ratings work, it is what it is and it's not really going to matter what is used.

[jerwoods]

i forgot to say that with my liquid ratings

that each player with have to have the potential to go to 99 if u use all your XP on him try to only get to ball to him each player will have to have a set rating but this way this does not break CCM mode of course maybe i could use a FBG set if i could get the ratings system changed to 2.0 - 9.9 or 1.2 to 9.9