Donny Moore, the 'Madden Ratings Czar', Leaving EA

[ggsimmonds]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

All you need to do is read their FAQ page. Lot's of stuff that should raise red flags in there. Here are some quotes:

"All of the data is collected in order to build a detailed picture of each player’s performance and production over the course of a season."

"We offer a different type of scouting, strictly based on performance and not technique or upside"

"We are looking for the result of that poor technique, not the poor technique itself. If poor technique results in a positive play, that is graded at the same level as good technique yielding a positive play."

"Essentially, we’ve created a new type of scouting that strictly looks at performance, not necessarily the process that gets there. In our dealings with NFL clients, we’ve referred to this as supplying the “what” as they supply the “why.” We can tell a team that an offensive tackle gives up an inordinate amount of bullrush pressure and they can determine if it’s a lack of technique, functional strength, or perhaps a combination of the two."



To me, PFF doesn't tell you how good a player is. Instead it tells us how well a player produces. There is a big difference between the two. They only look at the result of the play. Anytime someone attempts to justify rating players in Madden with PFF stats, I just roll my eyes and ignore it.

Depends on what you use or expect from PFF. To me nothing you quoted raises any kind of red flags. Sure, if a guy says player X must be better than Y because his grades are higher that would be unwise, but that is hardly the blame of PFF.

As for the use of PFF to rate players in Madden, I think there absolutely is justification. Madden's goal should be to replicate what happens in real life. PFF is results centric. The two go hand in hand. Not saying that ratings should be determined by PFF, but results based metrics should be considered. If someone has the position that because PFF is production based it should not play a role in rating players, then the only logical conclusion is that nothing production based should play a role. Meaning only rate players by scouting data. To me that is just as foolish as only relying on PFF or conventional stats.

I could be wrong, but Dan I think we have had this discussion before in the FBG thread.

[charter04]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

Depends on what you use or expect from PFF. To me nothing you quoted raises any kind of red flags. Sure, if a guy says player X must be better than Y because his grades are higher that would be unwise, but that is hardly the blame of PFF.

As for the use of PFF to rate players in Madden, I think there absolutely is justification. Madden's goal should be to replicate what happens in real life. PFF is results centric. The two go hand in hand. Not saying that ratings should be determined by PFF, but results based metrics should be considered. If someone has the position that because PFF is production based it should not play a role in rating players, then the only logical conclusion is that nothing production based should play a role. Meaning only rate players by scouting data. To me that is just as foolish as only relying on PFF or conventional stats.

I could be wrong, but Dan I think we have had this discussion before in the FBG thread.

It depends on how truly accurate you want ratings to be. If you want productions based ratings then EA's traditional ratings should work for you. It's the same basic method Tecmo Super Bowl used. That's why the TB Bucs had the best CB in that game. He had like 12 ints the year they based everything on. He wasn't even that good of a corner though. He just had one of those years.

Kind of like Larry Brown getting 2 ints in the Super Bowl and winning the MVP but, was an average talent and more a result of the system and just being at the right place at the right time.

I would rather let actual scouting data determine how players are rated and the results will be what they are. Just like real life.


Sent from my iPhone using Tapatalk

[charter04]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

And by the way I pay money for PFF's premium stats. So I'm not against that at all. I just don't think it's a good source for determining skills or attributes of players.

Not saying the info can't be valuable.


Sent from my iPhone using Tapatalk

[ggsimmonds]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

It depends on how truly accurate you want ratings to be. If you want productions based ratings then EA's traditional ratings should work for you. It's the same basic method Tecmo Super Bowl used. That's why the TB Bucs had the best CB in that game. He had like 12 ints the year they based everything on. He wasn't even that good of a corner though. He just had one of those years.

Kind of like Larry Brown getting 2 ints in the Super Bowl and winning the MVP but, was an average talent and more a result of the system and just being at the right place at the right time.

I would rather let actual scouting data determine how players are rated and the results will be what they are. Just like real life.


Sent from my iPhone using Tapatalk

Madden is not developed as well to do that though. No video game is. Every sports title uses production to grade players.

Yes, they have problems in how exactly they do it. I for one find their changes of player's physical ratings like speed to represent a very large problem in their rating process, but there is nothing wrong with the approach itself.

Madden is not a scouting tool, I would hope no one uses it as such. If Antonio Brown has a higher RTR in game than AJ Green I do not think anyone should take that as anything more than Madden's way to arrive at desired results.

I like to think of the electron and quantum physics. I cannot remember who exactly said it (Hawking maybe) but he mentioned that they don't know if the electron actually behaves this way in reality, nor do they care. They only care that this model allows for accurate predictions and it works. I view Madden in the same way.

Also, scouting data is not inherently superior. Madden would not become more objective if it primarily or only used scouting data. The two methods are equally subjective and opinion driven.

All I am saying is that there should be a combination, and not to disqualify PFF because they are not concerned with poor technique when Madden does not even replicate technique in the game!

[DCEBB2001]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

Meaning only rate players by scouting data. To me that is just as foolish as only relying on PFF or conventional stats.

I would only rate players based on the scouting data, so I wholeheartedly disagree. I do feel that PFF is less valid because of crap like this, and nothing will change my mind about that.

https://www.profootballfocus.com/blog/2013/05/24/secret-superstar-brad-jones/

[DCEBB2001]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

Madden is not developed as well to do that though. No video game is. Every sports title uses production to grade players.

Yes, they have problems in how exactly they do it. I for one find their changes of player's physical ratings like speed to represent a very large problem in their rating process, but there is nothing wrong with the approach itself.

Madden is not a scouting tool, I would hope no one uses it as such. If Antonio Brown has a higher RTR in game than AJ Green I do not think anyone should take that as anything more than Madden's way to arrive at desired results.

I like to think of the electron and quantum physics. I cannot remember who exactly said it (Hawking maybe) but he mentioned that they don't know if the electron actually behaves this way in reality, nor do they care. They only care that this model allows for accurate predictions and it works. I view Madden in the same way.

Also, scouting data is not inherently superior. Madden would not become more objective if it primarily or only used scouting data. The two methods are equally subjective and opinion driven.

All I am saying is that there should be a combination, and not to disqualify PFF because they are not concerned with poor technique when Madden does not even replicate technique in the game!

Scouting data is subjective, yes, but consider the sources. PFF uses three guys, who are admittedly, not scouts, to do their grading. My sources utilizes a team of professional scouts who are paid by an NFL front office to do their grading. You could sit here and argue all day that one is not better than the other, but as a person with a BA in history, I do know that the quality of your source material will no doubt affect the quality of your conclusions.

Just saying.

[ggsimmonds]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

I would only rate players based on the scouting data, so I wholeheartedly disagree. I do feel that PFF is less valid because of crap like this, and nothing will change my mind about that.

https://www.profootballfocus.com/blo...ar-brad-jones/

We would have to agree to disagree then. In most things, this included, limiting or restricting yourself to one thing is unwise. One should always consider all the evidence, utilize all resources, and consider different views or angles.

Scouting data is subjective, yes, but consider the sources. PFF uses three guys, who are admittedly, not scouts, to do their grading. My sources utilizes a team of professional scouts who are paid by an NFL front office to do their grading. You could sit here and argue all day that one is not better than the other, but as a person with a BA in history, I do know that the quality of your source material will no doubt affect the quality of your conclusions.

Just saying.

PFF's guys are not scouts no, nor do they claim to be. We both agree, and PFF acknowledges, that they are not in the business of scouting. So why discredit them because they are not professional scouts? Why would they be?

[KwiLL23]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

A couple of things I don't like about PFF is that they have some positions wrong. When the hell has Eric Weddle and Harrison Smith played SS. A lot of players are like that on their depth charts. I don't trust their grading system neither with the High Quality, Average, and etc. ratings.

[khaliib]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

What gets lost in the ratings discussion is "Animations" and that's the core of the systems problem no matter what data method is used.

For those ratings that are needed to trigger certain animations during gameplay, what is the best method/scale to be used as the mechanism to determine the triggering points?

If there are only "5" tackle animations the game will utilize, then any system that is used that incorporates data that goes beyond 5 is pointless because data after 5 isn't used.

Thus, is the issue with utilizing a 100 based scale for a particular rating, when there aren't 100 animations to be utilized for that rating.
Or any scale that's not in-line with number of possible triggering animations.

They should just render a particular rating according to the number of available animations to be the used.
As developement injects more animations for a particular rating, the scale should increase.
1) if there's currently 20 catch animations, then that's the scale for that rating.
2) if there's 5 tackle breaking animations, then that should be the scale.
We don't need to know the formula that dictates the win/loss of a triggered animation, just that said RB has/utilizes 3 tackle breaking animations out of 5, that he uses.
- not every back stiff arms, jukes, spin etc...

This is mainly dealing with those interaction ratings.
Of course ratings such Throw Power/Accuracy would require more under-the-hood structure even though they're animations also.

Not that this is the end all, be all answer to the issue, but it would provide a clean and applicable foundation that can be utilized within every mode of play or play style.

Just thinking out loud!!!

[raidertiger]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

Ratings are useless anyway in this game. Been watching some Madden challenge videos and it really makes apparent that in Madden, size and ratings are nearly irrelevant apart from the speed, throwing and kicking ratings.

[briz1046]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

All you need to do is read their FAQ page. Lot's of stuff that should raise red flags in there. Here are some quotes:

"All of the data is collected in order to build a detailed picture of each player’s performance and production over the course of a season."

"We offer a different type of scouting, strictly based on performance and not technique or upside"

"We are looking for the result of that poor technique, not the poor technique itself. If poor technique results in a positive play, that is graded at the same level as good technique yielding a positive play."

"Essentially, we’ve created a new type of scouting that strictly looks at performance, not necessarily the process that gets there. In our dealings with NFL clients, we’ve referred to this as supplying the “what” as they supply the “why.” We can tell a team that an offensive tackle gives up an inordinate amount of bullrush pressure and they can determine if it’s a lack of technique, functional strength, or perhaps a combination of the two."



To me, PFF doesn't tell you how good a player is. Instead it tells us how well a player produces. There is a big difference between the two. They only look at the result of the play. Anytime someone attempts to justify rating players in Madden with PFF stats, I just roll my eyes and ignore it.

Whilst I would agree that certain aspects of how PFF rates players can be worrisome to discount it's value entirely would be to throw out the baby with the bath water
It's oviously production based but in real life you rarely achieve consistent success in anything football included without the necessary talent. Over a short period you may get by with a combination of luck and/or help but in time the cream mostly rises to the top
Problems with PFF grades most often arise when sample sizes are small as they do not take any account of such factors as strength of opposition, minor injury etc which will affect smaller samples
Pure scouting data itself is far from infallible, it is after all at its heart opinion based , the opinions of experienced qualified professionals but still opinion and thus subjective
With all due respect to dans sources I'm sure you could find other equally qualified individuals who would produce markedly different grades
After all professional scouts are largely responsible for providing the data teams use in the draft and free agency where busts are frequent and potential stars go overlooked every year
Even the hard data such a 40 and split times and drills from the combine and pro days can be misleading . If you have ever followed the times run by sprinters over a year or career you would see peaks troughs trends as they have good/ bad days and the such as well as a career trajectory which often peaks in their late 20s . Basing a players speed /acceleration on a few runs in his early 20s is far from infallible
To sum up I'm sure a more methodical and spread out approach would be a benefit but let's have no illusions that this is more accurate just that's it's more likely to be so

[charter04]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

What gets lost in the ratings discussion is "Animations" and that's the core of the systems problem no matter what data method is used.

For those ratings that are needed to trigger certain animations during gameplay, what is the best method/scale to be used as the mechanism to determine the triggering points?

If there are only "5" tackle animations the game will utilize, then any system that is used that incorporates data that goes beyond 5 is pointless because data after 5 isn't used.

Thus, is the issue with utilizing a 100 based scale for a particular rating, when there aren't 100 animations to be utilized for that rating.
Or any scale that's not in-line with number of possible triggering animations.

They should just render a particular rating according to the number of available animations to be the used.
As developement injects more animations for a particular rating, the scale should increase.
1) if there's currently 20 catch animations, then that's the scale for that rating.
2) if there's 5 tackle breaking animations, then that should be the scale.
We don't need to know the formula that dictates the win/loss of a triggered animation, just that said RB has/utilizes 3 tackle breaking animations out of 5, that he uses.
- not every back stiff arms, jukes, spin etc...

This is mainly dealing with those interaction ratings.
Of course ratings such Throw Power/Accuracy would require more under-the-hood structure even though they're animations also.

Not that this is the end all, be all answer to the issue, but it would provide a clean and applicable foundation that can be utilized within every mode of play or play style.

Just thinking out loud!!!

Do you still have one of those rosters you did using the editor for NCAA FB 14?


Sent from my iPhone using Tapatalk

[DCEBB2001]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

We would have to agree to disagree then. In most things, this included, limiting or restricting yourself to one thing is unwise. One should always consider all the evidence, utilize all resources, and consider different views or angles.

PFF's guys are not scouts no, nor do they claim to be. We both agree, and PFF acknowledges, that they are not in the business of scouting. So why discredit them because they are not professional scouts? Why would they be?

I have considered using PFF in the past and found, in my opinion, that the data they provide is less reliable than other sources. Like I said before, they would be good for tracking tendencies, but I do not believe they would be good for assigning values to the actual ratings.

My reasoning for discrediting them is because actual scouts are trained observers. Viewing and recording is what they are paid to do. Like police testimony, their opinions hold a more weight. If I have 6 police officers tell me that they saw two planes fly into WTC 1 and 2 on 9/11/2001 and three hot dog stand operators say that they saw giant pterodactyls hit those buildings instead, I would be more apt to believe the trained observers in their version of the events.

In rating these players, if you really want validity and accuracy, you have to go with sources you trust. I use scouting data at FBG because I have been on that side of the business before and I believe that it offers a certain level of reliability. You could just as well start your own ratings site using PFF as your source material and you may reach different conclusions (ratings) or you may reach the same ones. What matters is that as the author you trust the data you are using. So long as you use data that you believe is valid, you really can't go wrong. In this case, I am highly skeptical of the stuff PFF publishes because of things like the Brad Jones = Secret Superstar article.

[DCEBB2001]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

Whilst I would agree that certain aspects of how PFF rates players can be worrisome to discount it's value entirely would be to throw out the baby with the bath water
It's oviously production based but in real life you rarely achieve consistent success in anything football included without the necessary talent. Over a short period you may get by with a combination of luck and/or help but in time the cream mostly rises to the top
Problems with PFF grades most often arise when sample sizes are small as they do not take any account of such factors as strength of opposition, minor injury etc which will affect smaller samples
Pure scouting data itself is far from infallible, it is after all at its heart opinion based , the opinions of experienced qualified professionals but still opinion and thus subjective
With all due respect to dans sources I'm sure you could find other equally qualified individuals who would produce markedly different grades
After all professional scouts are largely responsible for providing the data teams use in the draft and free agency where busts are frequent and potential stars go overlooked every year
Even the hard data such a 40 and split times and drills from the combine and pro days can be misleading . If you have ever followed the times run by sprinters over a year or career you would see peaks troughs trends as they have good/ bad days and the such as well as a career trajectory which often peaks in their late 20s . Basing a players speed /acceleration on a few runs in his early 20s is far from infallible
To sum up I'm sure a more methodical and spread out approach would be a benefit but let's have no illusions that this is more accurate just that's it's more likely to be so

Back up the train here. I never said that PFF wasn't valuable for something. PFF would be great for assigning traits to players in the game as they would likely follow tendencies - something PFF does quite well. I just don't believe that they are as valid for deriving the ratings themselves. The scouts do provide the data, but they don't make the picks. If I had a dollar for every time a GM went against the data to make a player selection...some of these guys (especially some of the more meddlesome owners) really just pick whoever the heck they want without using any data. It's happened before and will happen again. We are talking about billionaires who treat their teams as their hobbies, not the source of their wealth.

As for the 40s and their splits, I do follow the data from sources like speedendurance.com. They track particular sprints in the 100m dash for players throughout their careers. Keep in mind that many of these players still run 40s throughout their careers while training in the offseason. I know for a fact that under Holmgren, all players (including vets) still had to run timed 40s prior to training camps. It was part of their offseason schedule to evaluate everyone on their roster to see who was already in playing shape before camps began (they did other conditioning tests too, but that it besides the point).

The data that I get on NFL players' 40 times shows undoubtedly that players tend to lose about .005s on a 40 for every year in the league after being drafted. That is an average that includes guys who get hurt and guys who don't. It includes all positions (yes, some are more prone to losing more speed, likely through injury). Some players stay about the same after 10 seasons, while some drop off tremendously. It all depends on each individual player. I think I posted some of these in-house times on a forum here, but I don't remember which one.

The point is that I do have data for older players who recent timed in for their teams, and am building a model as we speak to include such data.

[briz1046]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

I have considered using PFF in the past and found, in my opinion, that the data they provide is less reliable than other sources. Like I said before, they would be good for tracking tendencies, but I do not believe they would be good for assigning values to the actual ratings.

My reasoning for discrediting them is because actual scouts are trained observers. Viewing and recording is what they are paid to do. Like police testimony, their opinions hold a more weight. If I have 6 police officers tell me that they saw two planes fly into WTC 1 and 2 on 9/11/2001 and three hot dog stand operators say that they saw giant pterodactyls hit those buildings instead, I would be more apt to believe the trained observers in their version of the events.

In rating these players, if you really want validity and accuracy, you have to go with sources you trust. I use scouting data at FBG because I have been on that side of the business before and I believe that it offers a certain level of reliability. You could just as well start your own ratings site using PFF as your source material and you may reach different conclusions (ratings) or you may reach the same ones. What matters is that as the author you trust the data you are using. So long as you use data that you believe is valid, you really can't go wrong. In this case, I am highly skeptical of the stuff PFF publishes because of things like the Brad Jones = Secret Superstar article.

The point at least in theory of using multiple sources for your input data would be to see where consensus is reached and point out where individual sources would vary from this so that any abberatins could be double checked
It may be that sources genuinely differ in opinion in which case only time will tell who's right but copying errors and the like can thus be identified and corrected
Individual scouts like PFF or anyone else will make mistakes and misjudgement occur no matter how reliable and highly regarded as the draft busts and free agent failures attest