Donny Moore, the 'Madden Ratings Czar', Leaving EA

[DCEBB2001]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

Where your position falls flat is the issue is not comparing the scouting ability of scouts to the scouting ability of statisticians. The real comparison is the ability of scouting to determine player value vs the ability of metrics and numbers to do so. Personally I think you need a mix of both, but to disregard a stat based system because they aren't trained scouts would make your analogy more like eating lunch from the squad car instead of the hot dog cart because the cops are trained observers.

PFF uses the same three people to observe every down BEFORE they run their metrics. The data I have access to utilizes over a dozen people who are trained to professionally observe football players and their abilities. It's not that I don't trust the mathematics involved at getting their results, I don't trust their ability to OBSERVE as keenly as what I would trust a dozen professional scouts. Like I said a million times before in threads like these, if the source data lacks validity, as I posit PFF does in comparison to data derived by professional scouts, then your results will lack validity. I believe that is the "transitive property" of logic.

Nothing anyone says here is going to make me change my mind about this. Especially when PFF gives you crap like this:

https://www.profootballfocus.com/blog/2013/05/24/secret-superstar-brad-jones/

To me, the guy doing the FBG Ratings, scouting data from professional scouts in an actual NFL front office > PFF. That's my opinion and I will not deviate from that. You can take that one to the bank.

[Bull_Dozer]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

A huge issue I have had with ratings is how "pursuit" is rated..Obviously it is too high, but when a DT with 55 speed and 90 pursuit can chase down a WR with 90 speed just because, as it appears, the pursuit takes over when chasing anyone with the ball...

Also, if pursuit works as I mention above, then DT's are going after a QB with something like 90 speed, yes?

Yeah, the whole Speed vs Pursuit thing is very weird to me. Like a slow DT just suddenly becomes fast because he's chasing the ball carrier. It makes no sense.

[DCEBB2001]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

Yeah, the whole Speed vs Pursuit thing is very weird to me. Like a slow DT just suddenly becomes fast because he's chasing the ball carrier. It makes no sense.

Agreed. That rating should be linked to the pursuit angle that the defender takes. Do they take a proper angle, and if so, do they deviate from it properly to avoid other players (the trash)?

That is what that attribute SHOULD do.

[briz1046]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

PFF uses the same three people to observe every down BEFORE they run their metrics. The data I have access to utilizes over a dozen people who are trained to professionally observe football players and their abilities. It's not that I don't trust the mathematics involved at getting their results, I don't trust their ability to OBSERVE as keenly as what I would trust a dozen professional scouts. Like I said a million times before in threads like these, if the source data lacks validity, as I posit PFF does in comparison to data derived by professional scouts, then your results will lack validity. I believe that is the "transitive property" of logic.

Nothing anyone says here is going to make me change my mind about this. Especially when PFF gives you crap like this:

https://www.profootballfocus.com/blog/2013/05/24/secret-superstar-brad-jones/

To me, the guy doing the FBG Ratings, scouting data from professional scouts in an actual NFL front office > PFF. That's my opinion and I will not deviate from that. You can take that one to the bank.

I appreciate your point that the guys at PFF aren't trained scouts but in all honesty this isn't as serious an issue as you make it because they aren't attempting to grade players based on techniques etc merely observing win / loss scenarios
Did a rushed beat his blocker , did a receiver catch or drop a pass etc
They state that most plays result in neutral or zero grades and only a relatively few are marked as wins or losses
So whilst a scout may point out a receiver who frequently body catches PFF would not even attempt to
As such the site has a use in it offers comparisons of a players success rate but no insight into any individual talents a player may have
Even then tho it has flaws in that it doesn't take any account of weight of schedule ( each team will only play 13 of the potential 31 opponents during a NFL season) or if players are carrying known injuries , factors I'm sure scouts consider in their grades
I believe it's wise to use as many sources as possible as in this way any errors that slip through the net are more easily identified and corrected
It's my opinion that the problem with madden ratings is less to do with the data used and more about how it is used , both the systems used and the fact that popular opinion, marketing etc plays a significant role

[DCEBB2001]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

I appreciate your point that the guys at PFF aren't trained scouts but in all honesty this isn't as serious an issue as you make it because they aren't attempting to grade players based on techniques etc merely observing win / loss scenarios
Did a rushed beat his blocker , did a receiver catch or drop a pass etc
They state that most plays result in neutral or zero grades and only a relatively few are marked as wins or losses
So whilst a scout may point out a receiver who frequently body catches PFF would not even attempt to
As such the site has a use in it offers comparisons of a players success rate but no insight into any individual talents a player may have
Even then tho it has flaws in that it doesn't take any account of weight of schedule ( each team will only play 13 of the potential 31 opponents during a NFL season) or if players are carrying known injuries , factors I'm sure scouts consider in their grades
I believe it's wise to use as many sources as possible as in this way any errors that slip through the net are more easily identified and corrected
It's my opinion that the problem with madden ratings is less to do with the data used and more about how it is used , both the systems used and the fact that popular opinion, marketing etc plays a significant role

I have stated this before, but will do so again.

I think that PFF is great for identifying tendencies, but not for rating attributes. The win/loss should be played out because of the combination of the individual attributes, not because PFF says that it should.

Further, those tendencies could be carried out to individual teams/coordinators to present an accurate depiction of what teams like to do and when with certain players. That is how I think PFF could be utilized. I just believe that it wouldn't be as good of a source compared to what I have.

However, I would love to see someone make their own ratings using PFF and see how that works. I have already researched PFF years ago, and simply found that it didn't carry over as well as other means. PFF will not tell me how agile or fast a player is, so I can't use it for the attributes.

Tendencies, maybe. Attributes, no.

[briz1046]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

I have stated this before, but will do so again.

I think that PFF is great for identifying tendencies, but not for rating attributes. The win/loss should be played out because of the combination of the individual attributes, not because PFF says that it should.

Further, those tendencies could be carried out to individual teams/coordinators to present an accurate depiction of what teams like to do and when with certain players. That is how I think PFF could be utilized. I just believe that it wouldn't be as good of a source compared to what I have.

However, I would love to see someone make their own ratings using PFF and see how that works. I have already researched PFF years ago, and simply found that it didn't carry over as well as other means. PFF will not tell me how agile or fast a player is, so I can't use it for the attributes.

Tendencies, maybe. Attributes, no.

I agree that PFF does not provide any data that would be useful in in assigning attribute ratings as such
However it can be used a a fail check , for example if the scouting data and PFF results differ wildly closer examination could be made. PFF scores on a results based system and unless a player is remarkably *lucky* good results should follow from good technique etc
Neither scouting nor metrics are infallible but cross checking is always good practice

[KG]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

I agree that PFF does not provide any data that would be useful in in assigning attribute ratings as such
However it can be used a a fail check , for example if the scouting data and PFF results differ wildly closer examination could be made. PFF scores on a results based system and unless a player is remarkably *lucky* good results should follow from good technique etc
Neither scouting nor metrics are infallible but cross checking is always good practice

I think that's the key point and why myself and others like the FBG data over the PFF. IMO it's a better indicator over the long run and will lead to more reliable data.

Now what EA does with the data and getting it to translate on the virtual field is a different matter. They (EA) are kind of stuck with their flawed ratings system at this point. Say EA implements a speed ratings system in line with accurate up-to-date 40 times. Fans and players themselves would throw a fit (although the action on the field wouldn't change IF everyone was on the same scale).

Great discussion btw.

[briz1046]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

In a idealised world and where data samples where large enough to eliminate normal variances the results based metrics should tally with the scouting reports
Good technique should bring corresponding good results
In reality this will not always occur however comparing the different sources and highlighting any differences for further analysis should produce the best results
No person is infallible no matter their qualifications or pedigree

[4thQtrStre5S]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

Regardless of the numbers, the software engine has to process them correctly and then it has to be represented upon the playing field correctly..

[4thQtrStre5S]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

Another reason for having ratings like FBG, is that results on the field, what the players do are team based...

Let me try to explain what I mean:

Emmitt Smith, came from big college florida and then went to the Cowboys and ran behind the biggest one on one blockers, and arguably one of the best lines ever..Smith ran a 4.70 40, and some people said he cant go 80 yards..

Barry Sanders, came from Oklahoma State and went to the Detroit Lions, one of the consistently bad teams and also didn't have anything for an offensive line, arguably one of the best runners..He ran a 4.37 40...

Madden would have these players rated high, and maybe equally high, though Barry Sanders may not look as Good in Madden because the offensive line would be rated lower than the Cowboys...

BUt with a solid ratings based system like FBG, Emmitt Smith should be rated well below Sanders and thus require the big line and Sanders would be rated higher and in relation to the other players in the league that he would shine behind even some of the worst lines...

For example, In Madden I would bet they rated Smith's agility about the same as Sander's, which would be wrong..

[DCEBB2001]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

Another reason for having ratings like FBG, is that results on the field, what the players do are team based...

Let me try to explain what I mean:

Emmitt Smith, came from big college florida and then went to the Cowboys and ran behind the biggest one on one blockers, and arguably one of the best lines ever..Smith ran a 4.70 40, and some people said he cant go 80 yards..

Barry Sanders, came from Oklahoma State and went to the Detroit Lions, one of the consistently bad teams and also didn't have anything for an offensive line, arguably one of the best runners..He ran a 4.37 40...

Madden would have these players rated high, and maybe equally high, though Barry Sanders may not look as Good in Madden because the offensive line would be rated lower than the Cowboys...

BUt with a solid ratings based system like FBG, Emmitt Smith should be rated well below Sanders and thus require the big line and Sanders would be rated higher and in relation to the other players in the league that he would shine behind even some of the worst lines...

For example, In Madden I would bet they rated Smith's agility about the same as Sander's, which would be wrong..

Wow. That is one of the best analogies I have read on here in regards to what I am trying to do. One of the things that the scouting data supplies is individualized data. So much of what we see in terms of production is based on the team effort. The scouting data allows us to break that down player-by-player and attribute-by-attribute.

If a WR is capable of catching 99 of 100 balls you throw to him, but he only gets 10 targets in the season and drops his one ball in those 10 targets, you would think that he only has a catch rate of 90% instead of 99%. PFF is giving you the game-observed 90% and is excluding what would happen if he had infinite opportunity.

The scouting data attempts to rate players under the assumption that all conditions are equal for all players (targets = opportunities). They go beyond what you see in a game and add in what they see from workouts, practices, etc. With this data, you get a more complete image of the player and what his abilities should be if all circumstances are equal.

[PGaither84]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

I honestly haven't seen the FBG page in the last 3-4 years. I always fully supported the idea of fan adjusted ratings. In the past DCEBB2001 and I have had disagreements on the results and methodology for arriving at acceleration vs speed numbers and if I recall we also talked about the zone/man coverage ratings for linebackers and defensive ends.

I have had personal success in past iterations of Madden when I sit down and do roster overhauls. They only apply to PLAY NOW, but they do greatly improve that "pick up and play" experience. I have written about it before and wish that it would be applied to the game holistically.

More spread out ratings and an overall lower ratings curve improves game play.

[DCEBB2001]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

I honestly haven't seen he FBG page in the last 3-4 years. I always fully supported the idea of fan adjusted ratings. In the past DCEBB2001 and I have had disagreements on the results and methodology for arriving at acceleration vs speed numbers and if I recall we also talked about the zone/man coverage ratings for linebackers and defensive ends.

I have had personal success in past iterations of Madden when I sit down and do roster overhauls. They only apply to PLAY NOW, but they do greatly improve that "pick up and play" experience. I have written about it before and wish that it would be applied to the game holistically.

More spread out ratings and an overall lower ratings curve improves game play.

Yeah, the whole debate over the speed v. acceleration issues in Madden 11's engine was totally proven by my research with CM Hooe. The SPD and ACC ratings did work symbiotically with one another in the sense that the SPD rating determined the maximum velocity when the acceleration phase was over, and the ACC rating affected how quickly you derived at that maximum velocity. Fast-forward a few years and EA finally states in writing that this was the case.

The way I had done things, knowing of the symbiotic relationship that was later proven in our analysis, was use the 40 to represent the average velocity over time and use the 10yd split as a segment to represent the initial burst. This however, was not the best way to do things, so I sought out some professional advice and basically found that there are two ways to do it:

1. Take the 10yd split and use that the correlate to initial burst (ACC) and use the 40-10yd time to match to the SPD rating.

2. Use the splits to create a cubic function and differentiate.

Option number 2 is by far the most accurate. Option 1 is valid because, and I am paraphrasing the expert I sought on this, "the lack of numerous data points means that using the difference in the times could still be around 95% accurate to accurately reflect the acceleration phase and the top velocity phase." I chose the 2nd option because I wanted to be as accurate as possible.


The issue regarding the MCV/ZCV attributes for LBs was something that had to be fixed, but not without supporting data. It was in my analysis of my source material that I found that many LBs were in possession of ZCV/MCV skills that rivaled even some of the best CBs in the game. As a result, I had to ensure that these ratings were accurately reflected.

[PGaither84]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

I really don't want to open up that can of worms about acceleration and speed, and you can make claims to how "accurate" your method is or isn't. The reality is that your speed and acceleration ratings for players were nearly identical while Donny's ratings back then had a lot more diversity. Wes Welker at the time had 99 acceleration and 88 speed, where you had something along the liens of like 93 and 91 each. Honestly, I don't remember your exact numbers and they don't even exits on your site anymore to cross reference... or if they are, they certainly aren't easy to find.

I just know that MOST of your ratings I agreed with, except for Speed/Acceleration, which I agreed with Donny. Also, it made players feel more dynamic, especially on 100 speed threshold... which is another topic in terms of game play.

Once I was shown how 100 speed threshold positively effected game play in Madden 11 and 12, I never went back. 0 threshold made the game play like high school football, where a single point of difference in speed made a world of difference. A 93 speed HB flew around the field leaving elite linebackers like Patrick Willis and his 89/90 in the dust. With 100 speed threshold, those same elite linebackers were tackling machines and only the elite HBs in the game (like Chris Johnson and Adrian Peterson) had a chance of outrunning them to the corner.

I don't know what in-game 40 times looked like with 100 speed threshold, but as I admit freely, I don't really care. I care about the quality of the game play more than anything. 100 speed threshold still had break away runs, kick off and punt returs for TDs, and the ability to throw over the top, but they weren't as common as they are on 50 or 0 speed threshold. you have to earn them. You had to get the right blocks, take the right angles, throw against the right coverage.

Also, at least in Madden 11 and 12, I used 100 speed threshold in conjunction with 100 fatigue and custom substitution and injury settings. Fatigue in those games was pretty strong. Players who were tired played worse than their maximum ratings, so wearing out a defense and resting yours had an impact on games and vise versa. The tempo and flow of games felt more fluid and natural. Going three and out in the third quarter could be devastating, while a 12 play drive that settles for a FG can turn the tide in your favor as your rested defense come back out to make a stand.

**All of those aspects of game play, which don't seem to function the same in Madden 25. I don't have Madden 15, though I might get 16 this year.

Maybe I am getting a bit off topic, but all of these things are directly related and tied to the user experience. I play Madden 25 for the offline CCM experience, but I put on Madden 12 every now and again becasue the on-field game play in PLAY NOW is so much more satisfying thanks to my roster overhaul and custom settings.

=============================

I completely agreed with you about LB ratings I don't remember if I influenced you to make those changes, but I certainly started a thread about it before I recall you even posted that you were taking over the FBG project. I certainly know you posted about it and gave great feedback on my thread years ago that helped me make tweaks to my custom rosters.

Line backer coverage skills is an important Madden topic for me anytime I give feedback about Madden. I just wasn't able to sell the idea to Donny while at Community Day for Madden 11.

[DCEBB2001]

Re: Donny Moore, the 'Madden Ratings Czar', Leaving EA

I really don't want to open up that can of worms about acceleration and speed, and you can make claims to how "accurate" your method is or isn't. The reality is that your speed and acceleration ratings for players were nearly identical while Donny's ratings back then had a lot more diversity. Wes Welker at the time had 99 acceleration and 88 speed, where you had something along the liens of like 93 and 91 each. Honestly, I don't remember your exact numbers and they don't even exits on your site anymore to cross reference... or if they are, they certainly aren't easy to find.

I can assure that at no point did I ever have Wes Welker at a SPD rating in the 90s.

Ever.

Looking at my archives the closest I had to anything like that for Welker was in 2011 when he had a SPD of 78 and and ACC of 88. Outside of that, since 2012, he has been closer to a 77 SPD and 76 ACC.

You may be referring to some ratings where the 10yd split was used solely as the ACC rating without context of the subsequent splits but that was done away with nearly 4 years ago (which makes sense if you haven't been back to the site in 3-4 years). Things have changed a lot since I started deferring to people with degrees in various fields to help ensure that the methodology is accurate. The method employed now is as accurate is it one can get in defining how fast a player accelerates and how fast he is moving at top speed. I have been assured of that.