FOF2K4: Mentors & Player Development - have we been assuming too much?

[cthomer5000]

After reading a remark someone made about being disappointed that they hadn't really seen any concrete test results on mentors, I decided to give it a try.

Here were my general assumptions going in:

1. Mentors help a player (who is receiving playing time) develop more quickly
2. It does not matter whether a mentor is active or inactive
3. Player development is more or less at a set rate, based on playing time (basically, i'd expect it to be pretty easy to replicate results)

Here is QB Kendrick Ortiz of the 2113 Salt Lake City Raiders. This is him on the eve of the first pre-season game. He is rookie, selected in the 4th round out of Nebraska-Omaha:



He's rated 17/51 overall. At this point I decided to sim 1 season a number of times to see how he developed.

- Every player around him was the same in all seasons except for the backup QB in the case of the mentor being active/inactive.
- Kendrick's playing time was set to "9" in every season
- Injuries were OFF, so as to not interfere with anything
- Kendrick started all 20 games (pre and regular season)
- All "end" ratings are taken at the end of the regular season, so no additional playoff games have come into play in any of these years.

NO MENTOR
26/51
27/51
28/51
28/51
30/51

total gain: 54


MENTOR (INACTIVE)
24/51
27/51
29/51
30/51
31/51

total gain: 56


MENTOR (ACTIVE)
30/51
30/51
30/51
33/51
32/51

total gain: 70


Some observations:

1. The results weren't as straight-forward as I was expecting. The variance was pretty large (7 points in one group).

2. Either there are other factors at play in player development, or player growth is more random than I had assumed.

3. Mentors being on the active roster apparently greatly increases their effect. At this stage I don't think I can even definitely say if the mentorship works when a player is not on the active roster.

Some questions:

1. Is my sample size way too small? One player, run 5 times in each fashion probably isn't enough to draw definitive conclusions, although I think it may give us a general idea.

2. Have we been completely wrong about mentors not needing to be on the active roster? (that's definitely the accepted thought around here)


I have screen shots of every season-ending player ratings if anyone wants to see them.

[cthomer5000]

Something that really has me thinking that a mentor may need to be active is the fact that Kendrick had his very worst season of development with an inactive mentor (even worse than ANY of the seasons with NO mentor!)

Also, the consistency shown with an active mentor says something to me.

[Dutch]

Mentors have been somewhat of a mystery since the beginning. The active vs. inactive issue is news to me and could offset some of my own thinking on mentors. I think we all know the basic premise that mentors will bring a young players actual ratings up quicker, but what other factors define this?

I have a sneaky suspicion that playing time or playing well has an effect. While watching the development of Michael Prescott of IHOF in "slow motion", I tended to start guessing if he would gain a point or not based whether the Knights won or lost (I *felt* like he was gaining points more often if the Knights won or if he did well.) Maybe gamestats play a role. (Edit: Prescott did not have a mentor, but he was the guy who was developing the most. I had a DE mentor and my young DE's who did not play would gain nothing. I had a K mentor and my young K who kicked almost every game nearly maxed out in one season. Maybe not the end-all of litmus tests, but simple observations.)
Also, I noticed some mentors have zero effect on players. That could be an early indicator that a young player won't develop anymore because he's a bust. Or perhaps longer experience as a player enhances the mentor's ability to help a young player. Or, as stated earlier, I may be noticing this stagnation because I had the mentor inactive.

[Grid Iron]

This was an excellent study. Well done and very interesting!

I added this thread to the FOF2K4 Codex.

[jbmagic]

where on FOF, i see if i have a mentor?

can you have more than one mentor on roster?

thanks

[CraigSca]

It'll show that a particular player is a mentor on his player card, where it shows all his biographical info, agent name, etc.

I believe you can only have one mentor per position set, but you can have multiple mentors across different positions.

[cthomer5000]

Originally Posted by CraigSca

It'll show that a particular player is a mentor on his player card, where it shows all his biographical info, agent name, etc.

I believe you can only have one mentor per position set, but you can have multiple mentors across different positions.

You can have multiple mentors at a position, but we have no idea if having 2 helps more than having 1. Again, seemingly no one has tested it.

[MIJB#19]

Originally Posted by Dutch

I have a sneaky suspicion that playing time or playing well has an effect. While watching the development of Michael Prescott of IHOF in "slow motion", I tended to start guessing if he would gain a point or not based whether the Knights won or lost (I *felt* like he was gaining points more often if the Knights won or if he did well.) Maybe gamestats play a role.

I have the same suspicion. It looks like a winning or 200yd throwing QB makes more developement, but it's not always the case.

Originally Posted by Dutch

Also, I noticed some mentors have zero effect on players. That could be an early indicator that a young player won't develop anymore because he's a bust. Or perhaps longer experience as a player enhances the mentor's ability to help a young player. Or, as stated earlier, I may be noticing this stagnation because I had the mentor inactive.

Maybe we're ignoring the 'intelligence' factor? Could it be that only intelligent players are 'vulnerable' to being mentored? Or maybe personality, leadership and/or chemistry come into play?

[MIJB#19]

Looking at the end of regular season overall ratings, the one figure jumping out is the 24/51. When taken out, the inactive mentor clearly was better than the non-mentor.

As I wrote in the strategy forum, resimming a single shows there can be differences in the progress of players, even when the same lineup is used. I remember a pre-season sim in the IHOF (f-ing elitist!) that had different progress outcomes as I always check player progress before narrowing down why my team lost.

[gstelmack]

I think that not only do you need a bit larger sample size than 5 seasons (something like 10 seasons throwing out the best and worst), but you also need more than one year. QBs especially need 3 or 4 seasons to really develop. Run this guy through 4 seasons 10 times under each condition to get a better picture.

[QuikSand]

Maybe the value of a mentor applies only if the young player is not getting playing time?

[Cap Ologist]

Mentors seem pretty useless. His kickholding ability never improved, why would you want a qb good at anything other than that?

[Icy]

Why do we need to have this mistery in FOF? won't be better to have at least some of this basic game features documented somewhere? I find fun to find what tactics help my team to win, trying to sign the best draftees, etc but not guessing how the game works, i still don't underestand lots of parts from FOF2k4 and only have read guesses about it after some members doing tons of studies that i bet aren't so fun for them either.

[Dutch]

Icy,

I just go by the documentation the NFL has on mentor's. Obviously, without that, the NFL would be just as frustrated.

[cthomer5000]

Originally Posted by QuikSand

Maybe the value of a mentor applies only if the young player is not getting playing time?

Interesting thought. Wish I'd thought to track the other QBs during this test.

I plan on running another later today, so I'll try to include a few more things during that one.

[cthomer5000]

Ok, I've started on another test. I've got 3 rookie QBs and 1 mentor. I'm going to skip the screenshots since not doing them will make things about 20 times faster.

[cthomer5000]

Originally Posted by QuikSand

Maybe the value of a mentor applies only if the young player is not getting playing time?

Ok, After just the first phase of the trial I think I'm ready to say that an inactive mentor has zero effect on a player who is not receiving playing time. My #2 and #3 QBs (who i don't believe attempted a single pass in any of the 10 seasons) gained nothing in any of the 10 trials.



Columns B and C are the current and future ratings, with those in row 7 being the preseason numbers. Columns E and F represent the Qb rating he had and the number of wins the team had (remember, i take these numbers from the end of the regular season, so playoffs never come into play).

I didn't think to chart the QB rating or wins until after the first run, and then simply forgot to check the QB rating in trial #8.

[cthomer5000]

Phase 2 - no mentor at all.

Results are so close that I'm really beginning to think that an inactive mentor is completely worthless.

[cthomer5000]

Well, seems pretty damned straight-forward to me. The Mentor is active, and we see far more consistent growth, a statistically significant difference over the course of 10 trials, and growth from the non-playing backup QBs for the first time ever. The inactive backup still gains nothing here.

[korme]

great work, ct..

[Ben E Lou]

Originally Posted by cthomer5000

The inactive backup still gains nothing here.

GRASSY KNOLL ALERT!!! GRASSY KNOLL ALERT!!! GRASSY KNOLL ALERT!!!

I'm almost certain that I recall someone doing an observation in an early version of FOF that an inactive backup gained current ability points in the presence of a mentor. Could this be a potential "under-the-hood" change done in one of the patches/updates?

[cthomer5000]

I've swapped the starter and backup here, and kept the mentor (Rich Gannon's alter-ego) at #3.

I think it's clear that LeTendresse is just going to mature at a faster rate than Jenkins. I mean, look at some of that development he receives for just sitting there holding a clipboard! I'm wondering if the real value in mentors is for developing the backups on the active roster. This whole test has been an eye-opener.

[VPI97]

I wonder if affinities/birthdays factor into this somehow.

[Icy]

It makes sense to me, usually you will have as backup your future prospect QB and the mentor should try to teach him, the starting QB can learn more from playing time and doesn't need anybody teaching him.

[Dutch]

Why does the Phase 2 no mentor test show the 3rd QB as Bruce?

[cthomer5000]

Originally Posted by VPI97

I wonder if affinities/birthdays factor into this somehow.

Not an issue among QBs though. They don't interact with each other, just other position leaders.

For the record, Jenkins has 1 affinity with a leader, LaTendresse has none.

LaTendresse's intelligence is 60 to Jenkins' 15, so it's possibly MIJB's theory holds some merit, although i don't know if we can isolate that sort of thing.

[cthomer5000]

Originally Posted by Dutch

Why does the Phase 2 no mentor test show the 3rd QB as Bruce?

Well, i just have those 3 listed because they're the 3 rookie QBs. The mentor is the 3rd QB and Bruce is inactive (as stated in the 2-3 lines above the data).

[Dutch]

Originally Posted by cthomer5000

Well, i just have those 3 listed because they're the 3 rookie QBs. The mentor is the 3rd QB and Bruce is inactive (as stated in the 2-3 lines above the data).

gotcha.

[cthomer5000]

Jenkins starting with no mentor. Again, the reserves gain absolutely nothing. On a % basis, looks like the mentor is more important to Jenkins than LaTendresse.

[VPI97]

Originally Posted by cthomer5000

Jenkins starting with no mentor. Again, the reserves gain absolutely nothing. On a % basis, looks like the mentor is more important to Jenkins than LaTendresse.

Just to humor me, what are their birthdays? (LaTendresse, Jenkins, mentor)

[cthomer5000]

Originally Posted by VPI97

Just to humor me, what are their birthdays? (LaTendresse, Jenkins, mentor)

mentor = 1/20
Jenkins = 1/28
LaTendresse = 11/8
Bruce = 11/1

[cthomer5000]

Do players even have to be on the depth chart to receive mentoring? NO!

Bruce is an active #4 QB here, but appears nowhere on the depth chart (not even as holder).

[jbmagic]

cthomer5000

great test.

so what your conclusion summary on mentors.

thanks

[VPI97]

On a % basis, looks like the mentor is more important to Jenkins than LaTendresse.

Originally Posted by cthomer5000

mentor = 1/20
Jenkins = 1/28
LaTendresse = 11/8
Bruce = 11/1

Using wade's chart from the IHOF Board, I get this:

Player Role	Birthday	Month	Day	Combined	Zod Number	Affinity 1	Affinity 2	Conflict
Leader Birthday	20-Jan	1	20	120	10
Follower Birthday	28-Jan	1	28	128	11	No Affinity	Affinity	No Conflict
Follower Birthday	8-Nov	11	8	1108	8	No Affinity	No Affinity	No Conflict
Follower Birthday	1-Nov	11	1	1101	8	No Affinity	No Affinity	No Conflict

Coincidence?

[cthomer5000]

Also, you can have a seemingly endles number of guys active but not on the depth chart receive mentoring. I just ran a season with 6 active QBs. All 3 that were not even on the depth chart showed solid growth over the course of the year.

[cthomer5000]

Just one last test with the QBs for now. I just wanted to make sure the Mentor didn't need to be on the depth chart, and i confirmed that. As long as he's active, it doesn't matter whether he's on the depth chart or not. The two guys on the right are two scrub UDFA 1st year QBs i added.

edit: Also, having LaTendresse be the holder didn't appear to give him any sort of extra playing time bonus.

[cuervo72]

Have you looked at all at what the QB's ratings are the next year?

[cthomer5000]

Originally Posted by cuervo72

Have you looked at all at what the QB's ratings are the next year?

are we talking off-season growth? no, i haven't checked that.

I plan on eventually checking to see if they have any effect in training camp as well.

[Yossarian]

I guess this means it could be useful / benificial to hire injured mentors who are going cheap - providing you can spare a roster spot for an active - but unable to get on the field player

[cthomer5000]

Tought to say if this is conclusive. My gut feeling is that multiple mentors have no more effect than 1 mentor. I think i'll need to repeat this trial sometime later with a player with more future potential.

[SackAttack]

Interesting study, Corey.

Have you tried comparing the difference between rookies and older players who aren't fully developed yet? Like R, 5, 10, and see how they benefit from mentors?

[cthomer5000]

Originally Posted by SackAttack

Have you tried comparing the difference between rookies and older players who aren't fully developed yet? Like R, 5, 10, and see how they benefit from mentors?

nope. this has been a lot of work, and for now i'm tired of it. If someone else would like to test some other elements, please feel free!

[Barkeep49]

This is the sort of work that seems to have only been rarely undertaken with FOF2k4 and is terrific stuff. Thanks CT.

[cthomer5000]

my conclusions:

• A Mentor has to be active to have any effect
• A player must be active (but not necessarily on the depth chart) to receive any benefit from a mentor
• Active backups will progress even without playing time in the presence of a mentor.
• Without a mentor, players do not progress at all without playing time.
• There doesn't appear to be a limit to the number of active players who can receive the benfits of a mentor

my thoughts:

The effects of mentors on starters have probably been vastly overrated. The more trials i ran, the more confident I became that the rate at which a player develops is largely specific to that individual player. One glance at the two different starting QBs in the major examples here shows us that one progressed faster whenever in the same role (be that starter or backup). Some players are simply going to develop faster than others - mentor or no mentor.

The primarily value in mentors may actually be in developing your backup players. It allows you to give them the effect of playing time (thought not at a 100% rate apparently), without them actually having to be in a game.

[Ben E Lou]

Originally Posted by Barkeep49

This is the sort of work that seems to have only been rarely undertaken published with FOF2k4 and is terrific stuff. Thanks CT.

Fixed it for you.

[cuervo72]

Originally Posted by cthomer5000

my conclusions:

• A Mentor has to be active to have any effect
• A player must be active (but not necessarily on the depth chart) to receive any benefit from a mentor
• Active backups will progress even without playing time in the presence of a mentor.
• Without a mentor, players do not progress at all without playing time.
• There doesn't appear to be no limit to the number of active players who can receive the benfits of a mentor

Now that you bring this up, I can say that I've had a QB mentor on my FOFL team since the start, and last year I had 3 relatively young QB's. Besides the guy who has started for 3 years, none have shown much improvement at all. I never bothered to think it could be because my mentor is usually inactive.

I drafted another QB project (the former starter is gone) and have the other two guys...it's a bit of a lost season, so I might just go with 4 active QB to a) have the mentor actually *do* something, and b) have a possible effect on all three.

[cthomer5000]

Originally Posted by cuervo72

Now that you bring this up, I can say that I've had a QB mentor on my FOFL team since the start, and last year I had 3 relatively young QB's. Besides the guy who has started for 3 years, none have shown much improvement at all. I never bothered to think it could be because my mentor is usually inactive.

I drafted another QB project (the former starter is gone) and have the other two guys...it's a bit of a lost season, so I might just go with 4 active QB to a) have the mentor actually *do* something, and b) have a possible effect on all three.

Yeah, you'd better believe this is going to change how i do business. I'm going to try to keep as many guys active as possible in positions where I have a mentor.

I weep for all the lost mentoring in my past seasons.

[cthomer5000]

Questions that need answering:

• Does mentoring effect veterans differently? (different rate or effect?)
• Does intelligence have anything to do with how strongly a mentor effects a players?
• Do affinities/conflicts have anything to do with how strongly a mentor effects a players?
• Are two mentors better than one? (early returns say "no" FWIW).

[cuervo72]

Yep. I had a wasted TE mentor in FOF as well, and I think an OG that was inactive half of the time.

No more trying to injure these guys to get them on IR either.

[wade moore]

Originally Posted by cuervo72

Yep. I had a wasted TE mentor in FOF as well, and I think an OG that was inactive half of the time.

No more trying to injure these guys to get them on IR either.

I think this is the biggest impact, particularly with MP Leagues.. in fact, you have to consider if you 'protect' your mentors a bit, just like you do your starters.. Plus, perhaps if you have say 3 QB's, a starter who is a rookie, and 2 developed QB's.. you may be less inclined to fight for a mentor?

I think this will also decrease the number of 'worthless' mentors on teams? The fact that you need them to be active, may make folks less inclined to sign mentors that have no business seeing the field... (in theory, if they are on your active roster, they may see the field even if not on your depth chart - particularly with like WR, CB, etc rather than RB and QB)....


Oh, and nice to see VPI use my spreadsheet ... I have to really wonder about the affinity/conflict factor... and again whether it matter if they are inactive... i'd be curious, if corey is up for it, to have this run with a different mentor that has an affinity with Jenkins...