It is kind of annoying, especially the hit you take for primary rivalry victories not counting. But as long as you don't have a one year contract (why do those even exist?) you should be safe.
In my last dynasty I was fired for not finishing above 4th place because two teams ahead of me played more conference games than they should have and it was statistically impossible to overtake them despite having the same amount of conference losses and victories over both. I haven't touched dynasty since that happened.