I understand what you're saying but I don't agree that the influence of judging is identical between a sport like gymnastics for instance, vs a sport like football.
While no one is denying the perhaps out-sized influence of officials in any sport in enforcing the rules, there is a world of difference between judges interpreting enforcement of the rules and judges directly declaring winners/losers. Continuing the football analogy, if the sport were run just like gymnastics or figure skating, the Giants and Cowboys would get together and play 4 quarters, presumably there would be some scoring in there and some nice plays (although with those 2 teams, who knows?), and then at the end of the allotted time, the referees would huddle up and take a poll of each of them asking them who they think scored higher? Then they'd tally up the votes and declare the the Giants with 7 points and the Cowboys with 9 points, so the Cowboys would win. That's literally what happens with gymnastics or figure skating.
Note: I am not saying that any sport is more or less legitimate than any other sport. I'm saying that just because all sports have some element of judgment in the enforcement of rules does not mean they are all equivalent in terms of how heavily those judgments factor into declaring winners/losers (directly vs indirectly).
Comment