I'd be doing the exact same thing if I was trying to hit from that camera. The only game I was ever able to do it well in was Bases Loaded - and that mostly had to do with up,down,left, and right being the only swing aiming controls, and the ball coming in at half the speed that it does in today's games.
I have great respect for anyone who can actually hit well from that camera angle, because I sure as hell can't.