Remote play got a real kick in the rear with the recent firmware update, and I now find the Show very playable when directly connecting to the PS4 and enabling 60 FPS. Internet connection (long-distance, in other words)....not so much.
Make sure that your remote play settings screen on the PS4 has "Connect directly with PS Vita/PSTV checked, and that the setting on the Vita is set to High/60 FPS. I was FLOORED with how much better the response time was when I first tried this after the update.