Sigh, it wasn't the light. I got my hopes up thinking it was because the game was swinging better but it started up again (I should have tested it out more before declaring that was it). Figured out it might just be a bad game disk. Talk to xbox support and we where trying everything. I decided to go into tiger woods demo and see. And sure enough in the demo no problems at all. Did the demo about 8 times in a row just to make sure, and it was reading my backswing correctly. Went back to the finish TW game and it just doesn't read the backswing. So going to return it and get another disk. I'm hoping it's just a bad disk.
Sorry to hear some are having difficulties. A week in and I am having lots of fun with the Kinect on the course - although menu navigation is awful with the Kinect. Luckily, it's easy to pick up a controller to navigate before on the course.