I'd do it and have a couple of times (moved from one comfortable position to another that was a better fit/more interesting). The last time I did it 3 years ago, we had 2 kids, the previous time I moved I was a week from us having out 2nd, and the time before that, it was a week before we got married. I told my wife that as long as we don't have another kid, I won't randomly quit my career again.
These days I would never be willing to drive more than 10 minutes or more to work. I use to drive 30 minutes each way and then realized that it was an hour per day, 5 hours per week, 50 weeks per years and that I was basically working nearly 6 weeks for free for the pleasure of working for that company. Now I do my own thing and work at home most days. When I go to meetings, I do them around 10 or 2 so traffic is way less. Since I made the move 3 years ago, I have never and could never go back (and I'll be honest, every time I made one of these 'risky' moves, it has opened up opportunities that made the move much better financially so I have lucked out). Good luck!