Bradley, I share your pain. This week I started experimenting with burning my first HD DVD's although via a standard DVD 5.
First I tried it using MPEG2 with the stock setting for that in DVDSP4. Worked fine, I had some menu problems but the media itself played fine. The MPEG2 encoding wasn't that great and there was alot of blockiness and noise in dark areas. Overall it looked a little soft to me. The original media was 720P and I was keeping it the same, not scaling to 1080i.
Not caring for MPEG2 I encoded again with H.264 using the stock 60 min setting in Compressor.
Compressor finished and I was able to play the sequence fine in Quicktime player.
However, when I dragged it into the assest window of DVDSP4 it did exactly the same thing yours did. It started loading and had the progress bar till about 2/3 way through and then it bailed with no indication as to why. It did not crash DVDSP, just terminated the process.
The audio came in fine as an AC3 file.
Without some kind of error message I am at a loss.