One method is to put a version of the clip, with the Time Remap removed, in the timeline. That way at least you preserve the duration and In/Out points. But that's not 100 per cent, either. If you're really happy with the Time Remap, you might want to just export that as a full-quality self-contained QuickTime movie and then cut it back into the timeline.
This is why I never use Time Remap -- whenever I need speed changes, I razor the clip and change each piece's speed individually. Gives more control, too.