HDV to ProRes422 Render Looks Crappy

Hello great LAFCPUG! Oh resource of unlimited knowledge! I've come to pay homage once again at the steps of your temple in great need of advice.

I have a project that I shot in HDV, and that's how I've got the project set up. I'm trying to render out some plates for VFX and don't want my VFX guy to have to deal with HDV compression, so I thought I'd just render out in ProRes for him to work on (no real need that I can see to render out uncompressed, it's not going to get any better and it's a huge files size difference - I'm having to upload the plates to Seattle). I set up my shots and rendered them out, but dropping them back into Final Cut to check them and the new plates look SIGNIFICANTLY more compressed than the HDV. The export settings don't allow me to choose pretty much anything but the Gamma Correction, which I tried both ways to no success, and even a render in HQ yielded identical results.

The compression is really bad, super choppy and big compression blocks that are certainly not perfect in the HDV source - but it's significantly better and now I'm confused.

I'm assuming that this is not the first time this has happened to someone - does anyone have any suggestions for how to get these outputs to look at least as good as the HDV?

I'm on the latest FCP version, fully updated, 8-core mac running latest OS.

Thank you guys for your generous help!!

Josh

Doesn't matter what you render HDV out as...ProRes, Uncompressed HD...the damage has been done once you recorded 4:2:0 HDV. That information is gone and there is no way to get it back. HDV is horrid for VFX work and green screen. That's why people avoid it when at all possible.


www.shanerosseditor.com

Listen to THE EDIT BAY Podcast on iTunes
[itunes.apple.com]

yeah, fair enough - I hear you. I'm just looking for the ProRes output to at least look as good as the HDV source footage - which isn't amazing, of course - but the grain structure is much better and smoother in the HDV. I'm just not sure why the ProRes seems to be compressing it further... Thanks for the response, tho! :-)

no real need that I can see to render out uncompressed, it's not going to get any better

So apparently you are seeing a real need to render out uncompressed after all ... certainly sounds like you need to give it a try.

yup, tried uncompressed and ran a difference test - pretty much the same result... does anyone think Compressor might do a better job that FCP's internal rendering?

Is there some kind of filter that's being applied that's causing this? I don't think it should be a lot worse than your source footage. Was your source footage damaged?

I've been taking HDV - in this case Canon's proprietary 1080f - editing in a Pro Res 422 HQ timeline and rendering out to Quicktime for mastering to tape for broadcast delivery.

In my case I have noticed "blocking" which mysteriously appears in the ProRes version and which is not in the original DHV recording.

Please let us know when you get to the bottom of this.

Best wishes,

Harry.

What version of FCP/QT are you running?

In your sequence settings, is render control set to 8 bit or render everything in high precision YUV? Duplicate your sequence and toggle that setting.

What sort of blocking? Can you export 1 second clips as both HDV and ProRes?



www.strypesinpost.com

Why not stick with HDV? If you export your project as HDV, the file will be small enough to send out to your post house. Let them convert the HDV to Uncompressed 1920x1080.

I use a Sony Z1U and FCP 5.04. I don't have problems with quality. Is ProRes supposed to be used with HDV in this way?

> Why not stick with HDV?

For VFX plates? Seriously?


www.derekmok.com

>I don't have problems with quality. Is ProRes supposed to be used with HDV in this way?

Yes, it is. I worked on 53 shows shot on HDV captured as ProRes, and I didn't have ONE issue with the quality of the conversion. All the glitches and crap I saw was on the HDV originals already. Conversion didn't add any more noise. The fact that yours is points to a problem...it isn't common.


www.shanerosseditor.com

Listen to THE EDIT BAY Podcast on iTunes
[itunes.apple.com]

Thing is, though, if I read the original post right, he didn't capture as ProRes -- he captured as HDV and then exported as ProRes. I haven't worked with HDV all that much, but I'm inclined to think that this makes a difference.


www.derekmok.com

Hi guys! Thanks so much for this discussion!

This has been a bit perplexing. I tried setting the sequence settings to render high YUV, came up with the same result. Yes, I captured at HDV and I'm now trying to export ProRes422. I've tried exporting in every higher format that makes sense, and I still end up with this resolution problem. I'd be happy to post stills for you guys to look at - is there a way to do that in this forum?

I'm on FCP 7.0.2.

I could maybe try bringing the clips into a ProRes422 sequence and see what that looks like...

Thank you guys again for helping me figure this out!

Josh

>he captured as HDV and then exported as ProRes. I haven't worked with HDV all that much, but
>I'm inclined to think that this makes a difference.

Not that I know of. The footage was acquired in HDV, so it doesn't matter if the conversion is done earlier or later in the chain, as capturing HDV via firewire does not incur a generation loss.

Some of us prefer to capture as ProRes through a capture card, because, firstly, an RS422 deck control is more reliable for recapture than Firewire. Secondly, ProRes is easier to work with than HDV. However, it takes up more storage.



www.strypesinpost.com

>I could maybe try bringing the clips into a ProRes422 sequence and see what that looks like...

Isn't your sequence already set to ProRes?


>I'd be happy to post stills for you guys to look at - is there a way to do that in this forum?

[www.lafcpug.org]



www.strypesinpost.com

No, my sequence is set to HDV as that's what I'm editing in. I'm exporting to ProRes from that sequence.

I uploaded two still from Photobucket, hopefully this works...

Test Still HDV

HDV converted to ProRes

Look particularly in the water near the drain, and on the edges of the white edges of the sink.

Let me know what you guys think - thanks again!

>my sequence is set to HDV as that's what I'm editing in. I'm exporting to ProRes from that sequence.

HOW are you exporting to ProRes? See, I capture HDV as ProRes, so I don't see the issue. But this...we need to know the exact steps you take.


www.shanerosseditor.com

Listen to THE EDIT BAY Podcast on iTunes
[itunes.apple.com]

That's quite drastic for a conversion. I'm seeing a gamma shift on top of that. How are you doing the conversion?

What you should do is, go into sequence settings, switch compressor to ProRes, render and export a self contained quicktime movie. (NOT via quicktime conversion).



www.strypesinpost.com

WOW...low light HDV...using this for VFX? Ugh...horrid.


www.shanerosseditor.com

Listen to THE EDIT BAY Podcast on iTunes
[itunes.apple.com]

Thanks Shane! ;-)
It's just a BG plate on to which I am compositing another element. It's for a web-only release, which is why the HDV doesn't concern me. It'll look fine embedded on the website.

Yeah, drastic is right! I'm glad it's not just me...

I have been Exporting using Quicktime Conversion from my HDV sequence - thanks for your advice, Strypes - I'll give that a shot shortly!

Thanks again, guys!

Please keep the information coming. I have had this problem in the past, so I really want to know the outcome.

Panting in anticipation, here.

Harry

Wow! Strypes ...

That makes an amazing difference (changing the sequence setting compressor to ProRes).

I can't believe it. I have exported yards and yards of material just going through the default Export To Quicktime setting.

There oughta be a law 'bout this.

Thanks a million, Strypes. I'm truly amazed at the difference in quality.

Harry.

>I have exported yards and yards of material just going through the default Export To Quicktime
>setting.

Export to quicktime setting? You mean Export>Quicktime conversion? If your sequence was rendered, Quicktime conversion will render off the rendered files. Quite bad if you already rendered to HDV.



www.strypesinpost.com

I used to simply go:
EXPORT>
QUICKTIME (The top option, not via compressor or anything)
and then you get dialog box asking for name and destination.

This is HDV. The SEQUENCE, SEQUENCE SETTINGS> COMPRESSOR WAS previously set to HDV by default.

I re-exported an old piece using your suggestion to change the SEQUENCE SETTING COMPRESSOR to ProRes 422 and the result was that there was no "blocking" on a very dark scene of someone playing a shiny black concert grand piano in a darkened room, whereas previously it looked bad by comparison.

I wish I had know this before. What an idiot I am. It was quite a shock.

Many thanks, Strypes. You really taught me a good on there. I owe you. One day I will come round and clean your car in return.

Best

Harry.

I can't swear to it, but I believe the way it works is this:

If you have HDV source material on a ProRes timeline, and you export a Quicktime movie (you know, the normal way, not with any additional nonsense on top), Final Cut decodes the HDV frames, then encodes them to ProRes, then writes them to disk.

If you have HDV source material on an HDV timeline, and you export a Quicktime movie in ProRes format using a conversion, then Final Cut decodes the HDV frames, then encodes them to HDV and caches them as render files, then decodes them again, then encodes them to ProRes and writes them to disk.

It's that second HDV encode that's killing you.

(Substitute any pair of formats for HDV and ProRes. Also, I reiterate that I don't know for a fact that this is how it works, but experiments have pointed in this direction.)

Hi all! I know I started all this and then disappeared - didn't mean to hit and run, but my RAID went down yesterday and won't be back up until Monday. I haven't even been able to try out Strypes advice! Harry's success has been incredibly heartening, however - and I second and third his many thanks and offers of car-cleaning on the next go'round!

Makes sense on that second encode, Jeff, even if it's NOT right, it sure SOUNDS right - and that's good enough for me!

Thanks again, everyone, my enthusiasm has been regained!!!

You guys rule.

Josh

I'm really curious about this.

I did notice that when you export with QT conversion, it is substantially faster if your footage was rendered (try it on a shot with multiple layers where rendering takes a substantial amount of time). Also, that if you keyed and tried to export to Animation with an alpha channel, the alpha channel will not go through if the sequence was rendered.

My two conclusions is that QT conversion will work off the rendered files if the sequence was rendered (as there is a significant difference in render times if the sequence was rendered), however it will render off the timeline bypassing compression to the sequence codec (as the alpha channel only goes through the conversion if the timeline is not rendered), but does it in 8 bit irregardless of codec or sequence settings.


But then, Josh, if both your stills are exported in the same manner, why am I seeing what seems like a gamma shift? It shouldn't happen because both codecs work in the same color space. Did you do any color correction to the clips? It is drastic because such a conversion will flunk a differential matte.

However I don't know what happens to HDV in an HDV timeline. I'm assuming that it follows the same process as any other codec, but re-conforms the GOP structure if you are exporting back out to HDV.



www.strypesinpost.com

For giggles then, I just grabbed some test images (XDCAM HD) and threw them in a native timeline, then exported using Export > Quicktime Movie w/ Settings: Apple ProRes 422 ... then changed the sequence compressor to Apple ProRes 422 and exported again using Export > Quicktime Movie w/ Settings: Current Settings

Compared the results of the two exports using a difference matte ... can't see any difference whatsoever, zilch, zero, nada ... they are identical.

So this tells us either:
1) the processing involved with 2 different export methods is identical (which is what I would have expected)
2) my eyes are failing (which is sadly probably true)
3) am an idiot and am just doing it wrong
4) it was a fatally flawed test due to my rubbish choice of test images
5) any combination (including none) of the above

I'll slide off back into my corner now

Did you gamma up the matte results? Differential mattes with ProRes can be hard to tell unless you gamma up the result.

I was testing ramp results using different exporting methods (quicktime conversion and qt export). I exported a ramp in 10 bit uncompressed SD and popped it into an 8 bit timeline, then exported them as 10 bit uncompressed using Quicktime conversion and QT export set to 10 bit uncompressed.

Here's a plugin for the plot scanline node in Nuke if anyone wants to test it:
[euqahuba.com]

With QT conversion it is always 8 bits (even if I was doing the export off a 10 bit timeline). However, if I'm exporting the clip from the ramp generator, there may be a gamma shift.

Exporting off the 8 bit sequence with QT export set to 10 bit Uncompressed, I still ended up with an 8 bit ramp (possible sequence compression), but the result produced less stair stepping than an 8 bit ramp export (8 bit sequence, render, export current settings). However, when I exported off a 10 bit timeline with QT export set to 10 bit Uncompressed, I had a 10 bit result.



www.strypesinpost.com

I went a little further, and ran the clips through the other export functions- MM recompress, nest and export, send to Compressor (which produced 8 bit values when coming off an 8 bit sequence, possible sequence compression), and finally popped in the SCQT 10 bit clip straight into Compressor.

All of the exports preserved the 10 bit information from the source, except during "send to Compressor", where 10 bit clips in an 8 bit timeline had values rounded off to 8 bits, but 10 bit sequences preserved the color information.


To sum it up, QT Conversion is indeed sketchy as it throws away 10 bit information irregardless of whether the sequence is 8 or 10 bit, so it shouldn't be used for round trip workflows. MM-recompress does preserve the bit depth of the source clips even if the sequence is set to an 8 bit codec, so it can be good for getting clips into the same codec for round trips (eg. Color).



www.strypesinpost.com