Simple ProRes proxy question

This is probably a simple concept, but I just wanted to confirm it.

Take a clip from my Canon 60D - convert it three ways (XDCam EX, ProRes Proxy, ProRes 422).

Open in QuickTime, look at properties:
XDCam - data rate = 36.62 mbps
ProRes Proxy - data rate = 44.11 mbps
ProRes 422 - data rate = 147.70 mbps

According to this, the data rate of PR Proxy is higher than XDCam EX...is it correct to assume that the overall quality of PR Proxy is higher? I'm not sure if there's anything else inside the PR Proxy codec that is affecting quality, other than just the data rate.

I'm trying to tweak my workflow. My computer/hard drives aren't fast enough to make using ProRes 422 all the time practical. For instance, this above mentioned clip is 30 seconds...putting a 3-way color corrector filter on it and rendering the clip takes 30 seconds in ProRes Proxy, 60 seconds in XDCam EX, and 120 seconds for ProRes 422.

Honestly, what I'm asking is if I could get away with leaving it in ProRes Proxy and having it look better than XDCam EX, but giving myself the option (depending on client's budget/time constraints), (and I'm not sure of the proper terms for this) to convert my original footage to ProRes 422 and re-link everything on a timeline, creating a PR 422 project.

Thanks!
Casey

Both Proxy and ProResLT should only be considered as offline CODECs - use 422 or 422HQ or 444 if you want to retain the image - even from 8bit XDCAM EX.

I'm currently editing a feature with ProResLT offline files and I would definitely NOT recommend it as a final as you have very little picture detail to play around with on certain types of shots. Most notably dark and/or subtle gradient shots.

You could get away with it but I'd do a proper image test and really push the image with the 3-way CC in all 3 codecs so you can see where they fall apart.



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]

Thanks Ben,

It's all about balance, though. Say I have 10 hours budgeted to edit an entire wedding (clients want more, but also want to pay less, but that's another conversation altogether!) into a 10 minute edit, it hasn't been practical for me to do it in PR422 because of the added render times.

I had been using the Sony Z1U for years and had mastered that workflow, but now am getting into Canon DSLRs, which require more work and time to deal with. It wouldn't make sense to convert the h264 files into HDV, nor does it make sense for me to use PR422 as I mentioned earlier, so these past couple projects, I have been using XDCam EX as my codec...trying to strike a balance with quality and overall edit time (from ingestion of footage to handing the client a DVD). I was trying to find out if Proxy (or LT) made more sense than XDcam in terms of speed and quality...also, in terms of consistency sake, if I had a larger budget, I would have the option to do it in Proxy and switch it to PR422 later. Does that make sense, or is there a better way I could be doing this?

Thanks!
Casey

Casey,
Don't you find the export time of XDCAM HD to be longer than a ProRes export? I am working on an all XDCAM HD show and the 55 min rough cut renders are killer. I would convert all my source footage to ProRes if I had the drive space. I can see how the conversion of the Canon files to your editing format would be the longer time investment than the 10min finished product but right now I would do anything to avoid cutting with native XDCAM footage.

If you wanted to try a ProRes LT to HQ workflow I would look into Bouke's utility for relinking files by source and timecode. It would help you avoid having to convert anything more than what's in your final edit. (The 5 seconds of the best man throwing up in a planter from an otherwise unused 10 minutes of waiting around footage)

ak
Sleeplings, AWAKE!

Actually, I was using XDCam EX, not HD. I ran a bunch of tests, and with my Mac Pro, which is 5 years old, would now be considered old and slow, and my hard drives aren't fast enough either. Using HDV footage as my baseline, working with XDcam EX took maybe 5% more time to render effects, exporting, etc., so that was pretty good to keep up with the workflow I have been accustomed to. Using ProRes422 literally took 2-3 times longer to do anything. Like I said above, rendering a 30 second clip with 3-way color correction on it, took 30 seconds to render in ProRes Proxy, 60 seconds to render in XDCam EX (HDV would have taken approximately 55 seconds), and ProRes 422 took 120 seconds to render.

I know I don't have the latest and greatest hardware setup, but it's what I have to work with, at least until clients start paying more and wanting less!

I'm not familiar with Bouke's utility...I will have to look that up.

Thanks!
Casey

It sounds like FCPX might actually be a better option for you as you could just edit all the formats together on the Timeline and chuck out out a final edit. That is if you have a Mac with the required GPU.

I use XDCAM EX on a ProRes 422 Timeline if there is no option/time of pre-transcoding the footage. Otherwise I get it all ProRes 422 as in my experience it is the most stable and fastest route.

I am shocked that you find XDCAM EX faster to render than ProRes 422! That doesn't seem quite right - will investigate on mine as I haven't run tests for a while.

What specific Mac set up do you use?



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]

Yeah, I've heard that before! A few years ago, I posted on here quite frequently and tried to figure it out why ProRes was so slow for me, but with no definitive answers. The best I can figure is my hard drives are too slow.

I am using a Mac Pro 2 x 3 GHz Dual-Core Intel Xeon with 10 GB 667 MHz DDR2 FB-DIMM memory, running OSX 10.6.8. I am using the latest Final Cut Studio software as well. The hard drives I have are 3 Western Digital MyBook Studio 1TB drives connected via FW800. I know they're not the best drives, but they're what I have to work with. I'm not editing Transformers...I am a wedding videographer who has been editing since the days of A/B roll and is just trying to keep up!

Thoughts?

Casey

I think this article from Larry Jordan explains why I am having trouble with ProRes:

[www.larryjordan.biz]

Look at the two tables referring to hard drive speeds and video data rates by codec.

According to this, HDV has a data transfer rate of 3.75mbps, XDCam EX is 5.2mbps, ProRes Proxy is 5.6mbps, and ProRes422 is 18.1mbps.

So, in my world where the HDV codec has been perfectly acceptable for 6 years, XDCam or ProRes Proxy would actually be steps up, while ProRes422 is a giant leap up.

Given that my hard drive speed is not very good...something that could be upgraded in the future, but for now, it would make more sense for me to transcode my h264 footage to ProRes Proxy...given the fact that the rendering times are twice as fast as XDCam, the quality is slightly higher, and that Proxy uses i-frames instead of GOP compression.

Also considering that 98% of my projects' final output is a standard definition DVD, I think I would be all right with this workflow, until I can get some more robust hardware so I can take advantage of ProRes422. I can always do this offline and finish in ProRes422 for higher end productions if necessary.

Casey

Datarate should only really be compared with the same codec with the same frame-rate, pixel size and colour depth.

For instance H.264 is considered better at lower datarates than MPEG-2.

Also not all encoders are equal and the quality varies between manufacturers and versions.


Besides I ran a test and figured out why you are having such bad render times...

I was basing my initial suggestions on that fact that I always set my final renders to 10bit High Precision YUV and indeed you will have ProRes Timelines automatically set to this too.

However an XDCAM timeline (HD and EX) with always be set to 8bit YUV rendering unless you change it to High Precision or plonk them on to a 10bit Timeline.

Rendering in 10bit massively increases the processing so you simply need to turn it to 8bit if you are only producing 8bit video from 8bit source and don't need the subtlety of 10bit graphics, grads and colour correction.

So here is my basic timing findings based on the mean average of three render passes of an original 2 minute clip of 1920x1080i50 10bit 422 HQ 25 fps with a 3-way Color Corrector added before rendering.

The Mac used for the testing is an 8-core 3 GHz Xeon MacPro 3,1 with 32GB RAM, Nvidia GTX285 and the Media Drive is Reading and Writing to a 750MBps RAID 6. Running Mac OS X 10.6.8 and FCP 7.0.3


I have included all the ProRes vs the HD XDCAM flavours so you can see the real difference. Note all 10bit is High Precision.

A. 2 min ProRes 422 HQ 1080i50
8bit YUV 1'02"
10bit YUV 2'37"

B. 2 min ProRes 422 1080i50
8bit YUV 0'55"
10bit YUV 2'30"

C. 2 min ProRes 422 LT 1080i50
8bit YUV 0'52"
10bit YUV 2'26"

D. 2 min ProRes 422 Proxy 1080i50
8bit YUV 0'46"
10bit YUV 2'22"

E. 2 min XDCAM 422 CBR 1080i50
8bit YUV 1'48"


F. 2 min XDCAM CBR 1080i50
8bit YUV 1'09"


G. 2 min XDCAM VBR 1080i50
8bit YUV 1'14"


H. 2 min XDCAM EX VBR 1080i50
8bit YUV 1'34"


So you can see if you converted your footage to ProRes LT and rendered at 8bit quality your render times would be cut by almost two thirds.

I had a look again at the ProRes Proxy Quality and as I said it should ONLY be used for Offline as the artefacts that are introduced around details are unacceptable for a professional output even if you are eventually ending up at Standard Definition.

XDCAM EX and ProRes LT are not entirely indistinguishable but good enough so that would be the option for now.

See Examples:

Original ProRes 422 HQ: [www.loudandfast.co.uk]

ProRes 422 LT: [www.loudandfast.co.uk]

ProRes 422 Proxy: [www.loudandfast.co.uk]

XDCAM EX: [www.loudandfast.co.uk]


If you copy the examples to your HDD and Zoom into the "C" of "PRODUCTIONS" you can see the compression artefacts more clearly certainly on the ProRes Proxy file.

And if you overlay the Original 422 HQ on each of the others and do a Difference composite/layer mode in FCP/Photoshop then you will see where the picture deviates from the original shows up as colour and where it doesn't as black.



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]

WOW!!

I think you hit it!!

I never noticed that it went to 10 bit high precision YUV when I tried ProRes.

I am running another test with a 30 second clip, this time, I changed it to 8 bit YUV:
XDCam - 1'35"
ProRes422 - 1'10"
ProRes Proxy - 0'40" (obviously faster...doing what it's intended to do)
ProResLT - 0'55"
HDV - 1'08" (what I've been using the past 6 years)

YOU ROCK!!!!

This changes things for me...this puts ProRes422 into the realm of possibility now...now I have to decide whether to use Proxy as an offline edit, LT or 422.

I definitely see the difference between Proxy and the others in the stills you linked.

I can also obviously see the difference in speed between your computer setup and mine...based on the Larry Jordan article, should I see results if I simply switch from FW800 to eSata drives?

Casey

Quote

I can also obviously see the difference in speed between your computer setup and mine...based on the Larry Jordan article, should I see results if I simply switch from FW800 to eSata drives?

Not really for simple renders with only one layer of video.

Your FireWire 800 should be giving you at least 35 to 70 MBps maybe more depending on the chipset and HDD but maxing out well short of 100MegaBytes per sec (800Mbits per sec).

An eSATA connector might give you as much as 120MBps+ when using the new 3TB Hitachi 7K3000 drives on 3G eSATA (not tested mine on a 6G yet).

You can download the AJA system Test: [www.aja.com] and test your HDD speeds.

The big bonus from faster drives comes when opening projects, importing and exporting and general scrubbing performance and realtime playback of multiple streams and if you are rendering complex multiple source AV elements.

If you want to increase render performance you are looking at a new faster Mac and with some filters/fx a faster GPU.

I would always advocate using a fast RAID for editing though. The general increase as mentioned above can drastically reduce the time you spend with the mundane elements of an edit. Not to mention the redundancy you will have with RAID 5 or 6.

There is an old article of mine on RAIDs in the SuperMag from last year - it's not quite out of date but doesn't cover the new Thunderbolt or 6G SATA tech, but still is relevant.

You can download it for free here: [supermeet.com]



For instant answers to more than one hundred common FCP questions, check out the LAFCPUG FAQ Wiki here : [www.lafcpug.org]