re: ProRes 422, 422 HQ and 4444 vs. Uncompressed HD

There have been many discussions about which codec to use for maximum quality, and I think the prevailing opinion is that even ProRes 422 is good enough for capturing HD. I'm trying to convince a producer to start using ProRes for HDCam captures rather than Uncompressed 10-bit HD; since they upload raw footage all the time, the transition would mean files that are three to four times smaller, drastically reducing upload/download time. Any opinions on this suggestion?

I captured a set of HDCam footage (HDCam tape, not SR) twice, using both Uncompressed 10-bit HD and ProRes 4444. A different matte seems to yield no discernible difference between the two captured clips.


www.derekmok.com

Since HDCAM is 8-bit, the Uncompressed 10-bit HD or ProRes 4444 (or even ProRes 422 (HQ)) capture is overkill, I would say...

Probably, ProRes 422 would work well for capture and storage of media files.

I would, however, use ProRes 422 (HQ) for the editing/timeline codec, or ProRes 4444 if you need alpha channel support.

Of course, the best thing is to do some testing and make sure everyone is in agreement. A few simple tests to show the viability/quality of ProRes to the producer will pay off in better performance during the edit (compared with uncompressed codecs)...


-Dave

Bear in mind that the compression applied to ProRes 422 (HQ) and ProRes 4444 is exactly the same. The difference between them is that one is 4:4:4 (duh) and 12-bit/16-bit, while the other is 10-bit 4:2:2. But in terms of how the images are compressed and what results you get after compression, they're exactly the same.

The difference in data rate between 422 and 422 (HQ) is non-trivial; it's about 145 Mbps versus about 220 Mbps, or about 50 percent more. If you use 422 (HQ), you're virtually guaranteed never to see a single compression artifact after the first generation, but you'll require half again as much storage space and half again as much time for network data transfers. If you use 422, you're almost certain never to see a single compression artifact, and you have less data to deal with.

Question is, can you live with the outside chance that you might, very rarely, see the occasional compression artifact? If so, then screw it, use 422. If you must never have a compression artifact, then it's worth it to bump up to either 422 (HQ) or 4444, depending on whether you need the extra resolution that 4444 gives.

I'd show the client a test. This is ten seconds of Uncompressed at that takes up this much space and time, this is ten seconds of Pro Res HQ that takes up this much space and time, this is ten seconds of Pro res that that takes up this much space and time. Can you see the difference? Which do you want?

I'd have tested HQ. Very VERY few people need 4444. People will now think that their Canon 5D and 7D footage will be converted to 4444 when captured thus. Or DVCPRO HD. I had to REALLY talk a DP out of going from DVCPRO HD to ProRes 4444.


www.shanerosseditor.com

Listen to THE EDIT BAY Podcast on iTunes
[itunes.apple.com]

I had to talk an editor out of going from XDCAM EX to 4444. And truth be told, I wasn't even all that sure myself. Her disks are ten times bigger and faster than she needs, so really ? why not? You know it's wasteful, I know it's wasteful, but if she doesn't mind everything taking a bit longer, what's the harm?

In a real workflow, of course, waste cuts into your profit margin, or to your time off to spend with your kids, which I guess are two sides of the same coin.

Quote
Jeff Harrell
I had to talk an editor out of going from XDCAM EX to 4444. And truth be told, I wasn't even all that sure myself. Her disks are ten times bigger and faster than she needs, so really ? why not? You know it's wasteful, I know it's wasteful, but if she doesn't mind everything taking a bit longer, what's the harm?

The "harm" is called willful ignorance, which is always unacceptable. There's a sea of difference between making a deliberate, informed decision and the facile "more is always better" decision. Argh...


Everything is a real workflow, so I would agree with Jeff that waste is not good. Ignoring the technical side of editing, which is often quite tedious, is nevertheless a risky venture...

-----

I do think that the discussion needs to be clear about the difference between capturing content into a particular codec and the codec chosen for editing and mastering (and archiving). The first is about maintaining what was "recorded/filmed" and the others are about preserving quality after filters, etc., are applied (and to help prevent generational loss and other degradation introduced when transcoding/compressing to other formats).


-Dave

>A different matte seems to yield no discernible difference between the two captured clips.

You need to amp up the gamma to see the difference when you're looking at ProRes. But of course, ideally you'll do it outside of FCP, since many of FCP's blend modes are 8 bit (surprise, surprise). Skip 4444, because you don't need the alpha channel or the additional 0:2:2 chroma subsamples, as the SDI signal from the HDCAM deck is 4:2:2. 10 bit Uncompressed is the best you can go, but if you need to upload and download, you may really want to use ProRes or ProRes HQ.

My case was different from Derek's. I tried to talk someone into using ProRes HQ instead of Uncompressed 8 bits, simply because it saves a lot more space, and you do not truncate your color information to 8 bits. But there are people who swear by their workflows so I just let them be.



www.strypesinpost.com

I was always under the impression that HDCam was 3:1:1. Am I missing something here?


www.derekmok.com

"3:1:1" is shorthand for 4:2:2 at 1440 samples per scan line. It's equally correct to say that HDCAM (not SR) is 1440x1080 4:2:2.

The HD-SDI is always full raster 4:2:2 and 10 bits.



www.strypesinpost.com

Why is ProRes 422 which actually compresses to a higher degree than DV25 so successful on 1920x1080 material?

First I'd better explain the comparison with DV25. DV25PAL 50i is just 25 Mb/s but that is just for the 720x576 frame. DV25PAL 50i, adapted to the 1920x1080 frame, would need to be 25*5 = 125 Mb/s. And DV25PAL 50i yields 4:2:0 chroma subsampling. If we further adapted it to 4:2:2 it would need to be 125*1.333 = 167 Mb/s. For comparison, the data rate for ProRes 422 1080 50i is 122 Mb/s and for ProRes 422 HQ it's 184 Mb/s. There have been big advances in interframe compression since 1995 but both DV25 and ProRes are purely intraframe DCT-based compressions where the science hasn't changed much, so I think these data rates are indicative of the compressions. Let's be careful about what exactly is being compared. The claim is that if we had a sharp-as-possible 720x576 50i uncompressed 8-bit 4:2:0 video and we compressed this to DV25, then the degree of image degradation would be similar to what occurs when a sharp-as-possible 1920x1080 50i uncompressed 10-bit 4:2:2 video is compressed to ProRes 422. For this comparison, viewing distance is adjusted so one pixel looks the same size in each case. The experiment can't be carried out exactly because 720x576 50i uncompressed 8-bit 4:2:0 can't be found. Black & white video can be used, but then the 1.333 factor should be omitted from the calculation.

ProRes 422 and even ProRes 422 HQ are pretty severe compressions. 1920x1080 50i uncompressed 10-bit 4:2:2 video streams at 1106 Mb/s. Getting this down to 122 Mb/s or 184 Mb/s, by purely intraframe compression, is something. You might believe that Apple performed miracles with its ProRes compression, so these data rate calculations and comparisons are irrelevant. I'm inclined to believe that the reason ProRes looks so good with 1920x1080 material is that video optics are seldom good enough to make sharp-as-possible images on that many pixels, especially on smaller sensors. If this is true, then we should expect ProRes 422 to be less successful when the 1920x1080 footage originates straight from graphics software or from a reduction of higher resolution imagery.

Even if ProRes 422 turns out to be ho-hum intraframe compression, with visual quality appropriate to its (slightly variable) bit rate, ProRes 422 is a godsend to FCP users. This is because FCP7 (with OSX 10.5.8, at least) manages to efficiently use all the cores of a MacPro when rendering in ProRes 422 codecs, but not in most other codecs.

Dennis Couzin
Berlin, Germany

I wrote a post on ProRes a few months ago:

[strypesinpost.com]

I included DV25 into the mix (it's right at the bottom of the post). Since it is a pure luma ramp, the difference isn't about the 4:2:0 chroma subsampling.

ProRes uses DCT based compression, but there is a LOT more compression going on than just DCT, and that is why ProRes is a slower codec than DV25 or even DV50. As far as quality is concerned, ProRes is 10 bits, DV is 8 bits. In some further tests that I did, I realized that ProRes compresses more heavily on the colors than on the luma channel. Black and white gradient ramps will usually return a black image with ProRes HQ even after a few generations, but the same can't be said about the color.


>we should expect ProRes 422 to be less successful

Okay, I do occasionally encounter issues with ProRes changing colors/gamma on render in FCP. I had this animation clip recently, and all I did was crop a frame, and the color in that frame changed (it wasn't just gamma, but the hue also shifted). The fix was to convert the video to Uncompressed 10 bits.

For the difference matte tests, everything is transcoded to 10 bit uncompressed, because AE access ProRes through QT, while it access the Uncompressed codecs natively. That causes some differences to show up. Difference matte tests in FCP is less reliable, because for most of the blend modes, FCP truncates data to 8 bits.



www.strypesinpost.com

Original comment removed. I was puzzled by the "8 Bit Source" graph in strypes' 27 June report. It shows a staircase where every sixth step has double height. That can't be right and doesn't reappear in the curves showing the 8-bit source accurately transcoded to 10-bit. So that 8-bit source graph must have undergone an unhappy scaling. The danger in graphical based technical arguments.

Dennis Couzin
Berlin, Germany

strypes, I confirmed your 27 June findings on ProRes SQ vs. DV25. That is, I rendered an uncompressed 8-bit gradient into each codec and from these exported to 8-bit uncompressed for examination. I know the ProRes should have been exported to 10-bit uncompressed, but I'm not well set up for examining those files, and it's easy to examine 8-bit uncompressed files. Indeed ProRes SQ reproduced the original gradient beautifully while DV25 introduced messy artifacts. The artifactual values are never beyond ±1 of what they should be, but they are disorderly, so they will be visible.

DV25's treatment of the gradient is embarrassing. Its DCTs needed better tidying. I retract my statement of there being little development in DCT compression since 1995. But tests with gradient images is a small part of evaluating a compressed codec. What happens to edges and lines and fine textures? How well is sharpness preserved and how much ringing and other jazz accompanies detail? I'd like to put one bright pixel in a surround of dark pixels and see what the codecs do, etc. If you think the ProRes evaluation you began in June should be continued, I'm preparing software to aid examination of 10-bit uncompressed 4:2:2 files.

Dennis Couzin
Berlin, Germany

I had two gradient ramps. One rendered in 8 bit Uncompressed and another rendered in 10 bit Uncompressed. The main point of the article was to look at what HQ and SQ does in relation to an 8 and 10 bit source, because there was a statement made that HQ "reinterprets 8 bit material as 10 bits", and therefore HQ should not be used for material other than RED or the higher end cameras. That was against my understanding of ProRes, where the main difference between the different ProRes flavours is that with HQ, you have less aggressive data compression, so I decided to test that out, and I found out that that is not true.

The plot scanline node in Shake lets you know if your source is 8 or 10 bits. A true 10 bit ramp will have a less stair stepped line, and a 16 bit ramp will look even finer on the plot scanline node, but 16 bits is much harder to tell, unless you zoom in. I don't expect it to plot the points too accurately, just accurate enough for you to tell if a ramp is 8 or 10 bits for the experiment.

What is interesting was that when the 8 bit ramp that was transcoded to ProRes SQ, and then converted back to 8 bit uncompressed, the difference matte returned a black result. Is the difference due to how AE reads 8 bit vs 10 bit Uncompressed in floating point space in RGB? Perhaps. But consistently, between SQ and HQ, the difference in 1 generation is fairly marginal. Across multiple generations, SQ deteriorates slightly faster than HQ.

As I mentioned in the article, that wasn't a definite test on generational quality, which requires hard edges, contours and colors and they should be run more than 1 generation. So I did another test but this time with colors, hard edges and chroma gradation and transcoded it 9 generations, but because this time I did the rotation in the motion tab in FCP instead of doing it within the image generator, the rotation algorithm was producing some aliasing. However, that didn't prove anything we didn't already know. ProRes HQ was lossy, ProRes SQ was lossier, but they were by far much better than any other codec such as DvcproHD and XDCAM HD.

I believe that there are more compression components than just DCT compression (eg. in H.264, you have an option of CABAC and CAVLC entropy coding), bit depth and chroma subsampling. We don't even know for sure that ProRes is DCT based (just that the artifacts produced on generational tests indicate beyond a shadow of a doubt what seems to be blatantly obvious DCT based compression artifacts, as opposed to a wavelet based compression).

There are definitely some interesting comparisons that I have yet to make (eg. between DNxHD and ProRes). However, in doing these tests, I have realized that there are other factors that come into play, and these affect accuracy more than the difference in compression between ProRes and Uncompressed.

One example is that when AE is accessing codecs such as ProRes, you get a different result than if you were to convert that ProRes image into Uncompressed 10 bits, which AE can access natively. I believe that is due to AE requesting for a Quicktime decode, and AE then converts the Y'CbCr into a Rec 709 profile in RGB channels in floating point, or AE may be asking for a Quicktime decode to ARGB, which it then bumps up into float.

Another example is when AE renders out a gradient as ProRes, the resultant gradient plotline seems to be dithered and possibly 8 bits, which does not happen when you render it out as a 16 bit tiff image sequence which produces a clean gradient plotline. I believe this happens because Quicktime is handling the RGB to Y'CbCr conversion.

Yet another interesting point, is when you have gradient in a 16 bit tiff image sequence, and you wrap it up in Quicktime (not transcode), the resultant image when you access the QT file looks to be 8 bits, despite there being no transcoding process. The most likely explanation is this:

Quote
http://www.reduser.net/forum/showthread.php?t=33640&page=3
Sloppy/old implementations are 8-bit only, as the original specification didn't include alternative pixel formats, resulting every tool having to fallback to the 8-bit RGBA mode with that broken 1.8 gamma (at random times.) As QuickTime came to the PC later, more QT implementations on the PC support deep pixels than from QuickTime on the Mac. Tools like Autodesk Combustion and Adobe After Effects can support QuickTime at 16-bits per channel (format 'b64a') (I think Scratch just added it, also Nuke, but not Fusion -- EyeOn, catch up), whereas most tools on the Mac seems to be 8-bit only for QT sources. For the tech. side of things is bizarre that the Mac every made as a production suitable platform (Avid did it spite of QuickTime.) FCP has support for some deep formats, but only YUV formats like 'v210' (10-bit 4:2:2) and 'r4fl' (32-bit float, yuva 4:4:4:4.) While FCP has this 32-bit float, most of its filters don't support it, the product is shockingly unpredictable when it choose to deep modes (although rendering is stable.) Color doesn't support Quicktime at all, and accesses ProRES and others, only through an internal Apple API, bypassing QuickTime. There are very few post production solutions that reliably maintain deep pixel precision using compression, one reason why Stu Maschwitz's Rebels guide recommends uncompressed AE finishing, as DPX is mostly commonly 4:4:4 10-bit RGB. CineForm has fought this battle and won in places to allow for deep pixel compressed workflows, Adobe is good at supporting deep pixel API (beyond the 16-bit QuickTime support it has), FCP works, and many Windows tools are pretty good with QuickTime. With ProRES, CineForm and Avid DNxHD now proving compression and deeper color is compatible, we are seeing more tools improve there support, although slower than you want, although I doubt QuickTime Player will ever get there.

But let me know, I am definitely curious about whether color and pixel accuracy is maintained when you go through software such as After Effects, Nuke or Combustion, and which workflow will accurately maintain color and pixel accuracy within broadcast safe (most compressed codecs really mess up outside of broadcast safe). Currently it seems that Uncompressed 10 bits (aka "v210" may be more reliable when you are round tripping outside of the FCS world, as most softwares can directly decode it, although I doubt you are likely to visibly see that much of a difference.



www.strypesinpost.com

strypes, was your report really a response to this insane statement from the COW FAQ:
"With the HQ version the CPU is actively interpreting all 256 levels of grey on encode but passing that back out re-interpreted with all 1024 levels on output, that is one HUGE Mathematical Processing task"? The COW writer seems not to understand binary numbers. An 8-bit scale can be converted to a 10-bit scale by simply adding two zeros to the end of each 8-bit number. This will exactly fit the legal 16-235 scale upon the legal 64-940 scale, and well enough fit the tolerated 1-254 scale upon the tolerated 4-1019 scale. The COW writer speaks of "all" the 10-bit levels being used to interpret the full set of 8-bit levels, but this is ambiguous. All the 10-bit levels are available to interpret the 8-bit image but they won't generally all be used. For example, an 8-bit image comprised of consecutive bands several pixels wide like so: 16, 126, 17, 127, etc. includes every 8-bit level from 16 to 235, but there's no reason why a 10-bit codec would convert the 220 bands in the original image into 877 (or whatever) bands unless that codec is highly compressed and greatly blurring edges.

If an 8-bit image consists of a beyond detailed random assignment of values to pixels, then a ProRes HQ transcode will use all or most of its available 10-bit values. Only an uncompressed 10-bit codec can avoid this.

This points up a problem with your use of gradient wedges to test depth precision in codecs. The jiggles you find around the staircase curves are not the result of lack of depth precision per se but lack of sharpness and some attendant spatial artifacts. If you made big patches instead of very fine bands of each level and examined what happened at the centers of the patches you'd probably find perfect bit precision from all the codecs.

The scary lesson of your post is that it's not enough to evaluate a codec itself because every piece of software (and hardware) that will work on the video can encode or decode differently.

Dennis Couzin
Berlin, Germany

What do you use to get pixel values from a video frame?


>but there's no reason why a 10-bit codec would convert the 220 bands in the original image into 877 (or whatever)
>bands unless that codec is highly compressed and greatly blurring edges.

Yes. It makes very little sense for a codec to alter an 8 bit source image to include new levels of gradation when it does not have to. In fact that would work in contradiction to the purpose of an intermediate post production codec, because it would not preserve the integrity of your source data.


>If you made big patches instead of very fine bands of each level and examined what happened at the centers of the
>patches you'd probably find perfect bit precision from all the codecs.

True, if you stand far enough from the screen, you won't see the difference between any form of video encoding.


>The scary lesson of your post is that it's not enough to evaluate a codec itself because every piece of software (and
>hardware) that will work on the video can encode or decode differently.

Yes. The differences are usually due to the different routes each codec goes through, and that was why I went back to Uncompressed to do the difference mattes in AE.



www.strypesinpost.com

strypes Wrote:
-------------------------------------------------------
> What do you use to get pixel values from a video frame?

I've been using SweetScape 010 Editor to view binary files. 8-bit uncompressed 4:2:2 files are quite transparent. Starting at a left edge of the image, the file reads Cb, Y', Cr, Y', Cb, Y', Cr, etc. row after row, from left to right through the image. Each triple <Cb, Y', Cr> corresponds to a chroma sampled pixel, and the next singleton <Y'> corresponds the next pixel. For greyscale images all the Cb and Cr are 128 (80 in hexadecimal) making the file even more transparent.

10-bit uncompressed 4:2:2 files are of much funkier construction. An Apple developer document from 1999 gives the details. My reading v210 files.pdf might be easier to understand. A spreadsheet to pull the scrambled Cb, Y', Cr, Y', Cb, Y', Cr... out of copied hunks from the SweetScape 010 Editor will be good enough for our investigations, but a real program added to SweetScape 010 would be nicer.

Dennis Couzin
Berlin, Germany

Note Added: 10-bit uncompressed 4:2:2 analyzer spreadsheet now available: here.

Quote
Jeff Harrell
"3:1:1" is shorthand for 4:2:2 at 1440 samples per scan line. It's equally correct to say that HDCAM (not SR) is 1440x1080 4:2:2.

Not true.

Dennis Couzin
Berlin, Germany