online treatment of Log footage with BT.709 footage

The 1st cameraperson is producing Canon Log footage. The 2nd cameraperson is producing BT.709 footage. The editor wants to edit online in FCP7 without sacrificing the dynamic range advantages of the Canon Log footage.

The plan is to apply an LUT to the BT.709 footage to make it mimic, as best it can, Canon Log footage. This is the opposite LUT to those usually used -- I might have to make it myself. Then the two footages will display equally horribly. The plan is to fix this by making a special monitor calibration (designed to make the LUTed BT.709 footage display correctly). The editor can then intercut the two footages, harmoniously. The editor likes to do color correction within editing. The BT.709 parts will be much less color correctable than the Canon Log parts, as should be.

Does this plan hold water?

Dennis Couzin
Berlin, Germany

> The editor likes to do color correction within editing

This part doesn't. FCP doesn't color correct very well as it doesn't have curves, a good keyer and tracking power windows, making it barely a color correction app.



www.strypesinpost.com

For curves I'm looking into using the Nattress plugin "Levels and Curves". Do they preserve 10-bit?
There's no need for keying or tracking.

In cinema, color correction -- grading/timing -- comprised exactly three controls: red exposure; green exposure; blue exposure. (You could only control the gamma of a shot by making extra generations, almost never done.) Yet grading/timing was an art, which kept someone busy. The toolset within FCP offers many more controls and can also keep someone busy. At some level of sophistication of controls, the art can lapse into fussing, unless there's no distinction between color correction and color effects.

Dennis Couzin
Berlin, Germany

It will never work, and here is why. Even with a Flattened treatment to the .bt709 footage. It will always pop, because the LUT isn't going to have the reach, because it's bt.709. It will need help 50% of the time or better. Thereby negating any time saved in not just applying filters to shots used.

Besides if the "Editor" is going to CC anyway. Why add a layer of noise and difficulty.

The scene captured in Canon Log consists of a range of luminances from Lmin to Lmax. There is a function relating these luminances to code values in the Canon Log video. The same scene captured in BT.709 consists of a proper subset of [Lmin, Lmax]. There is a function relating these luminances to code values in the BT.709 video. These are published functions, but you can also measure them. Here is how the LUT is constructed. For each code value in the BT.709 video, trace it back to the scene luminance, then trace it forward to the code value in the Canon Log video. Since the BT.709 video captures a smaller luminance range than the Canon Log video, this must work. The BT.709 video with the LUT applied includes fewer code values than the original BT.709 video. Since it's 10-bit this isn't harmful.

Now, as explained, calibrate the monitor to make the LUTed BT.709 footage display correctly. That is, so it displays exactly the way the original BT.709 footage displayed before the special calibration. Calibration software like Spectraview II, which allows tabulated custom gamma curves, can do this. The Canon Log video will now look just like the BT.709 video. Does properly displayed BT.709 footage "always pop"? Then curves must be applied in color correction to reduce the annoying "pop". Curves and corrections can be applied more effectively to the Canon Log footage which has many code values "in reserve". For example, a code value in the Canon Log footage that is higher than those in the LUTed BT.709 footage, can be accessed by simplying reducing the brightness of a Canon Log shot. Doing the same to a BT.709 shot just shows clipping. The purpose of this seemingly roundabout approach is to allow the editor to harmonize the two kinds of footage, as best possible, while seeing BT.709 broadcast-like colors on the monitor.

In order to edit and color correct online, you can't start by truncating the Log footage by applying the usual (Log-to-BT.709) LUT. The LUT must be applied last, after the color correction filters. I don't know if FCP + LUT Buddy allows this, and if the LUT requires rendering this will make color correction too difficult. In essence I'm applying the Log-to-BT.709 LUT within the monitor calibration. The the BT.709 footage then requires a BT.709-to-Log LUT.

Dennis Couzin
Berlin, Germany

Quote
dcouzin
...the Nattress plugin "Levels and Curves". Do they preserve 10-bit?

Hurrah, they do!
I checked this by applying a Nattress curve to a 10-bit grey wedge image. The original image scopes like this in the waveform monitor, and the processed and rendered image scopes like this. The FCP waveform monitor shows nothing about bit depth because the waveform monitor itself is 8-bit. (If you doubt this then count the 239 levels in curve -- which was about 725 pixels high in the screenshot.)

Looking into an exported .mov file, the "before" image ranged from 64 to 1019 and included 692 levels. (Being a PAL target it couldn't include more than 720 levels.) The "after" image ranged from 67 to 1014 and included 634 levels. The tiny clipping may be due to how I set the Nattress curve parameters. The further reduction of 50 levels is explained mostly by there being places where the "after" curve is flatter than the original. These can reduce the number of values, while the places where the "after" curve is steeper than the original can't increase the number of values. The Nattress plugin is honest, 10-bit preserving, and I should now pay the $50 to remove the watermark (evident in the "after" screenshot).

Dennis Couzin
Berlin, Germany

The plugin normally runs on the GPU using shaders that are in full floating point. In other words the math computations are float. They also occur on RGB data, so we have to account how the host app translates the underlying Y'CbCr to RGB and back again after render. Where we do a Y'CbCr conversion again it's all float.

Hope that helps,

Graeme

Graeme is absolutely right that if the host application extracted 8-bit R'G'B' from the 10-bit Y'CbCr and passed that on to the plugin, the plugin's high precision would be mostly wasted. Any plugin must be evaluated together with what it's plugged into. Happily, FCP7 passed on at least 10-bit R'G'B', so "Levels and Curves" plugged into FCP7 is 10-bit preserving.

Anyhow, since "Levels and Curves" supports FCP7, if FCP7 didn't do a decent job of the conversion, "Levels and Curves" would have taken over that step. Right?

* * * * *

What must be appreciated is that even if the 10-bit Y'CbCr were transformed to high precision floating point R'G'B', and the curves applied with high precision, and the transformation back to Y'CbCr done with high precision, those final 10-bit Y'CbCr would contain fewer code values than the original just because the curves are curved. A numerical example can show this without need to distinguish Y'CbCr and R'G'B'. Suppose pixels numbered 0 to 1023 are coded with their pixel number: f(p)=p. Slope = 1. Suppose a very gentle gamma = 1.2 curve is applied: g(f(p))= ((f(p)/1023)^1.2)*1023 with perfect precision. On return to integers there are only 955 code values. 69 code values have been dropped. "Rounding errors" is the generic explanation, but really this is about curves. Where the slope of the gamma curve is less than 1 some code values must be reused, here and there, so the total number used is reduced. Where it is more than 1 some code values must be skipped over, here and there, but the number of code values used can't change.

This reduction of the number code values used where slopes are lowered is harmless. Lowered slope means a corresponding reduction of the luminance range in the displayed image, and it is in the displayed image, rather than in the original scene, that the critical perceptual luminance discriminations occur. Banding arises when there are too few code levels for some portion of the displayed luminance range. So it is where the applied curve jacks up the slope, and the number of code levels is maintained, that there can be flirtation with banding.

Dennis Couzin
Berlin, Germany

Consider two approaches to online interediting of Log material plus BT.709 material in FCP7.

Approach 1: apply a Log-to-BT.709 LUT to the Log footage and merrily interedit. The applied LUT is like a video filter and it should be applied last. Working this way, other video filters, particularly color correction filters, work on the uncompromised Log material. Their effects are somewhat different from what they are on BT.709 material, but you can see their effects after rendering. That's the catch. You've a huge render job after every tweak of every filter. FCP7 color correction becomes practically impossible. So Approach 1 is unworkable.

Approach 2: apply a BT.709-to-Log LUT to the BT.709 footage, and recalibrate the monitor by essentially applying a Log-to-BT.709 filter to it. Then merrily interedit. The advantage of this approach is that the LUTing of the BT.709 footage can be done ahead of time, once and for all, so static effects like color correction applied to the BT.709 footage do not require rendering. It can be done ahead of time because a BT.709-to-Log LUT, unlike a Log-to-BT.709 LUT, doesn't sacrifice image information. (The loss of a fraction of the 10-bit precision is imperceptible.)

Approach 2 yields an edit which only displays correctly on the oddly calibrated monitor. It will need a Log-to-BT.709 LUT applied at the end. This makes Approach 2 seem wasteful: the BT.709 footage first gets a BT.709-to-Log LUT and afterwards gets a Log-to-BT.709 LUT. Perhaps that's what Ethan means by "why add a layer of noise and difficulty". But this doesn't add a layer of noise if the BT.709 material is 10-bit and the LUTs work at at least 10-bit precision. Sure it's difficult, and unlovely, but is there any better approach?

Dennis Couzin
Berlin, Germany

Progress report. The hybrid method -- LUT + calibration -- works, but I'm less sure it will be necessary.

Since different camerapeople prefer different exposures, I made five different BT.709-to-CanonLog LUTs at half-stop exposure steps.

Applying the different LUTs to the BT.709 footage I found that "bump-1" best got the two footages in the same ballpark. Note that these LUTs are non-destructive: the BT.709 footage is not clipped either high or low.

It is necessary to use the corresponding monitor calibration.

With the "bump-1" monitor calibration, the LUTed BT.709 footage is displayed with perfect accuracy as BT.709 footage should be.
The CanonLog footage is displayed not as it should be, because there is no "should be" for it, but as we guess it might be seen in BT.709 displays. The CanonLog footage has very much Y' outside the range that the monitor calibration shows (about 100-to-800), all in reserve. You can bring this into view by application of filters and curves, making a more handsome image, but one that differs from the BT.709 shot images.

I was surprised by how saturated the C100's CanonLog footage looked in this monitoring. Either the cameraperson cranked up chroma, or the Canon engineers were eating candy when setting the defaults, or else the C100 uses color transforms very different from BT.709's. That is, very different Y'CbCr -> R'G'B' transforms and/or very different primaries. The editor must deal with this as part of the harmonization task. Color space transformations can be accomplished by 3D LUTs.

Hard work for an editor, but I think that only the editor can do it. A colorist would get the work too late, after the decisions about shots which depend on their colors and looks are made.

The hybrid method has a funny downside. With the bump-1 monitor calibration the picture looks good but everything else looks awful. You can hardly read the menus. Pros who already know FCP7 like the back of their hand might find this amusing.

The hybrid method is premised on not wishing to apply a CanonLog-to-BT.709 LUT to the CanonLog footage on account of its needing to be applied after all other filters, and so requiring heavy rendering which ruins the preview of those filters. Antler Post might soon produce a LUT applying plugin for FCP7 which avoids this. Their plugin might be able to play LUTed HD in real time (provided Unlimited RT is enabled and Playback Video Quality is set to High). This might return me to the standard LUT method. The aesthetic problems will all remain.

Dennis Couzin
Berlin, Germany