Topic: Lifted blacks

I have been trying to sort my issue of playback of DCP looking milky with lifted black levels. Although I had tested OpenDCP on a short animation this is the first time I tried a feature length documentary. Source is a ProRes422 quicktime Rec709 file. I tried two methods of encoding the jpeg2000. Firstly from a tiff sequence using OpenDCP and secondly in Adobe media encoder using the j2k_V2.7 add on that gives direct DCI compliant jpeg200 from the quicktime. Both the OpenDCP and j2k versions have the same milky look when imported back in Resolve and in the cinema the xyz space is not showing the original gamma but a milky lifted black look. So my presumption is that there is a gamma error in both paths.

I note that the latest version 3.0 has a bug fix about lifted blacks and now has Rec709 complex in the menu. Can you elaborate on the problem that led to this change and please advise the correct workflow from a ProRes422 quicktime.

Re: Lifted blacks

There was no bug fix related to lifted blacks. An additional transform was added to handle the different ways applications encode the gamma. If you are seeing lifted blacks, use the non-complex formula.

Re: Lifted blacks

Ok thanks.

4 (edited by dcouzin 2014-12-22 15:50:03)

Re: Lifted blacks

Terrence Meiczin made the most detailed statement on this problem back in April:!topic … cF5WDnMhWI:

The gamma values depend on whether a simplified calculation is being made or not. sRGB is 2.2 when approximated and 2.4 when doing a complex calculation. Rec 709 does not have a set gamma, but more or less a defacto standard of 2.4 when approximating and 1/.45 when doing the complex calculation. The issue arises when the approximation and more complex calculations are mixed. This can lead to blacks either being elevated or compressed depending on direction. As of OpenDCP 0.29, the complex calculation was being used, but in the next release, the user will be able to select between the two methods.

Unfortunately that strand petered out before getting to the bottom of the problem.  At root is a hole in Rec709.  Rec709 specifies how to go from three exposures RGB via R'G'B' to a video coding Y'CbCr.  But Rec709 is also, indirectly, about displaying video since it specifies the display primaries and white point. To use Rec709 for display requires going from the video Y'CbCr via R'G'B' to RGB which will then drive those three primaries.  For the first step, Y'CbCr->R'G'B', everyone uses the inverses of Rec709's R'G'B'->Y'CbCr transforms (even though Rec709 is silent about this).  But for the second step, R'G'B' -> RGB, (where Rec709 is also silent) almost no one (except perhaps Mr. Meiczin before OpenDCP 0.30) uses the inverse of the Rec709 RGB->R'G'B' transform (so-called "precorrection").  Instead it is common to use a simple gamma transform.  But there is disagreement over what value to use for gamma.  Many believe it should be 2.2 (crudely extracted from the Rec709 precorrection transform).  The EBU recommends 2.35.

We shouldn't expect free OpenDCP to offer picture adjustment services.  It is enough if OpenDCP is well enough documented so we know what it's doing when it transforms R'G'B' -> RGB.  Then competent workers can make necessary compensations in their video.  So what is the "complex calculation" and what is the approximate calculation?  Seeing the words "1/.45 when doing the complex calculation" suggests that the complex calculation is the inverse of the Rec709 precorrection (see ITU-R BT.709-5 item 1.2).  It then is not a gamma transform.  In Rec709 language it is:

L = V/4.5   for V < 0.081
L = ((V+.099)/1.099)^(1/0.45)    for V >= 0.081

The gamma transform closest to this would have gamma around 1.90 ± 0.08 (depending on how you calculate closeness).  This would explain lifted near-blacks.

The words "more or less a defacto standard of 2.4" hark to the EBU recommendation which was based on studies of actual CRT's, but we shouldn't have to guess what it means.  OpenDCP should document what its non-complex calculation does. 

What's critically important is how what OpenDCP does to our Rec709 video corresponds to what we did to it when we color graded it.  What display gamma did we use on our Rec709 calibrated monitors?  If OpenDCP's non-complex calculation is a gamma=2.4 transform, and if you calibrate your monitor to Rec709 with gamma 2.4 and 55 nits and view it in a fully darkened room it should agree with the DCP projection of the OpenDCP creation.  If instead you grade at gamma 2.2, you can apply a gamma 0.917 filter to the finished video before feeding it to OpenDCP.

This with the understanding that video applications are scandalously sloppy in their gamma calculations.  Between them, Apple's FCP7 and Compressor3.5 calculate gamma three different ways: … msg-280060.  Disagreeing know-it-all programmers and an unwillingness to document. 

There are other related details in need of documentation.  How does OpenDCP handle sub-black values and super-white values in the Rec709 video?  And again, we must know how our Rec709 calibrated monitoring handled them while we graded the video. 

There are also questions about color.  The minimum DCI chromaticity gamut is larger than the Rec709 chromaticity gamut.  There may be Y'CbCr values in our video from which the Rec709 transform makes negative values among the R'G'B'.  Those negatives indicate chromaticities outside the Rec709 gamut.  They would be clipped in Rec709 monitoring, and anyhow the gamma transform can't handle negative values.  Does OpenDCP (or other Rec709-to-DCP software) push outside the Rec709 gamut?  If you don't like how your software does or doesn't do it, then you can apply a big fat 3D LUT to your finished video so the DCP is as you wish, but this requires knowing exactly what your Rec709-to-DCP software does to color.  Documentation please!

Dennis Couzin
Berlin, Germany