Shortest path from DNG to aces EXR

[ahemberger]

Hi there;

I have a rather antiquated hobbyist pipeline I've built for myself, wherein I take footage captured on a DSLR with Magic Lantern and try to get it into a useable state for VFX (which is ideally a linear acescg EXR). Magic Lantern produces a collection of cinemaDNG files, and I arrive at my output via a number of hoop jumps through tools like dcraw, exiftool, and so on.

I'd like to modernize this, and am wondering what the fastest path might be. I think it might be to use Colour to read the image, perform some transform, and write out an EXR, most-likely, I suspect, using oiio as the backend (I'm unsure if I can use oiio for this directly).

I'm curious if there's a reference for this, or if anyone might offer some advice/a sample code snippet?

[KelSolaar]

Hi @ahemberger,

There are many ways to do that, if you are inclined to use Python for your stack, you could look at Rawpy and take inspiration in this example: https://github.com/colour-science/colour-hdri/blob/develop/colour_hdri/examples/examples_advanced_processing_with_an_input_device_transform.ipynb

Cheers,

Thomas

[ahemberger]

Thanks Thomas;

I'm trying to wrap my head around this example; it is difficult. rawpy is new to me, that's neat to learn about.

Some questions:

• there is a hokey bit of rigamarole involved in getting a "mlv" Magic Lantern file converted into a series of DNGs - I use an app called RAWMagic that's apparently designed exactly for this purpose. There are settings in that app (white balance, DNG bit depth) that suggest "some stuff" is happening between the data my camera captures and what I actually get in my DNG files. I'm curious how this affects the actual rawpy conversion; is there potentially 'double work' being done to my files that might be unwanted? I realize this is WAY outside the purview of this repo, mostly I'm looking for ways where things might be going wrong without me knowing.
• There's a step where you do an RGB_to_RGB conversion from ACES to sRGB before your HDRI merge. It's unclear to me how the data generated by rawpy 'becomes' ACES, to then be converted to sRGB. Like, rawpy has an "aces" color conversion option, but this code specifically asks for raw pixel data. I ask this because I'm not clear what colorspace my mlv data is originally in, what space it exists in in the DNG files, and what's given back to be my rawpy given the parameters used in this call.

This stuff is hard and confusing.

[KelSolaar]

At first glance that looks about right to me! Once the DNG is read, rawpy will request the raw data, we compute the white balance multipliers and colour correction/formation matrix from Camera RGB to ACES2065-1 using the spectral sensitivities of the camera. The multipliers are applied by libraw before demosaicing to avoid artefacts. The image you get out of this process is in ACES2065-1 and you can then convert it to whatever you need.

Assuming you want to leverage that process, you could hook-it up with Houdini PDG and have multi-processing for free essentially. This is pretty much what we do at work at Special Projects.

[ahemberger]

Thank you so much Thomas!

I included a bunch of questions in the above code, though the lack of syntax highlighting makes them hard to spot. But, maybe the one I'm most curious about is this line:

SD_ILLUMINANT = colour.sd_CIE_illuminant_D_series(
    colour.temperature.CCT_to_xy_CIE_D(CCT * 1.4388 / 1.4380))

What are the 'magic numbers' (1.4388/1.4380) for there?

[ahemberger]

Hi again, bumping this question, it's hard to know if I'm actually doing this right.

Another question I have: in this utility I use to convert these .mlv files to DNG, there are settings to autodetect white balance. But, in this code, there is also what appears to be white balancing happening as well:

CCT = 6000 # White balance, in kelvins?
SD_ILLUMINANT = colour.sd_CIE_illuminant_D_series(
    colour.temperature.CCT_to_xy_CIE_D(CCT * 1.4388 / 1.4380)) # what does this do? What are these 1.4-ish numbers?
  
M_IDT, RGB_w = colour.matrix_idt(SENSITIVITIES, SD_ILLUMINANT)

I understand with actual RAW data, white balancing is just a multiplication that can be arbitrarily changed later without loss of data, but it leads me to question what's actually happening to the step whereby DNGs are being produced by this utility (which I think is just a question about the nature of DNG files vs. actual RAW [.cr2, .nef, etc.] files themselves). Why would this utility care about white balance if it's not actually doing any demosaicing? Is white balance a metadata field in DNGs that just controls how the raw data is displayed in previews?

Also: the output EXRs of this process seem surprisingly dark (by like a stop or two). Attached, for example, is the preview I see in my Macbook Finder of the output exr image (on top) and the DNG (on the bottom). I would expect a contrast/WB difference, but the overall luminosity difference surprises me.

To my knowledge, Magic Lantern produces RAW images that are 14bits in precision; there is an option in this RAWMagic conversion utility to write out either 12-bit or 16-bit DNGs; I choose the latter assuming that the former would potentially clip my data. But in this code, there's this step:

image.data = colour.utilities.as_float_array(
    rawpy.imread(input_path).postprocess(...)) / 65535 

the last magic number there is 2^16, and I wonder if it presumes that my data 'fills' the entire 16 bits, or if I should be using 2^14 there, to signify that my data is actually just 14 bits of precision (trying this yields an image that more-closely matches the luminosity of the DNG, but again I'm grasping at straws here because do not fully understand this).

Finally: rawpy offers an option to output data in ACES colorspace, yet that option is not used here (instead, the data asked for is in "raw" colorspace). I'm curious why this is?

Thank you!

[KelSolaar]

Thank you so much Thomas!

I included a bunch of questions in the above code, though the lack of syntax highlighting makes them hard to spot. But, maybe the one I'm most curious about is this line:

SD_ILLUMINANT = colour.sd_CIE_illuminant_D_series(
    colour.temperature.CCT_to_xy_CIE_D(CCT * 1.4388 / 1.4380))

What are the 'magic numbers' (1.4388/1.4380) for there?

They are not really magic and we document them in Colour. Long story short, it is related to a constant change, i.e. C_2, long explanation here:

[KelSolaar]

I understand with actual RAW data, white balancing is just a multiplication that can be arbitrarily changed later without loss of data, but it leads me to question what's actually happening to the step whereby DNGs are being produced by this utility (which I think is just a question about the nature of DNG files vs. actual RAW [.cr2, .nef, etc.] files themselves). Why would this utility care about white balance if it's not actually doing any demosaicing? Is white balance a metadata field in DNGs that just controls how the raw data is displayed in previews?

White-balance is never applied on the RAW data, irrespective of whether it is stored in a DNG, CR2/3 or NEF file. The various metadata values you get are typically the value your camera is set to which is used during the preview generation. The critical thing to understand is that the gains on the R, G, B channels must be applied BEFORE the image is demosaiced, otherwise artefacts will appear, e.g. Figure 18 Here. When we read the image with Rawpy/Libraw, we tell it which white-balance multipliers to use according to the computed colour correction/formation matrix derived from the spectral sensitivities of the camera.

Also: the output EXRs of this process seem surprisingly dark (by like a stop or two). Attached, for example, is the preview I see in my Macbook Finder of the output exr image (on top) and the DNG (on the bottom). I would expect a contrast/WB difference, but the overall luminosity difference surprises me.

Rawpy/Libraw scales the image according to the camera given black and white saturation-levels, I don't know what the utility you used is doing so I cannot really comment on the difference. To me, it is obvious that there is a Display Rendering Transform (DRT) applied to make the image pretty on a display but who know what it is doing?

the last magic number there is 2^16, and I wonder if it presumes that my data 'fills' the entire 16 bits, or if I should be using 2^14 there, to signify that my data is actually just 14 bits of precision (trying this yields an image that more-closely matches the luminosity of the DNG, but again I'm grasping at straws here because do not fully understand this).

We set the processing output to 16-bit integer, e.g. output_bps=16, so this is what you will get basically out of Rawpy, irrespective of the underlying sensor bit depth, and as previously mentioned, the data is scaled (stretched) to fill that entirely.

Finally: rawpy offers an option to output data in ACES colorspace, yet that option is not used here (instead, the data asked for is in "raw" colorspace). I'm curious why this is?

Because we prefer to use spectral sensitivities, it is much more precise than the values given by Adobe and used if you were to choose the ACES colourspace as an output.