What's the correct approach for handling Linear -> sRGB transform and tone mapping when using post?

[looeee]

This is a question that's been on my mind for a while and researching it has lead me down quite a rabbit hole of color theory. Interesting stuff but I'm still not sure of the answer 😅

My current understanding is that most processing passes (in Unity or Unreal, for example) assume that the input is in linear. If that's true for this library then the correct process is to leave the three.js output in linear, do all the passes in linear (except noise/grain which should be last), then do an sRGB transform followed by tone mapping as the final pass. Is that correct?

[vanruesc]

My current understanding is that most processing passes (in Unity or Unreal, for example) assume that the input is in linear.

Right, if you render a scene with postprocessing, the color data will be stored as-is in a render target. This data is assumed to be "linear" in subsequent steps regardless of what color space the input colors and textures were in. Since most color maps are in sRGB color space, this assumption will be incorrect unless we fix.

Going by the Unity docs, there are basically two approaches to handling sRGB input colors and textures:

• Gamma color space workflow
  • Treat color maps as linear, perform all calculations directly on these colors and skip the final gamma correction.
• Linear color space workflow
  • Convert color maps from sRGB to linear, perform calculations on the corrected colors and convert the result to sRGB.

Note that sRGB conversion is simply a more accurate version of gamma correction.

The gamma workflow can be a viable choice when you're only dealing with unlit materials because there are no lighting calculations involved. You could convert to linear and back, but you could also just render the colors and images as they are. Post processing effects can still produce good results but they will technically be wrong.

The linear workflow is the correct and recommended approach but it's harder to wrap your head around it and there are some issues in WebGL that make it less appealing for development. Three.js supports gamma correction for textures by converting them manually inside the shaders. It was probably implemented that way because WebGL didn't support hardware sRGB decoding without an extension. This solution has a slight impact on peformance. Support for SRGB8_ALPHA8 hardware decoding was recently added in r133, but this unearthed another issue related to the synchronous nature of the WebGL API. Hardware decoding is better in terms of performance and texture filtering quality, but it's not really usable at the moment.

If that's true for this library then the correct process is to leave the three.js output in linear [...] then do an sRGB transform followed by tone mapping as the final pass. Is that correct?

Gamma correction, or sRGB output encoding, should be the final step when rendering to screen. This library supports this via three's built-in output encoding system. Setting renderer.outputEncoding to sRGBEncoding ensures that any pass that renders to screen will apply this transformation. In three and postprocessing output encoding is also done in the shader because WebGL doesn't support hardware sRGB encoding, unlike OpenGL.

Tone mapping is used to map HDR colors to LDR for output on conventional monitors. Lighting calculations as well as post processing effects can all produce color values greater than 255. If you don't use tone mapping, those values will be clamped and the original brightness information will be lost. Tone mapping should be performed late in the pipeline but before color correction because most LUTs assume LDR/RGBA8 input colors.

It's also worth mentioning that you'll need medium precision buffers for the linear color space workflow as outlined in the readme to avoid losing precision in-between passes.

[looeee]

Thank you, this is a great write up.

Setting renderer.outputEncoding to sRGBEncoding ensures that any pass that renders to screen will apply this transformation

Awesome, this was the missing piece of info for me. Just spotted it in the Readme too, I missed that:

Simply set WebGLRenderer.outputEncoding to the desired target color space and postprocessing will follow suit.

So in summary:

1. As long as I'm already doing a linear workflow correctly in three.js, I don't need to change anything.
2. I should probably always set the framebuffer to HalfFloatType (any chance this can cause problems on mobile?)
3. I always need to add a tone mapping pass near the end (possible exception if doing tone mapping via LUT).
4. When the final pass is rendered to screen with renderer.outputEncoding = sRGBEncoding, the linear-sRGB to sRGB transfer will be be done automatically.

[looeee]

BTW is there a reason for choosing REINHARD2_ADAPTIVE as the default tone mapping? My understanding is that the closest thing there is to an industry standard is ACES Filmic.

[vanruesc]

So in summary:

1. Correct.
2. Yes, granted the device supports it.
3. I think that's generally the right way to do it, yes.
4. Correct.

HalfFloatType any chance this can cause problems on mobile?

Browser support for half float textures is good, but it's possible that older devices don't support it. Support for HalfFloatType is a requirement for accurate post processing effects using the linear color space workflow. Devices that don't support it can fall back to low precision buffers, but there will be color banding and artifacts. Checking whether the device supports half float textures (OES_texture_half_float and OES_texture_half_float_linear) is left up to the user.

is there a reason for choosing REINHARD2_ADAPTIVE as the default tone mapping?

No reason. We should probably change it to ACES in the next major release.

[looeee]

That's great, thanks for you help :)