index.bs

<pre class='metadata'>
Group: AOM
Status: FD
Text Macro: SPECVERSION v1.0.1
Title: HDR10+ AV1 Metadata Handling Specification
URL: https://AOMediaCodec.github.io/av1-hdr10plus
Shortname: av1-hdr10plus
Editor: Paul Hearty, Samsung
Editor: Bill Mandel, Samsung
Editor: Cyril Concolato, Netflix
Abstract: This document specifies how to use [=HDR10+ metadata=] within [[!AV1]] bitstreams, including when carried in [[!CMAF]].
Date: 2023-10-03
Repository: AOMediaCodec/av1-hdr10plus
Inline Github Issues: full
Boilerplate: property-index no, issues-index no, copyright yes
Markup Shorthands: css no
Test Suite: https://aomediacodec.github.io/av1-hdr10plus/conformance/
!Previously approved version: <a href="v1.0.0.html">v1.0.0</a>
</pre>

<pre class='biblio'>
{
  "AV1": {
  "href": "https://aomediacodec.github.io/av1-spec/av1-spec.pdf",
  "id": "AV1",
  "title": "AV1 Bitstream & Decoding Process Specification",
  "status": "Standard",
  "publisher": "AOM"
    },

  "AV1-ISOBMFF": {
  "href": "https://aomediacodec.github.io/av1-isobmff/",
  "id": "AV1-ISOBMFF",
  "title": "AV1 Codec ISO Media File Format Binding",
  "status": "Standard",
  "publisher": "AOM"
    },

  "CTA-861": {
  "href":  "https://shop.cta.tech/products/a-dtv-profile-for-uncompressed-high-speed-digital-interfaces-cta-861-h",
  "id": "CTA-861",
  "title": "ANSI/CTA-861-H, A DTV Profile for Uncompressed High Speed Digital Interfaces",
  "status": "Standard",
  "publisher": "CTA"
    },
    
  "CTA-5001": {
  "href": "https://shop.cta.tech/products/web-application-video-ecosystem-content-specification",
  "id": "CTA-5001",
  "title": "CTA-5001-C, Web Application Video Ecosystem - Content Specification",
  "status": "Standard",
  "publisher": "CTA"
    },
       
  "BT-2100": {
  "href": "https://www.itu.int/rec/R-REC-BT.2100",
  "id": "BT-2100",
  "title": "Recommendation ITU-R BT.2100-2 (07/2018), Image parameter values for high dynamic range television for use in production and international programme exchange",
  "status": "Standard",
  "publisher": "ITU"
    },
   
  "BT-2020": {
  "href": "https://www.itu.int/rec/R-REC-BT.2020",
  "id": "BT-2020",
  "title": "Recommendation ITU-R BT.2020-2 (10/2015), Parameter values for ultra-high definition television systems for production and international programme exchange",
  "status": "Standard",
  "publisher": "ITU"
    },
      
  "ST-2094-40": {
  "href": "https://ieeexplore.ieee.org/document/9095450",
  "id": "ST-2094-40",
  "title": "SMPTE ST 2094-40:2020, Dynamic Metadata for Color Volume Transform - Application #4",
  "status": "Standard",
  "publisher": "SMPTE"
    },

  "ST-2086": {
  "href": "https://ieeexplore.ieee.org/document/8353899",
  "id": "ST-2086",
  "title": "SMPTE ST 2086:2018, Mastering Display Color Volume Metadata supporting High Luminance and Wide Color Gamut Images",
  "status": "Standard",
  "publisher": "SMPTE"
    },
    
   "ST-2084": {
  "href": "https://ieeexplore.ieee.org/document/7291452",
  "id": "ST-2084",
  "title": "SMPTE ST 2084:2014, High Dynamic Range Electro-Optical Transfer Function of Mastering Reference Displays",
  "status": "Standard",
  "publisher": "SMPTE"
    },
    
  "T35": {
  "href": "https://www.itu.int/rec/T-REC-T.35-200002-I",
  "id": "T35",
  "title": "Recommendation ITU-T T.35 (02/2000), Procedure for the allocation of ITU-T defined codes for non standard facilities",
  "status": "Standard",
  "publisher": "ITU"
    },
    
  "CMAF": {
  "href": "https://www.iso.org/standard/71975.html",
  "id": "CMAF",
  "title": "ISO/IEC 23000-19, Information technology — Multimedia application format (MPEG-A) — Part 19: Common media application format (CMAF) for segmented media",
  "status": "Standard",
  "publisher": "ISO"
    },
    
  "DASH": {
  "href": "https://www.iso.org/standard/79329.html",
  "id": "DASH",
  "title": "ISO/IEC 23009-1, Information technology — Dynamic adaptive streaming over HTTP (DASH) — Part 1: Media presentation description and segment formats",
  "status": "Standard",
  "publisher": "ISO"
    },
    
  "DASH-IOP": {
  "href": "https://dashif.org",
  "id": "DASH-IOP",
  "title": "Guideline for Implementation: DASH-IF Interoperability Points V4.3: On-Demand and Mixed Services, HDR Dynamic Metadata and other Improvements",
  "status": "Guidelines",
  "publisher": "ISO"
    },

    "CICP": {
  "href": "https://www.itu.int/rec/T-REC-H.273",
  "id": "CICP",
  "title": "Recommendation ITU-T H.273 | ISO/IEC 23091-2, Coding-independent code points for video signal type identification",
  "status": "Standard",
  "publisher": "ITU / ISO"
    }

}
</pre>

<!-- Terms defined in other specifications -->
<pre class="anchors">
url: https://aomediacodec.github.io/av1-spec/av1-spec.pdf; spec: AV1; type: dfn;
    text: AV1 coded video sequence
    text: OBU
    text: Temporal unit
    text: Metadata OBU
    text: Metadata ITUT T35
    text: Padding OBU
    text: Tile group OBU
    text: Sequence Header OBU
    text: Frame Header OBU
    text: Temporal Delimiter OBU
    text: metadata_type
    text: itu_t_t35_country_code
    text: itu_t_t35_payload_bytes
    text: METADATA_TYPE_ITUT_T35
    text: color_config
    text: color_primaries
    text: transfer_characteristics
    text: matrix_coefficients
    text: color_range
    text: subsampling_x
    text: subsampling_y
    text: chroma_sample_position
    text: mono_chrome
    text: show_frame
    text: show_existing_frame

url: https://aomediacodec.github.io/av1-isobmff/; spec: AV1-ISOBMFF; type: dfn;
    text: AV1CodecConfigurationRecord
    text: configOBUs
    text: av1 metadata sample group
    text: Codecs Parameter String
    text: av01

url: https://www.iso.org/standard/79329.html; spec: DASH; type: dfn;
    text: Supplemental Descriptor
    text: @schemeIdUri
    text: @value

url: https://shop.cta.tech/products/web-application-video-ecosystem-content-specification; spec: CTA-5001; type: dfn;
    text: cdm4

url: https://shop.cta.tech/products/a-dtv-profile-for-uncompressed-high-speed-digital-interfaces-cta-861-h; spec: CTA-861; type: dfn;
    text: MaxCLL
    text: MaxFALL
    text: itu_t_t35_terminal_provider_code
    text: itu_t_t35_terminal_provider_oriented_code
    text: application_identifier
    text: application_mode

url: https://ieeexplore.ieee.org/document/8353899; spec: ST-2086; type: dfn;
    text: MDCV
</pre>

Introduction {#introduction}
============

This document specifies how to use [=HDR10+ metadata=] within [[!AV1]] bitstreams, including when carried in [[!CMAF]].
[=HDR10+ metadata=] enables devices to optimize rendering of HDR content based on the display capabilities and on a scene-by-scene and frame-by-frame basis.

Various tools, services and devices support creation and use of [=HDR10+ metadata=], which can be easily utilized directly in [[AV1]] systems.
Carriage of [=HDR10+ metadata=] in [[AV1]] leverages mechanisms specified in [[!T35]] and [[!CTA-861]].
[=HDR10+ metadata=] is placed in [=metadata OBUs=] of <code>[=metadata_type=]</code> equal to <code>[=METADATA_TYPE_ITUT_T35=]</code>.
This document covers details of the placement of these [=OBUs=] in [[!AV1]] bitstreams.

Use of HDR10+ in AV1 bitstreams {#UsingHDR10plus}
=================================

HDR10+ Metadata {#hdr10plus-metadata}
----------

In the context of this specification, the syntax and semantics of the <dfn>HDR10+ Metadata</dfn> are defined in [[!CTA-861]] and [[!ST-2094-40]] respectively.


An <dfn>HDR10+ Metadata OBU</dfn> is defined as [=HDR10+ Metadata=] carried in a [=Metadata OBU=].
The <code>[=metadata_type=]</code> of such [=Metadata OBU=] is set to <code>[=METADATA_TYPE_ITUT_T35=]</code> and the <code>[=itu_t_t35_country_code=]</code> of the corresponding [=Metadata ITUT T35=] element is set to <code>0xB5</code>.
The remaining syntax element of [=Metadata ITUT T35=], <code>[=itu_t_t35_payload_bytes=]</code>, is interpreted using the syntax defined in Annex S of [[!CTA-861]], starting with the <code>[=itu_t_t35_terminal_provider_code=]</code>, and the semantics defined in [[!ST-2094-40]].

According to the definition of the [=HDR10+ Metadata=], the first 6 bytes of the <code>[=itu_t_t35_payload_bytes=]</code> of the [=HDR10+ Metadata OBU=] are set as follows:

- <code>0x003C</code>, which corresponds to <code>[=itu_t_t35_terminal_provider_code=]</code> from Annex S of [[!CTA-861]]
- <code>0x0001</code>, which corresponds to <code>[=itu_t_t35_terminal_provider_oriented_code=]</code> from Annex S of [[!CTA-861]]
- <code>0x4</code>, which corresponds to <code>[=application_identifier=]</code> from Annex S of [[!CTA-861]]
- <code>0x1</code>, which corresponds to <code>[=application_mode=]</code> from Annex S of [[!CTA-861]]

All the remaining bytes of the <code>[=itu_t_t35_payload_bytes=]</code> identify as the [=HDR10+ Metadata=] and are associated with the syntax elements of the <code>user_data_registered_itu_t_t35</code> structure that is defined in Annex S of [[!CTA-861]].
For convenience, the structure of the [=HDR10+ Metadata OBU=] is illustrated in Figure 1. 

<center><img width="100%" src="figures/HDR10plus_Metadata_OBU.svg"></center>
<center><b>Figure 1.</b> HDR10+ Metadata OBU</center>

NOTE: [[AV1]] defines the general Metadata OBU syntax for [=HDR10 Static Metadata=] and ITU-T T.35 Metadata.

<dfn>HDR10 Static Metadata</dfn> is defined as a combination of three types of HDR related metadata, including [=MDCV=], [=MaxCLL=] and [=MaxFALL=].
[=MDCV=] shall be present while [=MaxCLL=] and/or [=MaxFALL=] may be present.

HDR10+ bitstream constraints {#bitstream-constraints}
-------------------

The following sections define constraints that apply to [[!AV1]] bitstreams when carrying [=HDR10+ Metadata=].

### Color Configuration ### {#color-configuration} 

Streams suitable for incorporating [=HDR10+ metadata=] as described in this specification shall use the following values for the [[!AV1]] <code>[=color_config=]</code>:
* <assert><code>[=color_primaries=] = 9</code></assert> [[!CICP]], which corresponds to the ITU-R BT.2020 color primaries [[BT-2020]] [[BT-2100]],
* <assert><code>[=transfer_characteristics=] = 16</code></assert> [[!CICP]], which corresponds to the PQ transfer characteristics function [[ST-2084]] [[BT-2100]],
* <assert><code>[=matrix_coefficients=] = 9</code></assert> [[!CICP]], which corresponds to the non-constant luminance Y'CbCr representation [[BT-2020]] [[BT-2100]].

Additionally, the following recommendations apply:
* <assert><code>[=color_range=]</code> should be set to 0</assert>,
* <assert><code>[=subsampling_x=]</code> and <code>[=subsampling_y=]</code> should be set to 0</assert>,
* <assert><code>[=mono_chrome=]</code> should be set to 0</assert>,
* <assert><code>[=chroma_sample_position=]</code> should be set to 2</assert>.

### Placement of HDR10+ Metadata OBUs ### {#placement} 

As defined in [[!AV1]], an [=AV1 coded video sequence=] consists of one or more [=temporal units=]. A [=temporal unit=] contains a series of [=OBUs=] starting from a [=Temporal Delimiter OBU=], optional [=Sequence Header OBUs=], optional [=Metadata OBUs=], a sequence of one or more [=Frame Header OBUs=], each followed by zero or more [=Tile Group OBUs=] as well as optional [=Padding OBUs=].

Consequently, <assert>for each frame with <code>[=show_frame=] = 1</code> or <code>[=show_existing_frame=] = 1</code>, there shall be one and only one [=HDR10+ metadata OBU=] preceding the [=Frame Header OBU=] for this frame and located after the last [=OBU=] of the previous frame (if any) or after the [=Sequence Header OBU=] (if any) or after the start of the [=temporal unit=]</assert> (e.g. after the [=Temporal Delimiter OBU=], for storage formats where [=Temporal Delimiter OBUs=] are preserved).

NOTE: As a consequence of the above statements [=HDR10+ Metadata OBUs=] are not provided when <code>[=show_frame=] = 0</code>. Additionally, for non-layered streams, there is only one [=HDR10+ Metadata OBU=] per [=temporal unit=]. 

For [[!AV1]] bitstreams encoded with multiple layers, [=HDR10+ Metadata=] may apply to one or more layers. However, the details are out of scope of this version of the specification.

Figure 2 shows a simplified example of placement of [=HDR10+ Metadata OBUs=] in an AV1 bitstream.

<center><img width="100%" src="figures/Placement.svg"></center>
<center><b>Figure 2.</b>&nbspExample of placement of [=HDR10+ Metadata OBUs=] in an AV1 bitstream</center>

### Provision for Film Grain Processing ### {#film-grain}

It is possible that some [[AV1]] bitstreams may contain both [=HDR10+ Metadata=] and film grain synthesis information. It is recommended that decoders in such scenarios perform the film grain synthesis prior to any [=HDR10+ Metadata=] processing.

Storage and Transport considerations {#transport}
=====================================

Constraints on AV1CodecConfigurationRecord {#codecconfigurationrecord}
--------------

<assert>For formats that use the <code>[=AV1CodecConfigurationRecord=]</code> when storing [[AV1]] bitstreams (e.g. ISOBMFF and MPEG-2 TS), [=HDR10+ Metadata OBUs=] shall not be present in the <code>[=configOBUs=]</code> field of the <code>[=AV1CodecConfigurationRecord=]</code></assert>.

ISOBMFF Constraints {#isobmff-constraints}
--------------

<assert>[=AV1 Metadata sample group=] defined in [[!AV1-ISOBMFF]] shall not be used.</assert>

[[!AV1-ISOBMFF]] indicates that [=Metadata OBUs=] may be protected.
This specification requires that <assert>[=HDR10 Static Metadata=] and [=HDR10+ Metadata OBUs=] are unprotected</assert>.

<assert>An ISOBMFF file or CMAF AV1 track as defined in [[!AV1-ISOBMFF]] that also conforms to this specification (i.e. that contains [=HDR10+ metadata OBUs=] and complies to the constraints from this specification) should use the brand <code>[=cdm4=]</code> defined in [[!CTA-5001]] in addition to the brand <code>[=av01=]</code></assert>. <assert>If the brand <code>[=cdm4=]</code> is used in conjunction with [[!AV1]] bitstreams, the constraints defined in this specification shall be respected</assert>.

HTTP Streaming Constraints {#httpstreaming-constraints}
--------------

<assert>The value of the [=Codecs Parameter String=] for [[!AV1]] bitstreams that is used when using HTTP streaming technologies shall remain unchanged when [=HDR10+ Metadata OBUs=] are included in the [[!AV1]] stream</assert>.

Additionally, <assert>[[!DASH]] content following [[DASH-IOP]] should include a [=Supplemental Descriptor=] with <code>[=@schemeIdUri=]</code> set to <code>"http://dashif.org/metadata/hdr"</code> and <code>[=@value=]</code> set to <code>"SMPTE2094-40"</code> in manifest files</assert>. This can aid players to identify tracks containing [=HDR10+ Metadata OBUs=].

Conformance Suite {#conformance}
==========================
A conformance suite for this specification is available at the following URL: 
> <a href="https://aomediacodec.github.io/av1-hdr10plus/conformance/">https://aomediacodec.github.io/av1-hdr10plus/conformance/</a>

Changes since v1.0.0 release {#changelist}
==========================================
- <a href="https://github.com/AOMediaCodec/av1-hdr10plus/pull/64">Transform redundant statement as a note</a>
- <a href="https://github.com/AOMediaCodec/av1-hdr10plus/pull/66">Add pointer to conformance suite</a>

<script>
if (window.location.hash) {
  const highlight_span = document.querySelector(window.location.hash);
  if (highlight_span) {
    highlight_span.parentNode.style.backgroundColor = "#F4EA565A";
    setTimeout(() => {
      highlight_span.parentNode.style.backgroundColor = "";
    }, 5000);
  }
}
</script>