From 3b33d6f2588ed3c4a47a36103cce7db1440956c7 Mon Sep 17 00:00:00 2001 From: Matt Wozniak Date: Mon, 18 Mar 2024 16:46:47 -0700 Subject: [PATCH 1/4] Define new metadata for cropping --- 06.bitstream.syntax.md | 19 +++++++++++++++++++ 07.bitstream.semantics.md | 37 ++++++++++++++++++++++++++++++++++++- 2 files changed, 55 insertions(+), 1 deletion(-) diff --git a/06.bitstream.syntax.md b/06.bitstream.syntax.md index 78c6f727..c7430df2 100644 --- a/06.bitstream.syntax.md +++ b/06.bitstream.syntax.md @@ -427,6 +427,8 @@ at least one byte of the payload data (including the trailing bit) shall not be | metadata_scalability( ) | else if ( metadata_type == METADATA_TYPE_TIMECODE ) | metadata_timecode( ) +| else if ( metadata_type == METADATA_TYPE_CROP ) +| metadata_crop( ) | } {:.syntax } @@ -572,6 +574,23 @@ at least one byte of the payload data (including the trailing bit) shall not be | } {:.syntax } +#### Metadata cropping syntax + +| --------------------------------------------------------- | ---------------- | +| metadata_crop( ) { | **Type** +| @@crop_width_minus_1 | f(16) +| @@crop_height_minus_1 | f(16) +| @@crop_offset_present | f(1) +| if ( crop_offset_present == 1 ) { +| @@crop_x_offset | f(16) +| @@crop_y_offset | f(16) +| } else { +| crop_x_offset = 0 +| crop_y_offset = 0 +| } +| } +{:.syntax } + ### Frame header OBU syntax #### General frame header OBU syntax diff --git a/07.bitstream.semantics.md b/07.bitstream.semantics.md index dceec2a0..f841ccac 100644 --- a/07.bitstream.semantics.md +++ b/07.bitstream.semantics.md @@ -575,7 +575,8 @@ Metadata OBUs may or may not have an OBU extension header. If there is no extens | 4 | METADATA_TYPE_ITUT_T35 | payload-specific | 5 | METADATA_TYPE_TIMECODE | N | 6-31 | Unregistered user private | - -| 32 and greater | Reserved for AOM use | - +| 32 | METADATA_TYPE_CROP | Y +| 33 and greater | Reserved for AOM use | - {:.table .table-sm .table-bordered } The semantics of the column “Layer-specific” and its values are defined in Section 6.2.2. @@ -1314,6 +1315,40 @@ coded video sequence. number of bits used to represent time_offset_value is equal to time_offset_length. When time_offset_value is not present, its value is inferred to be equal to 0. +#### Metadata crop semantics + +When present the metadata_crop OBU applies starting at the next frame in the sequence +with matching temporal_id and spatial_id, and shall apply until the next metadata_crop OBU. +The output picture should be cropped to the region as specified in the cropping metadata OBU. +When applied, the crop shall be after all normal decode operations as a post-processing step +(after film grain synthesis in the output process). This metadata has no effect on the decoding process. + +**crop_x_offset** specifies the minimum pixel column containing picture data which should be rendered. + +**crop_y_offset** specifies the minimum pixel row containing picture data which should be rendered. + +**crop_width_minus_1** specifies the number of pixel columns minus 1 which which should be rendered after applying crop_x_offset. + +**crop_height_minus_1** specifies the number of pixel rows minus 1 which which should be rendered after applying crop_y_offset. + +If RenderWidth or RenderHeight are present in the frame header, then render width and height +shall be applied after cropping. + +The total cropped area shall not exceed the frame size: + + * crop_width_minus_1 + crop_x_offset shall be less than or equal to frame_width_minus_1. + * crop_height_minus_1 + crop_y_offset shall be less than or equal to frame_height_minus_1. + +The crop area shall align with the chroma subsampling grid: + + * crop_x_offset and crop_width_minus_1 + 1 must both be a multiple of subsampling_x + 1. + * crop_y_offset and crop_height_minus_1 + 1 must both be a multiple of subsampling_y + 1. + +When muxed into a container that supports signaling cropping information, this metadata should +be removed from the bitstream and included in the container’s signaling mechanism. +If both the container and bitstream signal cropping information, then the container’s cropping +information takes precedence. + ### Frame header OBU semantics #### General frame header OBU semantics From 2ed153afd5b68b591d2887c14b623618436a010d Mon Sep 17 00:00:00 2001 From: Matt Wozniak Date: Mon, 18 Mar 2024 17:42:37 -0700 Subject: [PATCH 2/4] Re-scope to key frames --- 07.bitstream.semantics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/07.bitstream.semantics.md b/07.bitstream.semantics.md index f841ccac..cf6afe40 100644 --- a/07.bitstream.semantics.md +++ b/07.bitstream.semantics.md @@ -1318,7 +1318,7 @@ time_offset_value is not present, its value is inferred to be equal to 0. #### Metadata crop semantics When present the metadata_crop OBU applies starting at the next frame in the sequence -with matching temporal_id and spatial_id, and shall apply until the next metadata_crop OBU. +with matching temporal_id and spatial_id, and shall apply to all matching frames until the next Key Frame. The output picture should be cropped to the region as specified in the cropping metadata OBU. When applied, the crop shall be after all normal decode operations as a post-processing step (after film grain synthesis in the output process). This metadata has no effect on the decoding process. From 5ac8502945fe1b19390897e1bf71239d77613c3f Mon Sep 17 00:00:00 2001 From: Matt Wozniak Date: Mon, 18 Mar 2024 18:29:33 -0700 Subject: [PATCH 3/4] clarify both key and additional metadata --- 07.bitstream.semantics.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/07.bitstream.semantics.md b/07.bitstream.semantics.md index cf6afe40..0159a4cf 100644 --- a/07.bitstream.semantics.md +++ b/07.bitstream.semantics.md @@ -1318,10 +1318,11 @@ time_offset_value is not present, its value is inferred to be equal to 0. #### Metadata crop semantics When present the metadata_crop OBU applies starting at the next frame in the sequence -with matching temporal_id and spatial_id, and shall apply to all matching frames until the next Key Frame. -The output picture should be cropped to the region as specified in the cropping metadata OBU. -When applied, the crop shall be after all normal decode operations as a post-processing step -(after film grain synthesis in the output process). This metadata has no effect on the decoding process. +with matching temporal_id and spatial_id, and shall apply to all matching frames until the next Key Frame +or metadata_crop OBU. The output picture should be cropped to the region as specified in the +cropping metadata OBU. When applied, the crop shall be after all normal decode operations as a +post-processing step (after film grain synthesis in the output process). This metadata has no +effect on the decoding process. **crop_x_offset** specifies the minimum pixel column containing picture data which should be rendered. From bb04abdc8a5b5f0e9a5d5b68973b1d1b4e92f759 Mon Sep 17 00:00:00 2001 From: Matt Wozniak Date: Mon, 8 Apr 2024 12:48:11 -0700 Subject: [PATCH 4/4] Feedback fixes --- 07.bitstream.semantics.md | 24 +++++++++++------------- 1 file changed, 11 insertions(+), 13 deletions(-) diff --git a/07.bitstream.semantics.md b/07.bitstream.semantics.md index 0159a4cf..4e2d9dd1 100644 --- a/07.bitstream.semantics.md +++ b/07.bitstream.semantics.md @@ -1318,19 +1318,20 @@ time_offset_value is not present, its value is inferred to be equal to 0. #### Metadata crop semantics When present the metadata_crop OBU applies starting at the next frame in the sequence -with matching temporal_id and spatial_id, and shall apply to all matching frames until the next Key Frame -or metadata_crop OBU. The output picture should be cropped to the region as specified in the -cropping metadata OBU. When applied, the crop shall be after all normal decode operations as a -post-processing step (after film grain synthesis in the output process). This metadata has no -effect on the decoding process. +with matching temporal_id and spatial_id, and shall apply to all frames with a matching +temporal_id and spatial_id until the next Key Frame or metadata_crop OBU. If no extension +header is present the metadata_crop OBU applies to all layers. The output +picture should be cropped to the region as specified in the cropping metadata OBU. +The crop shall be applied after all normal decode operations as a post-processing step. +This metadata information has no effect on the decoding process. -**crop_x_offset** specifies the minimum pixel column containing picture data which should be rendered. +**crop_width_minus_1** specifies the number of pixel columns minus 1 which should be rendered after applying crop_x_offset. -**crop_y_offset** specifies the minimum pixel row containing picture data which should be rendered. +**crop_height_minus_1** specifies the number of pixel rows minus 1 which should be rendered after applying crop_y_offset. -**crop_width_minus_1** specifies the number of pixel columns minus 1 which which should be rendered after applying crop_x_offset. +**crop_x_offset** specifies the minimum pixel column containing picture data which should be rendered. -**crop_height_minus_1** specifies the number of pixel rows minus 1 which which should be rendered after applying crop_y_offset. +**crop_y_offset** specifies the minimum pixel row containing picture data which should be rendered. If RenderWidth or RenderHeight are present in the frame header, then render width and height shall be applied after cropping. @@ -1345,10 +1346,7 @@ The crop area shall align with the chroma subsampling grid: * crop_x_offset and crop_width_minus_1 + 1 must both be a multiple of subsampling_x + 1. * crop_y_offset and crop_height_minus_1 + 1 must both be a multiple of subsampling_y + 1. -When muxed into a container that supports signaling cropping information, this metadata should -be removed from the bitstream and included in the container’s signaling mechanism. -If both the container and bitstream signal cropping information, then the container’s cropping -information takes precedence. +In cases where cropping information is present in both the bitstream and the delivery or container format, the latter should be preferred. ### Frame header OBU semantics