Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow an image to indicate its own density and correct its intrinsic size #5574

Merged
merged 16 commits into from
Jul 8, 2021
Merged

Allow an image to indicate its own density and correct its intrinsic size #5574

Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
978645b
Allow an image to indicate its own density and correct its intrinsic …
noamr May 24, 2020
08e4a01
Use image request, refactor to use 'prepare' algo
noamr Jun 5, 2020
a0ff979
nit
noamr Jun 5, 2020
9c1e85e
Fix CR comments
noamr Jun 8, 2020
873dfb5
Bail from metadata reading for non-origin-clean images
noamr Jul 8, 2020
7f09870
Apply same-origin clarifications after #5165
noamr Sep 8, 2020
cd6d4db
Nit req/img
noamr Sep 9, 2020
84fce1d
Declare an img pointing to img's req
noamr Sep 9, 2020
71af446
Use better terms for cross-origin
noamr Sep 9, 2020
fc718c4
More nits
noamr Sep 16, 2020
00b5724
Fix typo px -> pt
zcorpan May 28, 2021
9f5e047
Fix source formatting
zcorpan May 28, 2021
98ccb1d
Apply review comments
noamr Jun 24, 2021
550cd15
Apply review comments
noamr Jun 24, 2021
da9f571
CR comments
noamr Jun 25, 2021
441ca72
Fix wrong variable references (img does not exist at the call sites) …
domenic Jul 8, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
137 changes: 115 additions & 22 deletions source
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3704,6 +3704,7 @@ a.setAttribute('href', 'https://example.com/'); // change the content attribute
<li>The <dfn data-x-href="https://drafts.csswg.org/css-values/#vw">'vw'</dfn> unit</li>
<li>The <dfn data-x-href="https://drafts.csswg.org/css-values/#in">'in'</dfn> unit</li>
<li>The <dfn data-x-href="https://drafts.csswg.org/css-values/#px">'px'</dfn> unit</li>
<li>The <dfn data-x-href="https://drafts.csswg.org/css-values/#pt">'pt'</dfn> unit</li>
<li>The <dfn data-x-href="https://drafts.csswg.org/css-values/#funcdef-attr">'attr()'</dfn> function</li>
<li>The <dfn data-x-href="https://drafts.csswg.org/css-values/#math-function">math functions</dfn></li>
</ul>
Expand Down Expand Up @@ -28054,14 +28055,30 @@ was an English &lt;a href="/wiki/Music_hall">music hall&lt;/a> singer, ...</code
<p>Each <code>img</code> element has a <dfn>last selected source</dfn>, which must initially be
null.</p>

<p>Each <span>image request</span> has a <dfn>current pixel density</dfn>, which must initially be undefined.</p>
<p>Each <span>image request</span> has a <dfn>current pixel density</dfn>, which must initially be
1.</p>

<p>Each <span>image request</span> has <dfn>preferred density-corrected dimensions</dfn>, which is
either a struct consisting of a width and a height or is null. It must initially be null.</p>

<p>To determine the <dfn>density-corrected intrinsic width and height</dfn> of an
<code>img</code> element <var>img</var>:</p>

<ol>
<li><p>Let <var>dim</var> be <var>img</var>'s <span>current request</span>'s <span>preferred
density-corrected dimensions</span>.</p></li>

<p>When an <code>img</code> element has a <span>current pixel density</span> that is not 1.0, the
element's image data must be treated as if its resolution, in device pixels per <span
data-x="'px'">CSS pixels</span>, was the <span>current pixel density</span>. The image's
<dfn>density-corrected intrinsic width and height</dfn> are the <span data-x="intrinsic
dimensions">intrinsic width and height</span> after taking into account the <span>current pixel
density</span>.</p>
<li><p>If <var>dim</var> is null, set <var>dim</var> to <var>img</var>'s <span>intrinsic
dimensions</span>.</p></li>

<li><p>Set <var>dim</var>'s width to <var>dim</var>'s width divided by <var>img</var>'s
<span>current request</span>'s <span>current pixel density</span>.</p></li>

<li><p>Set <var>dim</var>'s height to <var>dim</var>'s height divided by <var>img</var>'s
<span>current request</span>'s <span>current pixel density</span>.</p></li>

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this then require the UA to retrieve the EXIF block directly or can the values be obtained indirectly through the use of the retrieval of XMP? I would hope that both would be supported, since a UA may want the more comprehensive XMP and could then make a single call to load image metadata.

Copy link
Contributor Author

@noamr noamr Jul 20, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, this is about reading EXIF directly., as that's also where orientation is read from and it keeps images small (XMP is verbose if you're just specifying numbers). It's possible to add XMP in the future, of course

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@noamr - As I read the text, what you say is not actually true. It simply says that you use the values from EXIF - not anything about where or how you obtain them. That therefore would indeed allow an arbitrary UA to choose whatever model works best for them.

It should be noted that while you may find more Photos (.jpg) with native EXIF, you will find many more images (GIF, PNG, AV1F, WebP, etc.) which use XMP instead - since it is the more modern and open (by being an ISO standard) approach to asset metadata.

So restricting XMP from this work would be quite limiting.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

True enough. The current proposal leaves this open to how EXIF is embedded in the image. The initial jpeg implementation is, however, limited to binary exif.

<li><p>Return <var>dim</var>.</p></li>
</ol>

<p class="example">For example, if the <span>current pixel density</span> is 3.125, that means
that there are 300 device pixels per <span data-x="'in'">CSS inch</span>, and thus if the image
Expand Down Expand Up @@ -28345,7 +28362,8 @@ was an English &lt;a href="/wiki/Music_hall">music hall&lt;/a> singer, ...</code
data-x="img-req-state">state</span> is <span data-x="img-all">completely
available</span>.</p></li>

<li><p>Update the presentation of the image appropriately.</p></li>
<li><p><span data-x="prepare an image for presentation">Prepare <var>current request</var>
for presentation</span> given <var>img</var>.</p></li>

<li><p>Set <span>current request</span>'s <span>current pixel density</span> to <var>selected
pixel density</var>.</p></li>
Expand Down Expand Up @@ -28583,13 +28601,16 @@ was an English &lt;a href="/wiki/Music_hall">music hall&lt;/a> singer, ...</code

<p>Each <span data-x="concept-task">task</span> that is <span data-x="queue a
task">queued</span> by the <span>networking task source</span> while the image is being
fetched must update the presentation of the image, but as each new body part comes in, it must
replace the previous image. Once one body part has been completely decoded, the user agent
must set the <code>img</code> element's <span>current request</span>'s <span
data-x="img-req-state">state</span> to <span data-x="img-all">completely available</span> and
<span>queue an element task</span> on the <span>DOM manipulation task source</span> given the
<code>img</code> element to <span data-x="concept-event-fire">fire an event</span> named <code
data-x="event-load">load</code> at the <code>img</code> element.</p>
fetched must update the presentation of the image, but as each new body part comes in, if the
user agent is able to determine the image's width and height, it must <span data-x="prepare an
image for presentation">prepare the <code>img</code> element's <span>current request</span>
for presentation</span> given the <code>img</code> element and replace the previous image.
Once one body part has been completely decoded, the user agent must set the <code>img</code>
element's <span>current request</span>'s <span data-x="img-req-state">state</span> to <span
data-x="img-all">completely available</span> and <span>queue an element task</span> on the
<span>DOM manipulation task source</span> given the <code>img</code> element to <span
data-x="concept-event-fire">fire an event</span> named <code data-x="event-load">load</code>
at the <code>img</code> element.</p>
<!--TODO what if the image is broken?
TODO change state and fire in the same task? -->
</dd>
Expand All @@ -28609,8 +28630,9 @@ was an English &lt;a href="/wiki/Music_hall">music hall&lt;/a> singer, ...</code
available</span>.</p></li>

<li><p>Otherwise, if the user agent is able to determine <var>image request</var>'s image's
width and height, and <var>image request</var> is <span>current request</span>, update the
<code>img</code> element's presentation appropriately and set <var>image request</var>'s
width and height, and <var>image request</var> is <span>current request</span>, <span
data-x="prepare an image for presentation">prepare <var>image request</var> for
presentation</span> given the <code>img</code> element and set <var>image request</var>'s
<span data-x="img-req-state">state</span> to <span data-x="img-inc">partially
available</span>.</p></li>

Expand Down Expand Up @@ -28653,7 +28675,8 @@ was an English &lt;a href="/wiki/Music_hall">music hall&lt;/a> singer, ...</code
<li><p>If <var>image request</var> is the <span>pending request</span>,
<span>abort the image request</span> for the <span>current request</span>,
<span>upgrade the pending request to the current request</span> and
update the <code>img</code> element's presentation appropriately.</p></li>
<span data-x="prepare an image for presentation">prepare <var>image request</var> for
presentation</span> given the <code>img</code> element.</p></li>

<li><p>Set <var>image request</var> to the <span data-x="img-all">completely
available</span> state.</p></li>
Expand Down Expand Up @@ -28687,7 +28710,6 @@ was an English &lt;a href="/wiki/Music_hall">music hall&lt;/a> singer, ...</code

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should have this here as this cannot be tested, right? Or do we plan to add orientation here as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be removed, and re-added if we put orientation in the spec which would make this testable.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

<ol>
<li><p>Forget <var>image request</var>'s <span data-x="img-req-data">image data</span>, if any.</p></li>

<li><p>Abort any instance of the <span data-x="concept-fetch">fetching</span> algorithm for
noamr marked this conversation as resolved.
Show resolved Hide resolved
<var>image request</var>, discarding any pending tasks generated by that algorithm.</p></li>
</ol>
Expand All @@ -28701,6 +28723,74 @@ was an English &lt;a href="/wiki/Music_hall">music hall&lt;/a> singer, ...</code
</ol>


<h6>Preparing an image for presentation</h6>

<p>To <dfn>prepare an image for presentation</dfn> for an <span>image request</span>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this still true? The discussion seems pretty settled.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should keep it until the issue is formally closed.

<var>req</var> given image element <var>img</var>:</p>

<ol>
<li><p>Let <var>exifTagMap</var> be the EXIF tags obtained from <var>req</var>'s <span
data-x="img-req-data">image data</span>, as defined by the relevant codec. <ref
spec=EXIF></p></li>

<li><p>Let <var>physicalWidth</var> and <var>physicalHeight</var> be the width and height
obtained from <var>req</var>'s <span data-x="img-req-data">image data</span>, as defined by the
relevant codec.</p></li>

<li><p>Let <var>dimX</var> be the value of <var>exifTagMap</var>'s tag <code
data-x="">0xA002</code> (<code data-x="">PixelXDimension</code>).</p></li>

<li><p>Let <var>dimY</var> be the value of <var>exifTagMap</var>'s tag <code
data-x="">0xA003</code> (<code data-x="">PixelYDimension</code>).</p></li>

<li><p>Let <var>resX</var> be the value of <var>exifTagMap</var>'s tag <code
data-x="">0x011A</code> (<code data-x="">XResolution</code>).</p></li>

<li><p>Let <var>resY</var> be the value of <var>exifTagMap</var>'s tag <code
data-x="">0x011B</code> (<code data-x="">YResolution</code>).</p></li>

<li><p>Let <var>resUnit</var> be the value of <var>exifTagMap</var>'s tag <code
data-x="">0x0128</code> (<code data-x="">ResolutionUnit</code>).</p></li>

<li><p>If either <var>dimX</var> or <var>dimY</var> is not a positive integer, then
return.</p></li>

<li><p>If either <var>resX</var> or <var>resY</var> is not a positive floating-point number, then
return.</p></li>

<li><p>If <var>resUnit</var> is not equal to <code data-x="">2</code> (<code
data-x="">Inch</code>), then return.</p></li>

<li><p>Let <var>widthFromDensity</var> be the value of <var>physicalWidth</var>, multiplied by 72
and divided by <var>resX</var>.</p></li>

<li><p>Let <var>heightFromDensity</var> be the value of <var>physicalHeight</var>, multiplied by
72 and divided by <var>resY</var>.</p></li>

<li><p>If <var>widthFromDensity</var> is not equal to <var>dimX</var> or
<var>heightFromDensity</var> is not equal to <var>dimY</var>, then return.</p></li>

<li>
<p>If <var>req</var>'s <span data-x="img-req-data">image data</span> is
<span>CORS-cross-origin</span>, then set <var>img</var>'s <span>intrinsic dimensions</span> to
<var>dimX</var> and <var>dimY</var>, scale <var>img</var>'s pixel data accordingly, and return.</p>
</li>

<li><p>Set <var>req</var>'s <span>preferred density-corrected dimensions</span> to a struct with
its width set to <var>dimX</var> and its height set to <var>dimY</var>.</p></li>

<li><p>Update <var>req</var>'s <code>img</code> element's presentation appropriately.</p></li>
</ol>

<p class="note">Resolution in EXIF is equivalent to <span data-x="'pt'">CSS points per
inch</span>, therefore 72 is the base for computing size from resolution.</p>

<p class="XXX">It is not yet specified what would be the case if EXIF arrives after the image is
already presented. See <a href="https://github.com/w3c/csswg-drafts/issues/4929">issue
#4929</a>.</p>



<h6>Selecting an image source</h6>

<p>When asked to <dfn>select an image source</dfn> for a given <code>img</code> or
Expand Down Expand Up @@ -29277,8 +29367,9 @@ was an English &lt;a href="/wiki/Music_hall">music hall&lt;/a> singer, ...</code
since this algorithm started, then let <span>pending request</span> be null and abort these
steps.</p>

<li><p>Let the <code>img</code> element's <span>last selected source</span> be <var>selected source</var>
and the <code>img</code> element's <span>current pixel density</span> be <var>selected pixel density</var>.</p></li>
<li><p>Let the <code>img</code> element's <span>last selected source</span> be <var>selected
source</var> and the <code>img</code> element's <span>current pixel density</span> be
<var>selected pixel density</var>.</p></li>

<li><p>Set the <var>image request</var>'s <span data-x="img-req-state">state</span> to <span
data-x="img-all">completely available</span>.</p></li>
Expand All @@ -29288,7 +29379,8 @@ was an English &lt;a href="/wiki/Music_hall">music hall&lt;/a> singer, ...</code

<li><p><span>Upgrade the pending request to the current request</span>.</p></li>

<li><p>Update the <code>img</code> element's presentation appropriately.</p></li>
<li><p><span data-x="prepare an image for presentation">Prepare <var>image request</var> for
presentation</span> given the <code>img</code> element.</p></li>

<li><p><span data-x="concept-event-fire">Fire an event</span> named <code
data-x="event-load">load</code> at the <code>img</code> element.</p></li>
Expand Down Expand Up @@ -125457,6 +125549,7 @@ INSERT INTERFACES HERE
Niklas Gögge,
Noah Mendelsohn,
Noah Slater,
Noam Rosenthal,
Noel Gordon,
Nolan Waite,
NoozNooz42,
Expand Down