[Bug]: Image expand broken in 0.19.6-beta

[dessalines]

Requirements

• Is this a bug report? For questions or discussions use https://lemmy.ml/c/lemmy_support
• Did you check to see if this issue already exists?
• Is this only a single bug? Do not put multiple bugs in one issue.
• Do you agree to follow the rules in our Code of Conduct?
• Is this a backend issue? Use the lemmy-ui repo for UI / frontend issues.

Summary

Its reported here that the new max thumbnail size for fetched images has far too small a default.

Its good that we added this, but it should probably be increased, I'm not sure what to.

Real life example: https://lemmy.ml/post/22217391

Context: https://lemmy.ml/post/22203615

Steps to Reproduce

1. Upload an image
2. Its too small for most web UIs and apps

Technical Details

NA

Version

main, release/v0.19

Lemmy Instance URL

lemmy.ml

[Nothing4You]

I don't think this is necessarily a backend issue.
Increasing (default) thumbnail resolution is not an universal solution for this, especially for webcomics or screenshots of text, as the resizing will quickly make text unreadable there.

I can see this being a bit challenging from the ui perspective, as you'd ideally want to know whether it's an image before you try to load it, as thumbnails can include metadata from non-image URLs.
When I expand an image, as a user I'd expect to load the full resolution version, to the extent permitted by screen space.

[dessalines]

I spose our use of the term thumbnail here is confusing.

This is actually a picture resizing done whenever pictrs fetches a thumbnail from another site (IE images from metatags, or images from sites), and ppl don't use the pictrs image upload. This was done to decrease the amount of storage necessary.

The web and app UI's should still be fine for their thumbnail displays for any image (because they use pictrs's on-demand resizing). The problem is that we have our max_resize set way too low.

See LemmyNet/lemmy#5107

[dessalines]

I really don't know what a good default should be here. This page discusses some different image sizes for various sites: https://www.istockphoto.com/blog/best-practices/design/what-are-standard-photo-sizes

1000-1200 seems like it would work.

[Nothing4You]

I know that PR, I linked it in the original issue.

You'll have to either optimize for storage or for quality, you can't have both.

It doesn't make a difference whether the image is uploaded to pictrs or external, as a separate thumbnail url will be generated regardless.
Example: https://lemmy.ml/post/17035654
URL: https://lemmy.ml/pictrs/image/3c1ae8cf-8712-431a-9f5f-53457bbaf46d.png
Thumbnail: https://lemmy.ml/pictrs/image/6505016a-62cc-4746-9b69-bb99e3363b2b.png

Would-be thumbnail with current settings: https://lemmy.ml/pictrs/image/3c1ae8cf-8712-431a-9f5f-53457bbaf46d.png?thumbnail=256
Would-be thumbnail with 1200x1200: https://lemmy.ml/pictrs/image/3c1ae8cf-8712-431a-9f5f-53457bbaf46d.png?thumbnail=1200
Text is still completely unreadable with that resizing.

The thumbnail is perfectly fine for the preview icon on the small square, but the expanded image should be full resolution.
The alternative would be that users always have to leave lemmy-ui to navigate to the original image to properly read it.

Additionally, lemmy-ui currently links to the thumbnail URL on the expanded image when it probably should link to the post URL, but that's a separate issue.

[dessalines]

I downloaded that image, and it's around 1800 width, so this one doesn't look distorted.

https://lemmy.ml/pictrs/image/3c1ae8cf-8712-431a-9f5f-53457bbaf46d.png?thumbnail=1800

Basically if we use pictrs image resizing at all, we're guaranteed to create distortions with fetched images that are too large. I still think the solution is to just choose a sane default that's bigger than 256.

It doesn't make a difference whether the image is uploaded to pictrs or external, as a separate thumbnail url will be generated regardless.

I'll have to check the code again for what value it puts in thumbnail url if its a local image upload.

[Nutomic]

Are you talking about the image that gets expanded when you click on the thumbnail? That one is definitely too small. Here are the images:

• https://lemmy.ml/pictrs/image/ed0e4095-f719-4f08-80f1-91853d98f5dd.png (118x256): post thumbnail
• https://lemmy.ml/pictrs/image/ed0e4095-f719-4f08-80f1-91853d98f5dd.png (118x256): expanded image after clicking on thumbnail

And from the api response:

"url": "https://lemmy.ml/pictrs/image/8d7a0f63-a706-44a1-beff-dba0643eceda.png" (405x879)
"thumbnail_url": "https://lemmy.ml/pictrs/image/ed0e4095-f719-4f08-80f1-91853d98f5dd.png" (118x256)

So its clear that the problem is with image expanding in lemmy-ui. Instead of loading the full-size image, it actually uses the thumbnail url. This used to work fine as thumbnails were not downscaled before, but now it obviously breaks. So lemmy-ui needs to be fixed to use the url for thumbnail expanding.

[Nothing4You]

Yes, that's exactly what I reported in #2793.

[dessalines]

Ok, I'll try to get this fixed now.

[aeharding]

Hi! Can this be reopened? This is still breaking apps such as Voyager that are using thumbnail_url. I don't think that voyager is necessarily at issue here, since it was doing the same logic as lemmy-ui before #2797 was merged. Of course Voyager can change its behavior but it would be best to have a bit of time to test before rolling out to lemmy.ml since this is a breaking change.

There are a few problems with the approach in #2797.

post.url doesn't preserve user privacy

The url can be an arbitrary instance. thumbnail_url allows instances to connect directly to their instance.

To your point @Nutomic, I don't think the following happens that often:

And from the api response:

"url": "https://lemmy.ml/pictrs/image/8d7a0f63-a706-44a1-beff-dba0643eceda.png" (405x879)
"thumbnail_url": "https://lemmy.ml/pictrs/image/ed0e4095-f719-4f08-80f1-91853d98f5dd.png" (118x256)

Usually, I see something like this:

Viewing the post https://lemmy.ml/post/22236108 (so I'm connected to lemmy.ml).

thumbnail_url=https://lemmy.ml/pictrs/image/ce86b43f-5f6a-4dad-9c87-82b6ece43efe.webp
url=https://lemmy.blahaj.zone/pictrs/image/3603d06f-3295-4128-8cbe-f364a3801dbf.webp

thumbnail_url is my home instance, lemmy.ml but it is low resolution. url is NOT my home instance (leaks IP address to lemmy.blahaj.zone) but is full size.

Before, Voyager and lemmy-ui used thumbnail_url which was fullsize and preserved privacy.

The default thumbnail_url size is way too low.

Let's pretend the former issue didn't exist. Clients still need a way to access a high quality thumbnails for non-media posts, for example links to news articles.

In this case, thumbnail_url is the only image available for use as the thumbnail since url is literally a link to the article, not an image. It looks OK on lemmy-ui because the thumbnails on lemmy-ui are tiny. But this is often not the case on mobile clients. Example (it looks especially bad on an actual phone):

Before	After

Solution?

Honestly I'm not sure, but I think clients need exposed an image link that's not compressed, and is hosted/proxied locally to preserve privacy.

I do recognize the instance hosting size issues but there has to be a better way to clean up these cached high quality images after some period of time, and potentially re-fetch as needed later?

[Nothing4You]

For the privacy aspect, instances do have the option to run an image proxy.
It still has some issues that prevent it from being fully usable due to issues like rate limits on certain upstream hosts, but this is an instance setting.

[Nutomic]

post.url doesn't preserve user privacy

As already mentioned, this can be handled by instance admins setting config value pictrs.image_mode = ProxyAllImages. For 0.20 it would make sense to use that value as default, and potentially remove StoreLinkPreviews entirely. (LemmyNet/lemmy#5176)

The default thumbnail_url size is way too low.

How about increasing it to 512? I wouldnt go higher than that because it will result in unnecessary network traffic for most clients which display small thumbnails, and dont request a specific size from the server.

[dessalines]

I'd be good with upping it to 512, but ya as stated above, this was a bug in the lemmy-ui code that wrongly used the thumbnail url for the expanded image.

The back-end proxyallimages setting is the way to do proxied images for security reasons.

[Nothing4You]

unfortunately this is still technically a breaking change

[dessalines]

I don't know if I'd call it breaking, there were no API changes in the thumbnail or url columns. Only changed the back-end resolution of the thumbnail column, which is entirely up to servers.

I just fixed broken lemmy-ui code that incorrectly was using the thumbnail column where it should've used the url one for images.

[aeharding]

The back-end proxyallimages setting is the way to do proxied images for security reasons.

I agree, but using thumbnail_url was the 'poor mans' approach prior to this feature. Again, I actually thought it was intentional, because lemmy-ui did it this way.

For 0.20 it would make sense to use that value as default, and potentially remove StoreLinkPreviews entirely.

This sounds like the best approach. Then, the thumbnail_url for, say,

https://www.space.com/space-exploration/tech/nasas-x-59-quiet-supersonic-jet-test-fires-engine-for-1st-time

would be the og:url, aka https://cdn.mos.cms.futurecdn.net/HGNXXdRpfZBRcCqgWVHaHZ-1200-80.jpg with ProxyAllImages off, and https://lemmy-alpha/api/v3/image_proxy?url=https://cdn.mos.cms.futurecdn.net/HGNXXdRpfZBRcCqgWVHaHZ-1200-80.jpg with ProxyAllImages on, right?

I'd love that approach. I would suggest reverting the changes until this proposal is merged (and ProxyAllImages is stable and on by default).

I don't know if I'd call it breaking

Not in terms of the API model, but in terms of user experience it is :)

How about increasing it to 512? I wouldnt go higher than that because it will result in unnecessary network traffic for most clients which display small thumbnails, and dont request a specific size from the server.

Social graph images are recommended to be 1200 x 630 on high resolution devices (facebook link warning). Again, thumbnail_url is the only option for link preview images for non-image content, like news articles etc. However, the pict-rs resizing logic is a bit strange last I checked - whatever algorithm it is using for downsampling reduces quality more than necessary for an images' dimensions. Its one of the reasons Voyager avoids image downscaling on all but the smallest images (like 30x30 community/user icons)

[aeharding]

I just learned that 0.19.6 was released, this is quite disappointing.

[dessalines]

I checked a few other apps (jerboa, thunder), and luckily they weren't using my bad lemmy-ui code for post images.

I'd be up for using proxyallimages as the default in a future release, but that's a long way away yet.

If I were you I'd just fix the code that's incorrectly using the thumnail column for full size images. There's no guarantee of the picture resolution for that column, for any server. It's a server-specific setting.

[aeharding]

It doesn't really matter. Voyager was broken, you guys were aware, and despite that a release was cut.

I'll release an update, but come on. Releases take time to propagate! Even a week would have been good. Ugh!

[dessalines]

I apologize for that, we'd been delaying this release for months so we've been tryisg to get it out for a while. We also have very few ppl helping test our release builds, so the picture resolution change would've gotten discovered sooner if we had more testers.

The next release has a lot of breaking api and federation changes, so we'll give at least a month or so for app devs to prepare for that one.

[dessalines]

^ Oops, misclick on mobile.

[aeharding]

Thanks for the apology.

Another problem with this approach is regarding random remote instances going down while viewing your feed.

I think originally Voyager used post.url directly but some instance was down and Voyager users couldn't view the image, but when they checked lemmy-ui it worked because it used post.thumbnail_url which is local to your instance. So I changed Voyager to use thumbnail_url which is more reliable to instance outages (instead of the remote instance and your own instance needing to be up to view a post image, you only need your local instance to be up).

Another example is if a post goes viral from a small instance, with this change all those users are gonna hit the small instance. Versus using thumbnail_url they will be "proxied".

So I hope proxying (and caching) fullsize images on the local instance is prioritized.