fetchart hangs forever if server doesn't respond

[jackwilsdon]

Problem

Running this command in verbose (-vv) mode:

$ beet -vvv fetchart album:"TRON: Legacy R3C0NF1GUR3D"

Led to this problem:

user configuration: /home/jack/.config/beets/config.yaml
data directory: /home/jack/.config/beets
plugin paths: 
fetchart: google: Disabling art source due to missing key
fetchart: lastfm: Disabling art source due to missing key
Sending event: pluginload
library database: /mnt/tank/media/Music/.beets/library.db
library directory: /mnt/tank/media/Music
Sending event: library_opened
Parsed query: AndQuery([SubstringQuery('album', 'TRON: Legacy R3C0NF1GUR3D', fast=True)])
Parsed sort: NullSort()
fetchart: trying source filesystem for album Daft Punk - TRON: Legacy R3C0NF1GUR3D
fetchart: trying source itunes for album Daft Punk - TRON: Legacy R3C0NF1GUR3D
fetchart: getting URL: https://itunes.apple.com/search?term=Daft+Punk+TRON%3A+Legacy+R3C0NF1GUR3D&entity=album&media=music&limit=200
fetchart: iTunes search for 'Daft Punk TRON: Legacy R3C0NF1GUR3D' got no results
fetchart: trying source coverart for album Daft Punk - TRON: Legacy R3C0NF1GUR3D
fetchart: getting URL: https://coverartarchive.org/release/d0ddc2ca-5300-38ba-bb7d-83053369319c
<hangs indefinitely>

https://coverartarchive.org/release/d0ddc2ca-5300-38ba-bb7d-83053369319c redirects to https://archive.org/download/mbid-d0ddc2ca-5300-38ba-bb7d-83053369319c/index.json but archive.org is currently down.

Setup

• OS: NixOS 24.11 (Vicuna)
• Python version: 3.12.5
• beets version: 2.0.0 (although beet version reports 1.6.1 because 33ab22d didn't make it into 2.0.0)
• Turning off plugins made problem go away (yes/no): plugin issue

My configuration is:

fetchart:
  sources:
    - filesystem
    - itunes
    - coverart
    - amazon
    - albumart

[jackwilsdon]

Interestingly we do set a timeout but it hangs anyway:

beets/beetsplug/fetchart.py

Lines 271 to 272 in 03f1205

	if "timeout" not in send_kwargs:
	send_kwargs["timeout"] = 10

[snejus]

Hopefully archive.org is back soon: https://x.com/brewster_kahle/status/1844790609573277792

Trying to curl your URL gives me

$ curl https://coverartarchive.org/release/d0ddc2ca-5300-38ba-bb7d-83053369319c -L
curl: (60) SSL certificate problem: unable to get local issuer certificate
More details here: https://curl.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the webpage mentioned above.

fetchart returns a similar error and does not hang on my side:

fetchart: downloading image: https://coverartarchive.org/release/d0ddc2ca-5300-38ba-bb7d-83053369319c
fetchart: error fetching art: HTTPSConnectionPool(host='archive.org', port=443): Max retries exceeded with url: /download/mbid-d0ddc2ca-5300-38ba-bb7d-83053369319c/index.json (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)')))

[jackwilsdon]

I think I was attempting this whilst archive.org was actively being DDoS'd, so curl would hang at Trying <archive.org ip>:443... after the coverartarchive.org redirect.

I feel like there's probably still an issue here, as fetchart shouldn't hang if a service never responds.

[jackwilsdon]

I can't reproduce this now even if I try it with a test server that hangs in the same way archive.org did (no ACK).

[XBagon]

I still have this problem, either it makes it through after some (mostly very long) time or just hangs forever. But it never times out. I guess disabling fetchart or using a different source is best for now?

[snejus]

You can try excluding the coverart source by specifying every other one in your configuration:

fetchart:
  sources:
    - filesystem
-   - coverart
    - itunes
    - amazon
    - albumart
    - wikipedia
    - google
    - fanarttv
    - lastfm
    - cover_art_url

Since I've got store_source: yes configured, you can see which ones have been useful in my library:

$ beet list art_source::.. -f '$art_source' | sort | uniq -c | sort -n
      1 fanarttv
      7 wikipedia
     19 lastfm
     22 spotify
    118 cover_art_url
    320 coverart
    386 filesystem
   1950 itunes
   3985 bandcamp

[jackwilsdon]

Reopening as I can reproduce this again and I think I've found the root cause. We have a timeout set for _logged_get, but we don't set one when we're actually downloading the artwork:

beets/beetsplug/fetchart.py

Lines 364 to 366 in f8b1071

	self.request(
	candidate.url, stream=True, message="downloading image"
	)

By default, requests has no timeout, but it does end up timing out eventually (seemingly after a few minutes). We should set a reasonable timeout here (maybe 10s like in _logged_get?)

I've also noticed that the Wikipedia source has a timeout of 2500 seconds, which seems a bit high - maybe it was meant to be 2.5s?

[gskorokhod]

I am having a similar issue:

mayday@fedora ~/Music/Soulseek > beet -vvv fetchart
user configuration: /home/mayday/.config/beets/config.yaml
data directory: /home/mayday/.config/beets
plugin paths: 
lyrics: To use the google lyrics source, you must install the beautifulsoup4 module. See the documentation for further details.
lyrics: To use the genius lyrics source, you must install the beautifulsoup4 module. See the documentation for further details.
lyrics: To use the tekstowo lyrics source, you must install the beautifulsoup4 module. See the documentation for further details.
artresizer: method is ImageMagick
thumbnails: using ImageMagick to write metadata
thumbnails: using GIO to compute URIs
fetchart: google: Disabling art source due to missing key
fetchart: lastfm: Disabling art source due to missing key
fetchart: To use Spotify as an album art source, you must install the beautifulsoup4 module. See the documentation for further details.
Sending event: pluginload
library database: /home/mayday/.config/beets/library.db
library directory: /home/mayday/Music/Library
Sending event: library_opened
Parsed query: AndQuery([TrueQuery()])
Parsed sort: NullSort()
fetchart: The 1975 - A Brief Inquiry Into Online Relationships: has album art
fetchart: The 1975 - Being Funny in a Foreign Language: has album art
fetchart: The 1975 - I like it when you sleep, for you are so beautiful yet so unaware of it: has album art
fetchart: The 1975 - Notes on a Conditional Form: has album art
fetchart: Артек Электроника - Не смотри назад: has album art
fetchart: trying source filesystem for album Артек Электроника - Последний день в СССР
fetchart: trying source coverart for album Артек Электроника - Последний день в СССР
fetchart: getting URL: https://coverartarchive.org/release/9158cb0b-b0c8-474f-abaf-8b1ca886fab3
fetchart: downloading image: http://coverartarchive.org/release/9158cb0b-b0c8-474f-abaf-8b1ca886fab3/21769879272.jpg

Interestingly, the link opens just fine in my browser, so the archive server doesn't seem to be down at the moment?
UPD: never mind, that was just cached

[snejus]

but we don't set one when we're actually downloading the artwork:

beets/beetsplug/fetchart.py

Lines 364 to 366 in f8b1071

	self.request(
	candidate.url, stream=True, message="downloading image"
	)

@jackwilsdon This seems to call self.request / RequestMixin.request which calls _logged_get, I think?

class RequestMixin:
    """Adds a Requests wrapper to the class that uses the logger, which
    must be named `self._log`.
    """

    def request(self, *args, **kwargs):
        """Like `requests.get`, but uses the logger `self._log`.

        See also `_logged_get`.
        """
        return _logged_get(self._log, *args, **kwargs)

[jackwilsdon]

@snejus yeah you're right - not quite sure where I thought I was going with that one 🤦