acoustid submission error: response is not valid JSON - 504 gateway time-out

[Extarys]

I'm adding new releases in MB database and I was curious after tagging why the fingerprint wasn't on the website and I received no errors so I explicitly tried with the submit comment.

Output:

$ beet -vvv submit "./"
user configuration: /home/user/.config/beets/config.yaml
data directory: /home/user/.config/beets
plugin paths: /home/user/.beets-fetchartist
/home/user/.local/lib/python3.8/site-packages/beets/mediafile.py:21: UserWarning: beets.mediafile is deprecated; use mediafile instead
  warnings.warn("beets.mediafile is deprecated; use mediafile instead")
Sending event: pluginload
inline: adding item field my_samplerate
inline: adding item field my_bitrate
inline: adding item field is_flac
inline: adding item field is_album
inline: adding item field multidisc
inline: adding item field vinyl
inline: adding item field compilation_display
inline: adding album field fileformat
inline: adding album field av_bitrate
inline: adding album field avg_samplerate
inline: adding album field avg_bitdepth
library database: /media/music/.beet_library.blb
library directory: /media/Nas/Music
Sending event: library_opened
chroma: /media/Nas/Music/KNEECAP/Singles/2017 - C.E.A.R.T.A [MP3 320]/01 - C.E.A.R.T.A.mp3: fingerprint exists, skipping
chroma: submitting MBID
chroma: submitting 1 fingerprints
chroma: acoustid submission error: response is not valid JSON
Sending event: cli_exit

I tried multiple tracks, I always get that error - I also confirmed my API key.

Config:

original_date: no
per_disc_numbering: yes
threaded: yes
clutter: ["Thumbs.DB", ".DS_Store", "*.m3u", ".pls"]
ignore: ["*~", "System Volume Information"]

import:
    languages: en de fr jp
    move: yes
    copy: no
    bell: yes

ui:
    color: yes
    colors:
        text_success: green
        text_warning: yellow
        text_error: red
        text_highlight: red
        text_highlight_minor: lightgray
        action_default: turquoise
        action: cyan

pluginpath:
  ~/.beets-fetchartist

plugins: inline convert bpm discogs lastgenre absubmit acousticbrainz scrub fromfilename chroma acousticbrainz beatport the ftintitle fetchart embedart extrafiles edit fetchartist

aunique:
    keys: albumartist album year
    disambiguators: albumtype year country label catalognum albumdisambig releasegroupdisambig
    bracket: '()'

set_fields:
    how: ''

item_fields:
    my_samplerate: str(round(samplerate / 1000))
    my_bitrate: str(round(bitrate/1000))
    is_flac: 1 if format == "FLAC" else 0
    is_album: 1 if albumtype.upper() == "ALBUM" else 0
    multidisc: 1 if disctotal > 1 else 0
    vinyl: 1 if media in ['12" Vinyl', 'Vinyl'] else 0
    compilation_display: '''Soundtracks'' if albumtype.upper() == ''SOUNDTRACK'' else ''Compilations'''
    #first_artist: albumartist.split(', ',1)[0:1][0]

album_fields:  
    fileformat: |
        formatList = []
        for item in items:
            formatList.append(item.format.upper())
        return formatList
    av_bitrate: |
        total = 0
        for item in items:
            total += item.bitrate
        return str(round(total / len(items) / 1000))
    avg_samplerate: |
        total = 0
        for item in items:
            total += item.samplerate
        return str(round(total / len(items) / 1000))
    avg_bitdepth: |
        total = 0
        for item in items:
            total += item.bitdepth
        return str(round(total / len(items)))

paths:
    #default: %upper{%left{%asciify{%the{$albumartist}},1}}/$albumartist/%if{$year,$year - ,%if{$original_year,$original_year - }}$album%aunique{} ($format)/%if{$multidisc,$disc-}$track - $title
    # Albums/A/ASCI Artist Name, The/[YEAR] ASCI Album Name, The [EP]/01 - Track Name.mp3
    # %bucket{%upper{%left{%the{$albumartist},1}}}/
    default: '%the{$albumartist}/%if{$original_year,$original_year,%if{$year,$year,0000}} - $album%aunique{} [%ifdef{how,$how }%if{$vinyl,Vinyl }%upper{$format}%if{$is_flac, $avg_samplerate-$avg_bitdepth, $av_bitrate}]/%if{$multidisc,$disc-}$track - $title'
    albumtype:single: '%the{$albumartist}/Singles/%if{$original_year,$original_year,%if{$year,$year,0000}} - $album%aunique{} [%ifdef{how,$how }%if{$vinyl,Vinyl }%upper{$format}%if{$is_flac, $avg_samplerate-$my_samplerate, $my_bitrate}]/%if{$multidisc,$disc-}$track - $title'
    albumtype:ep: '%the{$albumartist}/EPs/%if{$original_year,$original_year,%if{$year,$year,0000}} - $album%aunique{} [%ifdef{how,$how }%if{$vinyl,Vinyl }%upper{$format}%if{$is_flac, $avg_samplerate-$avg_bitdepth, $av_bitrate}]/%if{$multidisc,$disc-}$track - $title'
    albumtype:soundtrack: Soundtracks/$album%aunique{}%if{$year, ($year),%if{$original_year, ($original_year)}} [%ifdef{how,$how }%if{$vinyl,Vinyl }%upper{$format}%if{$is_flac, $avg_samplerate-$avg_bitdepth, $av_bitrate}]/%if{$vinyl,$track_alt,%if{$multidisc,$disc-}$track} - $title
    comp: $compilation_display/%if{$year,$year - ,%if{$original_year,$original_year - }}$album%aunique{} [%ifdef{how,$how }%upper{$format}%if{$is_flac, $avg_samplerate-$avg_bitdepth, $av_bitrate}]/%if{$multidisc,$disc-}$track - $title
    singleton: '%the{$artist}/Non-Album/$title%if{$original_year, ($original_year),%if{$year, ($year),}} [%ifdef{how,$how }%upper{$format}%if{$is_flac, $samplerate-$bitdepth, $my_bitrate}]'

replace:
    '[\\/]': _
    '^\.': _
    '[\x00-\x1f]': _
    '[<>:"\?\*\|]': _
    '\.$': _
    '\s+$': ''
    '^\s+': ''
    '^-': _
    '[\u2018\u2019]': ''''
    '[\u201c\u201d\u2033]': '"'

check:
    import: yes
    write-check: yes
    write-update: yes
    convert-update: yes
    threads: 4

ftintitle:
    auto: yes
    drop: no
    format: feat. {0}

lyrics:
  fallback: ''
  google_engine_ID: xxxxxxx:xxxxxxx
  google_API_key: xxxxxxx-xxxxxxx

lastgenre:
    min_weight: 80
    count: 5
    prefer_specific: no
    separator: '; '

extrafiles:
    patterns:
        artworkdir:
          - '[sS]cans/'
          - '[aA]rtwork/'
          - '[aA]rtworks/'
          - '[cC]overs/'
        lyrics:
          - '*.lrc'
        lyrics2:
          - 'Lyrics/'
        nfo:
          - 'album.nfo'
    paths:
        artworkdir: $albumpath/artwork
        lyrics: $albumpath/$filename.lrc
        lyrics2: $albumpath/Lyrics

embedart:
    # I dont want album art auto-embedded either
    auto: no

fetchart:
  auto: yes
  minwidth: 400
  maxwidth: 2000
  sources: filesystem coverart itunes amazon albumart

fetchartist:
  filename: "poster"

acousticbrainz:
    force: yes

absubmit:
  extractor: ~/streaming_extractor_music
  auto: yes

acoustid:
    apikey: xxxxxxx

chroma:
    auto: yes

Any tips?

[sampsyo]

Hmm; that's odd. Any chance you could view the API request and response to see if there's an underlying error? Observing the traffic with Wireshark or similar would work, as would just printing out the response inside the Python library.

[BNolet]

It's a gateway timeout it seems:

This is from trying to match a recording as well. Seems like the API is broken atm?

[Extarys]

Oh maybe. Thanks @BNolet I just install wireshark, I forgot I could use that.

The library returned error wasn't very helpful then, maybe it should've said: Service unavailable.

It seems beet (and picard) uses only album/artist to find matches indeed. II can't find a status page for the apis, we'll have to wait.

[sampsyo]

Yeah! A PR here would be welcome if anyone has a moment to add a better error message for 5xx error codes.

[BNolet]

Welp, Picard is working just fine so I'm going to use that to tag my library :) https://picard.musicbrainz.org/

[Extarys]

@BNolet On my end I can tag using beet, I just can't use fingerprints or upload them - same as Picard ATM.

Using picard if I try to upload a fingerprint I see the request in the logs with no error but it never shows on the website and it usually shows the fingerprint on the website pretty fast.

[BNolet]

@Extarys Ah maybe Picard is just working for matching recordings and not submissions then, sorry to hear that

[Extarys]

I think this need further investigation.

This is Picard log:

D: 11:00:57,461 /usr/lib/picard/picard/acoustid/__init__._run_next_task:224: Starting fingerprint calculator '/usr/bin/fpcalc' '/media/NAS/Music/KNEECAP/Singles/2017 - C.E.A.R.T.A [MP3 320]/01 - C.E.A.R.T.A.mp3'
D: 11:01:52,669 /usr/lib/picard/picard/acoustid/manager.submit:95: AcoustID: submitting total of 1 fingerprints...
D: 11:01:52,670 /usr/lib/picard/picard/acoustid/manager._batch_submit:111: AcoustID: submitting batch of 1 fingerprints (0 remaining)...
D: 11:01:52,671 /usr/lib/picard/picard/webservice/__init__.post:470: POST-DATA 'user=MYAPIKEY&fingerprint.0=AQADtErERKGSJFPA4skUNOuO8Joi6NwRakfTLR--HLmeCP1yNFK2Qj--iTOeC_0h_nh29B2h3_hBVZWHfELdWMJH4kKqFPpRyqnwXPjRZByqJgtn_JC3o40PP0T742jyoOyO68KJ1kdTp7iYH8-O8JsCZwsP5pHwHT1XJPWRfjweo3YTPEefoeGJncOXrEd-4fnx-gm083iOh4efYs-hnqjyIh94XA-xH0W0b_gMMU2O44vyIy9-Ge_xpPhhSQ3qIb-CPwcjaPmLUzim_Wik4dKhH19DrEOo9YGtK4J7oUcppWjs49yQ8ULlHD32HbOcB0czBo6FPpiZ49fx7WieD3p4NJl89DOakDp2psO3XEF_NM8C6bpxJTReTDl7hE-O-LiikMdz9FMFhzrxhB8Ov0ifog8iiYoE64fjg_lwBTr-YgqtHJOaHLWILQ8eaNpwH_1x-MF97D3k50If9AelVUavBV3R8MfKRIeO4yfa8LBOXDwGP8cPccM3Mke2H0ePK7ge43FyMIm04w-aPELyBfmPUoFzLwmsUzLaaKiYJCOaHdeFSiwR5h-ubDE85XiZ4-4xH0-Ia8F_tG3xw-olfMvh4E7gG7WGSQ55XEefgdWOY94LLfmO6uiTwxVj_MeDG5rf4cqCHj9sfdj14yv6C_3gG5odMGMeiMuDPjzkBv-Jf5CTcC6a9MH7IUUf9BSaJvTBOQsR53iw41K-4GqOKUOPUNqUoKmLeDyeGSM1cegTQsekG9VnNNR0XCEjnAe9w8-gR2GNNw92-BnuB2cY3NEha1EaCUcTa5Tw5IPhp-gDfdJRIVpSNF-LBzzxmUVzRPqUopoEncEp4oGzGxcD3fCRH_4VvNjCHyH-h-gFhwOpHU1afMnRE38yQ-uN50oFTdqX4C96fEbjo1aWDDcvkDh-XNmD68FfXAvxp_ihazqOv-imybjwo_FxHuoFc_iDr8eF9-AZBn7wH2cIXRfGLZEhXw5uPsGDFz2m_PgymvhzHDyeK9ij4_iN6olxJjAfyMKDx7A1_DO2d8XVwJGWE-Jx5gK1o6t0PApkHpMdaKmIn8PxiUb-6Kj-wM8VpD_wQ8uPS0PPWJiUQ8fBP_h8PMOb5OhDPEJ16D0-hehp_DjRpEkOWfFxEj2a5-Bx5biE8mi-HF-xhxgfXbgX40fHZE-Cs0AOU0e9hQZt5MWe4CUhT-Hxfah2OM_xELeUw9zx4D-eFv6h3LhnwgxO6cKnoz_YLsYV4_WhEVMeBTrzCT_e6PiD-FfQhxx-_EgflMfHDJPuQ9t0HHwOPVxw5Ti-HNqD_yjVCz8efNmhXSoatbjwEV1h9FmOn1B3MHbQOMdxQg-uSDbi6-j-GG_yBP2PNwi7oHkdSJ8SXEds7Dly5rhy4smOQ0udotoS3EJ_WNFy9KFxCX_RZGkOVdMx-cPN40_AJEen7MUH-ZhydBqenPjxh7iOJmSV4RNVwHfxYvOJUuOD6nh59OCoHH_QnkUzBrXQ5MNbH49xZsdR60K7BvXQOMWlBz-ao9uPTzpyNR3-oR-aH08SpcZP_EifKBzU7RvyOcFPCUyeox051D_-B3mQTDruHn_w5seP00d0NdDJSccOnQueH2HVB_6qwOgfFKkTQoqiiAWuY9x46Fk2Y7KD3OhutOvxSRZS6dAHu8hvo4mO4zrqIY35CH_ADdoavbh0nGieYup4_Hh05A7xrxJ6Bo7zwPGRH4ePslDk88JLIzx4XOKxP8iXoI8EbQ9zfEPO4Tkeoxd-uComK9HxFh-p42jUHP2Rh02gR0eWKCt6vD2e4zzyMDnUhhmNdnyC8Dn-olfR9OC-0IiDH3tefIeOH02rDa1iXDm05_CPakS-40XXI_1xamijYz305-h5UPIOzVpwog-mqIxxHdfxU_gRLrrgOB8xNkXa49qEJn-gi2BEF1ZPwQ-mB4_44M-hPcGWhjp8BW0G_dAVJcePXkT3I08WNFl0MEe-HOUL8QyP-phZtCqF5tRxMQvOC7pxyhK-B2EY4pM-7EcaHj_E8PikDBlnXJzR54jMBN-Lv_iEJmYMbdTxvfgRiv3RPHhvPB0aF5MWp0fuQz9exgMzNFV0PDqO6sWU-MST48ed4x9nJNPhH8dXFw9-jNKOE-FK8Iee43K4YNKOJlqClznsZMVDfEsOnegf7EEvfNFhHn8EKUfzGnIzC2182Cne7Kje4_ihPD3-HHKH6k_wPDiPw2MQWjciOj2u4yZ44VEe7PLwo8_hbDyeKFDN4IG548W_YntwF2J2fLsIalYu_GgUDz8uYW3RjMxR9sQZfLSQJ8d_oemPn1SwpcYlOEfIo9d6SArxo_kx97iicsGFxoJaY8qYQ12k48HOcKgVqcPRK5iaHYoS_EE_NM-MNjqOvYf1oNoOPhlOBu8tPAm-1fif4D9-PEPD6OhvfFES_OCDyyJ2Hf8H7YKpxkKtBX2iw3zwo6KCZzU0Sn1hzZgqHd1w6sOfHM71gpFIPKPxpjjEkXmg8MePqxTxCN0v5PuCBz_yHD5KHtNDHppt4-CP58EafYSOFx9x5tgP4wwgShAjFAEGCgmEUcIQQJShgoEhFSGOAQIAAMwQwhgAgghFGEBEGQWAMgoYI5ABQgAgBBMKAWABIQopApUhFhBgAAFOAimBAUAYoIxRAjlhhBQKcCKQAIYJAADAQBggCDRACAOAMRYIQI0ATgmigAgAOOAIAosgYxgCwrgkiBAEIKWAMAIhQYxTQBlnBDCOEQQFAURAoowxgAiKBBFIMASEYoAAgBBAAABBBWCMAAQUMIABYKwgRhhgECAKAgAEMQgwIoQAnDkiDCCWAYQARAwYIIDBAFAhFFQIAIIEA0oLBJhDBgEjDAACGIAgMYIAAIBxRjkLlGMKCSCMIsBQAaQSxCFgEKIMOYOUIsAAxgwUQDBKJABCAGAYARQsQgRwFABlkGUAAEOdYQoAgAQQQDrGEGAEMeAIIAAwRgICBGBmhOMAqQSFE4oYoIAFgDEKjACQAAIYEIgB45QSRBDAgIECGAQMkAoJIIRgkiHAkRHSEQQBIEgwACxQiBogFBKOKUYtQ4YwAgAShAAEhDNGAUQkMVYBA4gwwkggCCBGAaUMUkIISBQBCgpFEJFCEGAEwAgRKhAQgBoohEAGCaoAAMAABIxRQkpkBAFCOEAEAEABYIgQxDlhkCQAASKIowYBYAQSAgHnlCACUCMpcSIAQQwhAAhAiBAMGQAoAUiYIowSkgjFgEDACSEoAAowYxBSBkmklEIEMEWVEk4KAARQAimHjCDQMAGIYcYAYQQwADCCDAHCAUCQUswZABhAQFBCmAECGuCMBUAwo5BRSgAAAEBAUQAQBAIoJxBiQhFjBMACgcMMk9ZAhAgQTgCDBDEIAQcMAUpAI7RFhAEgEGCGEEaMEMAYJBVTACiHmEJIAWGAEIgZZISATAABBAAoECCYERAJYAEzzBhBiFSMKEIFIcQBQQQACEAmBTECEEQUMMYIAhgSghAihBDCAGKAYdZAIAhhFikDqCDMAIaAIJBYoQA&duration.0=222&mbid.0=65161f21-ae51-48f4-84c0-2f7f53662081&client=v8pQ6oyB&clientversion=2.3.1.final0&format=json'
D: 11:01:52,678 /usr/lib/picard/picard/webservice/ratecontrol.get_delay_to_next_request:113: ('api.acoustid.org', 80): First request
D: 11:01:52,680 /usr/lib/picard/picard/webservice/ratecontrol.increment_requests:138: ('api.acoustid.org', 80): Incrementing requests to: 1
D: 11:01:53,156 /usr/lib/picard/picard/webservice/ratecontrol.decrement_requests:146: ('api.acoustid.org', 80): Decrementing requests to: 0
D: 11:01:53,156 /usr/lib/picard/picard/webservice/__init__._handle_reply:419: Received reply for http://api.acoustid.org:80/v2/submit: HTTP 200 (OK) 
D: 11:01:53,159 /usr/lib/picard/picard/acoustid/manager._batch_submit_finished:142: AcoustID: 1 fingerprints successfully submitted
D: 11:01:53,160 /usr/lib/picard/picard/file.update:614: Updating file <MP3File '01 - C.E.A.R.T.A.mp3'>
D: 11:01:53,161 /usr/lib/picard/picard/acoustid/manager._batch_submit:100: AcoustID: submitted all fingerprints
D: 11:01:53,161 /usr/lib/picard/picard/webservice/ratecontrol._out_of_backoff:222: ('api.acoustid.org', 80): oobackoff; delay: 1000ms -> 500ms; slow start; window size 1.000 -> 2.000

Fingerprint was uploaded: https://musicbrainz.org/recording/65161f21-ae51-48f4-84c0-2f7f53662081/fingerprints

I think yesterday I didn't properly submit in Picard, that is why I thought it wasn't wokring. If it works in Picard but not in Beet, I think it can impact the number of fingerprints that will end up on the Brainz server. 😞

[skapazzo]

I also get the same error here (504), but querying the api with the same parameters through browser/wget/curl works fine.

[skapazzo]

Correction: if I send the request as plain text through curl it works, if I send it gzipped as I think pyacoustid does I get a 504. But I could easily have gotten something wrong in these tests.
The 504 might be a problem server side then. Anyone know who to talk to for that?

[skapazzo]

Further info: if I disable gzipping in acoustid.py everything seems to work correctly. My picard sends the request as plain text (urlencoded) and works.

[Extarys]

@skapazzo Can you explain how to disable gzip in acoustid? I just use it straight in beet though and the beet plugin config doesn't show much options. Do I need to change this directly in the python file?

[skapazzo]

Yes, I just hacked acoustid.py. I forked pyacoustid and uploaded the changes I used for testing. You can see the diff here: master...skapazzo:skapazzo-temporary
Beware, this was just a hack for testing, no idea about side effects.

[sampsyo]

Oh wow, thanks for tracking that down! Seems like something is wrong on the server side with gzipped requests.

Paging @lalinsky in case he hasn't seen this thread yet. It also looks like there is a related server issue filed at acoustid/acoustid-server#50.

[lalinsky]

Thanks guys. The service has been heavily overloaded during the last weeks, so I had to work on some frontend proxying and it's possible that some gzipped request support got broken. I'll have a look quickly.