On first search in web "too many 429 error responses" #2178

uhlmul · 2024-09-04T06:58:05Z

System OS

Windows

Python Version

3.11 (CPython)

Install Source

pip / PyPi

Install version / commit hash

4.2.7

Expected Behavior vs Actual Behavior

When doing the first search in the web interface, no results are returned.

Steps to reproduce - Ensure to include actual links!

spotdl web --web-use-output-dir --bitrate 160k --m3u playlist

Then do the first search in the web interface

Traceback

 spotdl web --web-use-output-dir --bitrate 160k --m3u playlist
Using cached web app. To update use the `--force-update-gui` flag.
Files are stored in current directory to save them to temporary directory disable the `web_use_output_dir` option
Starting web server

INFO:     Started server process [32240]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://localhost:8800 (Press CTRL+C to quit)
INFO:     ::1:52750 - "GET / HTTP/1.1" 200 OK
INFO:     ::1:52750 - "GET /assets/index-CqL-qAH-.js HTTP/1.1" 200 OK
INFO:     127.0.0.1:52751 - "GET /assets/index-DRB2_ZPb.css HTTP/1.1" 200 OK
INFO:     ('::1', 52753) - "WebSocket /api/ws?client_id=2424b15c-8aed-4bc5-a8cc-93c0bd98a2f7" [accepted]
INFO:     Client 2424b15c-8aed-4bc5-a8cc-93c0bd98a2f7 connected
INFO:     connection open
INFO:     ::1:52750 - "GET /api/version HTTP/1.1" 200 OK
INFO:     127.0.0.1:52751 - "GET /api/settings?client_id=2424b15c-8aed-4bc5-a8cc-93c0bd98a2f7 HTTP/1.1" 200 OK
Max Retries reached
INFO:     127.0.0.1:52751 - "GET /api/songs/search?query=shakira HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
urllib3.exceptions.ResponseError: too many 429 error responses

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\requests\adapters.py", line 667, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\urllib3\connectionpool.py", line 944, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\urllib3\connectionpool.py", line 944, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\urllib3\connectionpool.py", line 944, in urlopen
    return self.urlopen(
           ^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\urllib3\connectionpool.py", line 934, in urlopen
    retries = retries.increment(method, url, response=response, _pool=self)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\urllib3\util\retry.py", line 519, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.spotify.com', port=443): Max retries exceeded with url: /v1/artists/0EmeFodog0BfCgMzAIvKQp (Caused by ResponseError('too many 429 error responses'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\spotipy\client.py", line 270, in _internal_call
    response = self._session.request(
               ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\requests\sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\requests\sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\requests\adapters.py", line 691, in send
    raise RetryError(e, request=request)
requests.exceptions.RetryError: HTTPSConnectionPool(host='api.spotify.com', port=443): Max retries exceeded with url: /v1/artists/0EmeFodog0BfCgMzAIvKQp (Caused by ResponseError('too many 429 error responses'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\uvicorn\protocols\http\h11_impl.py", line 408, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\uvicorn\middleware\proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\fastapi\applications.py", line 292, in __call__
    await super().__call__(scope, receive, send)
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\starlette\applications.py", line 122, in __call__
    await self.middleware_stack(scope, receive, send)
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\starlette\middleware\errors.py", line 184, in __call__
    raise exc
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\starlette\middleware\errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\starlette\middleware\cors.py", line 83, in __call__
    await self.app(scope, receive, send)
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\starlette\middleware\exceptions.py", line 79, in __call__
    raise exc
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\starlette\middleware\exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\fastapi\middleware\asyncexitstack.py", line 20, in __call__
    raise e
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\fastapi\middleware\asyncexitstack.py", line 17, in __call__
    await self.app(scope, receive, send)
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\starlette\routing.py", line 718, in __call__
    await route.handle(scope, receive, send)
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\starlette\routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\starlette\routing.py", line 66, in app
    response = await func(request)
               ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\fastapi\routing.py", line 273, in app
    raw_response = await run_endpoint_function(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\fastapi\routing.py", line 192, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\starlette\concurrency.py", line 41, in run_in_threadpool
    return await anyio.to_thread.run_sync(func, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\anyio\to_thread.py", line 33, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\anyio\_backends\_asyncio.py", line 877, in run_sync_in_worker_thread
    return await future
           ^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\anyio\_backends\_asyncio.py", line 807, in run
    result = context.run(func, *args)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\spotdl\utils\web.py", line 375, in query_search
    return get_search_results(query)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\spotdl\utils\search.py", line 76, in get_search_results
    return Song.list_from_search_term(search_term)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\spotdl\types\song.py", line 198, in list_from_search_term
    Song.from_url(
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\spotdl\types\song.py", line 96, in from_url
    raw_artist_meta: Dict[str, Any] = spotify_client.artist(primary_artist_id)  # type: ignore
                                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\spotipy\client.py", line 396, in artist
    return self._get("artists/" + trid)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\spotdl\utils\spotify.py", line 195, in _get
    response = self._internal_call("GET", url, payload, kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\santi\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\spotipy\client.py", line 311, in _internal_call
    raise SpotifyException(
spotipy.exceptions.SpotifyException: http status: 429, code:-1 - /v1/artists/0EmeFodog0BfCgMzAIvKQp:
 Max Retries, reason: too many 429 error responses

Other details

No response

The text was updated successfully, but these errors were encountered:

R0GUE-A5H · 2024-09-08T18:36:56Z

will look into it
Edit: urllib3.exceptions.ResponseError: too many 429 error responses
try after some time

Concara3443 · 2024-09-17T14:41:17Z

Same problem, running the command with only the query as parameter

R0GUE-A5H · 2024-09-17T14:43:32Z

Same problem, running the command with only the query as parameter

Did you try in v4.2.8 or 4.2.7?

Concara3443 · 2024-09-18T13:02:06Z

Same problem, running the command with only the query as parameter

Did you try in v4.2.8 or 4.2.7?

On v4.2.8

R0GUE-A5H · 2024-09-21T09:31:41Z

Spotify's API rate limit is calculated based on the number of calls that your app makes to Spotify in a rolling 30 second window. If your app exceeds the rate limit for your app then you'll begin to see 429 error responses from Spotify's Web API, and you may hear from users about unexpected behavior that they have noticed while using your app. The limit varies depending on whether your app is in development mode or extended quota mode.

The issue you're encountering is related to Spotify's API rate limits. The limit is calculated based on the number of API requests made within a 30-second window.
I recommend retrying after 30 seconds. I may implement an automatic retry mechanism using the Retry-After header in the future, but for now, it's just a rate limit issue on Spotify's side.

I suggest we close this thread since the problem is related to the API rate limits.

Also it is working fine on my machine

C:\Users\xxxx\Downloads>spotdl web --web-use-output-dir --bitrate 160k --m3u playlist
Using cached web app. To update use the --force-update-gui flag.
Files are stored in current directory to save them to temporary directory disable the web_use_output_dir option
Starting web server

INFO:     Started server process [4416]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://localhost:8800 (Press CTRL+C to quit)
INFO:     ::1:3226 - "GET / HTTP/1.1" 200 OK
INFO:     ::1:3226 - "GET /assets/index-CqL-qAH-.js HTTP/1.1" 200 OK
INFO:     ::1:3227 - "GET /assets/index-DRB2_ZPb.css HTTP/1.1" 200 OK
INFO:     ('::1', 3228) - "WebSocket /api/ws?client_id=220a1c76-36f5-4faa-bb16-fa30fae72066" [accepted]
INFO:     Client 220a1c76-36f5-4faa-bb16-fa30fae72066 connected
INFO:     ::1:3226 - "GET /api/version HTTP/1.1" 200 OK
INFO:     connection open
INFO:     ::1:3229 - "GET /api/settings?client_id=220a1c76-36f5-4faa-bb16-fa30fae72066 HTTP/1.1" 200 OK
INFO:     ::1:3264 - "GET /api/songs/search?query=shakira HTTP/1.1" 200 OK

isle9 · 2024-09-21T17:10:47Z

The issue you're encountering is related to Spotify's API rate limits. The limit is calculated based on the number of API requests made within a 30-second window.
I recommend retrying after 30 seconds. I may implement an automatic retry mechanism using the Retry-After header in the future, but for now, it's just a rate limit issue on Spotify's side.

Waiting for a while does nothing. I've been getting this error for weeks now.

R0GUE-A5H · 2024-09-21T17:30:32Z

Ok, I just want to better understand how you're making the API requests. For example, if you're using the search feature, how many searches are you doing, and how quickly are they happening? It would really help me out if I could reproduce the rate limit issue on my side, so I can figure out a fix. Could you let me know exactly what you're doing or any patterns you've noticed that seem to trigger the problem? Thanks

isle9 · 2024-09-21T17:48:12Z

I personally just do spotdl download saved every day, and it seems like this script has stopped working around a month ago.

But it seems like my problems might be related to #2111 instead.

R0GUE-A5H · 2024-09-21T22:02:06Z

Please look into #2142. Many users have shared their solutions. One user mentioned this, and I recommend trying it:

using the --user-auth option fixed the issue for me (specifying client id/secret did not work). You'd have to be logged in on your browser though.

isle9 · 2024-09-21T22:04:59Z

Please look into #2142. Many users have shared their solutions. One user mentioned this, and I recommend trying it:

using the --user-auth option fixed the issue for me (specifying client id/secret did not work). You'd have to be logged in on your browser though.

Yeah, I've already fixed it by switching to my own app.

xnetcat · 2024-09-23T19:52:36Z

Duplicate of #2142

uhlmul added the Bug Unexpected problem or unintended behavior that needs to be fixed label Sep 4, 2024

xnetcat marked this as a duplicate of #2142 Sep 23, 2024

xnetcat closed this as completed Sep 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

On first search in web "too many 429 error responses" #2178

On first search in web "too many 429 error responses" #2178

uhlmul commented Sep 4, 2024

R0GUE-A5H commented Sep 8, 2024 •

edited

Loading

Concara3443 commented Sep 17, 2024

R0GUE-A5H commented Sep 17, 2024

Concara3443 commented Sep 18, 2024

R0GUE-A5H commented Sep 21, 2024

isle9 commented Sep 21, 2024

R0GUE-A5H commented Sep 21, 2024

isle9 commented Sep 21, 2024

R0GUE-A5H commented Sep 21, 2024

isle9 commented Sep 21, 2024

xnetcat commented Sep 23, 2024

On first search in web "too many 429 error responses" #2178

On first search in web "too many 429 error responses" #2178

Comments

uhlmul commented Sep 4, 2024

System OS

Python Version

Install Source

Install version / commit hash

Expected Behavior vs Actual Behavior

Steps to reproduce - Ensure to include actual links!

Traceback

Other details

R0GUE-A5H commented Sep 8, 2024 • edited Loading

Concara3443 commented Sep 17, 2024

R0GUE-A5H commented Sep 17, 2024

Concara3443 commented Sep 18, 2024

R0GUE-A5H commented Sep 21, 2024

isle9 commented Sep 21, 2024

R0GUE-A5H commented Sep 21, 2024

isle9 commented Sep 21, 2024

R0GUE-A5H commented Sep 21, 2024

isle9 commented Sep 21, 2024

xnetcat commented Sep 23, 2024

R0GUE-A5H commented Sep 8, 2024 •

edited

Loading