Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Throttling to avoid API query limits #93

Open
ddffnn opened this issue Mar 8, 2024 · 1 comment
Open

Throttling to avoid API query limits #93

ddffnn opened this issue Mar 8, 2024 · 1 comment

Comments

@ddffnn
Copy link

ddffnn commented Mar 8, 2024

Flickr limits API usage, and after using flickr-download I appear to have gone over the limit because progress has slowed to a crawl. Looking at my app stats, I was over 3600 queries per hour for 3 hours, so I'm assuming my app is limited (at least temporarily).

I still have more pictures to download, so I'm hoping I can resume at full speed after waiting a while. That being said, I assume I'll run into the same situation again without throttling queries from flickr-download. I think a one query per second rate is the obvious choice.

Flickr api documentation:

Limits: Since the Flickr API is quite easy to use, it's also quite easy to abuse, which threatens all services relying on the Flickr API. To help prevent this, we limit the access to the API per key. If your application stays under 3600 queries per hour across the whole key (which means the aggregate of all the users of your integration), you'll be fine. If we detect abuse on your key, we will need to expire the key, or turn it off, in order to preserve the Flickr API functionality for others (including us!). We also track usage on other factors as well to ensure no API user abuses the system.

@beaufour
Copy link
Owner

Oh, good find. Yes, we should probably add an option to throttle requests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants