Replies: 10 comments
-
|
I'm doing around 500 files a day with a 20 seconds cooldown, may be way to high but no temp bans so far. |
Beta Was this translation helpful? Give feedback.
-
Yeah 20s is too high. You can at least lower it to 10s and stay super-stafe.
Not sure if you are aware, but auto-plugin does not actually use a cooldown delay (#374) and can easily get you a temporary ban. |
Beta Was this translation helpful? Give feedback.
-
|
Yeah, I disable the auto-plugin when I do that, I just add the files first then use the batch tag feature. Gonna give it a try with 10 seconds and will report back. |
Beta Was this translation helpful? Give feedback.
-
|
The sheer amount is most likely an issue here, not the cooldown. If the batch tagging was used on previously untagged archives then actual requests to HTML pages are made to find the archive and only then API request is sent. No matter how long the cooldown will be, if you consistently make requests to search specific archive and then fetch it's data via API it will be seen as mass-scraping and result in temp ban. In fact I believe 4s is a lot already when I upload 5 or 6 archives and have to wait quite a long time for them all to be processed. |
Beta Was this translation helpful? Give feedback.
-
I do not think that is true. Do it slowly enough and you should be fine.
5 or 6 archives will not pose a problem anyway and a cooldown is not needed at all. It is needed when you tag hundreds of archives, and I think 10k is not too different from something like 1k, because you would be banned a lot earlier than 10k if the cooldown is not safe. A 3s cooldown got me temporarily banned fairly easily when I tested batch tagging back then, so I thought 4s would be the minimum required. |
Beta Was this translation helpful? Give feedback.
-
EH is actively banning mass scrapers, as such if you are going to run a scrape over days it will undoubtly be catched and banned, even if you set 30s or 1m cooldown. You are fighting two things:
|
Beta Was this translation helpful? Give feedback.
-
|
yeah, it seems to be more a volume than cooldown thing. I ran 500 daily with a 10 seconds cooldown and today I got another temp ban near the end. I guess I will adjust and do it once or twice every week till I'm done. |
Beta Was this translation helpful? Give feedback.
-
Very interesting. I would think 10s is totally safe but it did get you banned, although theoretically there could be other issues like shared IP. It is not very relevant now, but did you get banned tagging 500+ archives when you used a 20s cooldown before? I will do some testing as well when I get time. |
Beta Was this translation helpful? Give feedback.
-
|
I'll move this to a discussion for now since the cooldown issue seems up to debate. |
Beta Was this translation helpful? Give feedback.
-
|
I get banned multiple times a day while batch tagging from EH, currently with a delay of 25sec. |
Beta Was this translation helpful? Give feedback.
-
This user got temporarily banned when he used batch tagging to tag 10k archives with the 4s default cooldown, so 4s could still be unsafe. It is also possible that he got banned due to other reasons like shared IP, so there is no concrete proof yet. I do not think we need to change this default setting yet and I am just leaving a report here. 10k is a lot and 4s could be safe for fewer archives. Still, it should be raised to 5s if more users report this problem.
Previously I thought 4s would be the minimum required to safely avoid a ban, but I did not test 4s that much and I said that mainly because 3s would occasionally get me banned. Maybe I should have recommended 5s but I wanted it to run as quickly as possible.
Beta Was this translation helpful? Give feedback.
All reactions