important notice for instance hosters and public v7 api users (api.cobalt.tools)

[wukko]

what happened?

several bad actors are continuously scraping public cobalt instances for youtube videos. as a result, many (sometimes personal) instances either run out of bandwidth, cookies, or experience reduced performance. last week it got especially bad, completely knocking out several community instances.

in an attempt to stop this, i asked @hyperdefined and @KwiatekMiki to take down their instance lists. it didn't help much, as they've both been cached or saved via other means. the scraping continues, more instances become affected, hosters don't know what to do.

what are we doing to stop this?

we've implemented several types of authentication that hosters can use to protect their instances. they can use turnstile, api keys, or both at the same time.

as for protecting the main instance, we've enabled turnstile and api keys. we plan on keeping it this way for the foreseeable future.

if you use the old v7 api (/api/json), it will be shut down on november 11th, 2024. unfortunately, scraping gets worse as cobalt becomes more known, and we can't keep up the free public api without impacting the regular users of cobalt.tools. you can host your own cobalt instance to keep using the cobalt api after that date. you can see a timeline for public api brownout in this post.

instance lists will not go back up as-is. all future lists will be opt-in and instances must be added only after hoster's explicit consent to participate.

it really sucks that it has come down to this, but we have no other choice :(

i have an instance, what can i do to protect it?

i made an instruction document that will guide you through the process of configuring cloudflare turnstile on your instance. it also covers the configuration of api keys in case you don't use your instance from a web interface.

if you're not using the latest cobalt api package and want to use all the security features, now is the perfect time to upgrade. aside from new features, you will also be able to use cobalt without hosting your own frontend instance. all you need to do is add your processing instance domain in cobalt settings. support for api keys in web interface is coming very soon!

who is scraping instances?

we have several suspects, but prefer to not reveal this information for sake of not giving them any publicity (that they clearly crave).

better docs

as a part of actions taken, we updated all documentation for cobalt on github (including the main readme), it's easier than ever to read and navigate: https://github.com/imputnet/cobalt

what does this all mean?

if you don't host an instance or don't use the cobalt api, you don't have to do anything and nothing will change for you!

cobalt.tools isn't affected and will keep working for the foreseeable future, nothing about its availability will change.

[wukko]

if you have any suggestions, feel free to leave them in comments here :3

[DataM0del]

Oh, cool.

[DataM0del]

I wonder how they could ever patch it... 💀

[AngeloD2022]

I wonder how they could ever patch it... 💀

Object.defineProperty(navigator.__proto__, 'webdriver', { get() { return false }})

[DataM0del]

Anyways, I've been working on it for a while, but I'm currently rewriting it to use undetected_chromedriver. That should fix the problem, or at least most of it.

[WhenTheBegininging]

fix cobalt

[wukko]

just posted the timeline for old api brownout: #876

[Szprinktrap]

can't have nice things on the internet :(
at least cobalt is open source and selfhostable tho

[trainingdummy]

It seems I picked exactly the wrong day to discover this API. 😂

[axellse]

even if you do host your own instance its not backwards compatible with the old api, is this something that could be added?

[wukko]

no, there won't be backwards compatibility with v7 schema, just update your app to follow the new (improved) schema: https://github.com/imputnet/cobalt/blob/main/docs/api.md