Remove R2 keys requirement

[o-az]

Since the data is public and freely available, why not make the R2 link accessible without having to manually add this link to a shell?

https://docs.indexed.xyz/dataset/awscli#aws-credentials

Since R2 offers the ability to make url publicly accessible w/o auth, this will make it easier and more accessible to developers to access the data as it will be a matter of a network request instead of needing to configure a shell.

[jeffling]

hey @o-az! thanks for your question. You can use https://data.indexed.xyz to download the data through http. However, unfortunately r2 requires the API key if you need to use the S3 API, which allows you to list the actual file.

You will get better performance if you use the data.indexed.xyz http path to download over using the S3 API, but you will need to use the S3 API to get the files list.

[o-az]

how do I use https://data.indexed.xyz? It 404s when I visit it

[hemaaanth]

You need to authenticate with the access key / secret key provided in the documentation, eg. page here: https://docs.indexed.xyz/dataset/rclone

We outline a few approaches to downloading & using the data in our documentation, but if there's a specific use case you have in mind happy to hear more and share some advice.