Read S3 Storage Without Object Versions

[dvnatanael]

Is it possible to use datachain.read_storage to read from an S3 bucket without the ListObjectVersions permission? I am just starting to check out DataChain, and I currently do not have this permission for the bucket in question.

The documentation on remote storage suggests that I should be able to pass {"version_aware": False} to the client_config parameter. However, the implementation of the S3 client always sets the pagination method to list_object_versions:

https://github.com/iterative/datachain/blob/b57275a36c552081b7e5d1f6bfb58b2a135e42fe/src/datachain/client/s3.py#L104-L114

Finally, are there other permissions that DataChain requires which are not listed in the documentation?

Thanks in advance!

[dmpetrov]

thank you @dvnatanael!
it's disabled in many buckets by default. I've created an issue #1103

[dvnatanael]

Thank you!

[dmpetrov]

@dvnatanael it should be fixed by #1127. Could you please check it?

[dvnatanael]

I can read the storage without the ListObjectVersions permission now. Thank you!