Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Uploading state snapshots to an object store #2197

Open
2 tasks
Tracked by #1807
tillrohrmann opened this issue Nov 1, 2024 · 2 comments · May be fixed by #2310
Open
2 tasks
Tracked by #1807

Uploading state snapshots to an object store #2197

tillrohrmann opened this issue Nov 1, 2024 · 2 comments · May be fixed by #2310
Assignees
@pcholakov

Comments

@tillrohrmann
Copy link
Contributor

tillrohrmann commented Nov 1, 2024 •
edited by pcholakov
Loading

Ideally, we support a wide range of different object stores and filesystems. That way users can run Restate also w/o an object store. Moreover, the upload process should happen in the background.

Libraries to look at could be: https://crates.io/crates/opendal and https://crates.io/crates/object_store.

Tasks
Beta Give feedback
@tillrohrmann tillrohrmann mentioned this issue Nov 1, 2024
Open
@pcholakov pcholakov self-assigned this Nov 11, 2024
@Xuanwo
Copy link

Xuanwo commented Nov 16, 2024 •
edited
Loading

As mentioned in #1882 (comment), please let me know if there's anything we can do from the opendal side.

We have recently polished our VISION to One Layer, All Storage with the Open Community, Solid Foundation, Fast Access, Object Storage First, and Extensible Architecture principles. We are also actively working towards our v1.0 release.

@pcholakov pcholakov linked a pull request Nov 18, 2024 that will close this issue
Open
@pcholakov
Copy link
Contributor

Hey @Xuanwo, I unfortunately only just saw your comment yesterday. We evaluated OpenDAL and decided to go with object_store for the first iteration of our snapshotting integration because of its narrower focus. Our requirements are very simple at the moment (just GET and multi-part PUT against a handful of services for the foreseeable future), but thanks very much for getting in touch and we'll certainly keep re-evaluating this as our needs evolve!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pcholakov pcholakov

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Introduce SnapshotRepository and object store integration restatedev/restate
3 participants
@pcholakov @Xuanwo @tillrohrmann