-
Notifications
You must be signed in to change notification settings - Fork 3
Open
Description
While uploading an object, we first write all contents in a temporary file before sending it to Hoku.
This is clearly bad for performance, especially for big files. Ideally, we would stream the bytes from the request directly to Hoku. We didn't do this in the first version because of:
- The Hoku SDK method
add_readerrequired aAsyncSeek(this is not true anymore) - We need a way to calculate the md5 of the file and pass it as metadata
So, writing to a temp file was the easiest way out of this.
One solution is to wrap the body request in an AsyncRead implementation that calculates the MD5 and passes it to add_reader. But we wouldn't be able to get the final md5 hash value before calling add_reader. So in addition to that, we would need a way to add metadata to an existing Hoku object (we don't have that Hoku operation right now).
Metadata
Metadata
Assignees
Labels
No labels