You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Scenario: pipe a larger CSV file through OpenAI API to create embeddings to store somewhere. Basically a mini batch-workflow across external systems. This is common for other engines (airflow, etc.) but would be great to show how simple that also is in Restate.
Request -> pull (stream in) CSV file (maybe from S3, whereever) in a regular handler
file chunked (chunks of 100 rows) and sent to a worker handler, but in parallel (rpc, but only collect promise, don't await)
worker handler loops over rows and makes side-effect call to API, then store in DB.
The text was updated successfully, but these errors were encountered:
Scenario: pipe a larger CSV file through OpenAI API to create embeddings to store somewhere. Basically a mini batch-workflow across external systems. This is common for other engines (airflow, etc.) but would be great to show how simple that also is in Restate.
The text was updated successfully, but these errors were encountered: