[v2] Implement dataproxy

[EngHabu]

Motivation: Why do you think this is important?

Implement dataproxy

Goal: What should the final outcome look like, ideally?

Dataproxy

Describe alternatives you've considered

N/A

Propose: Link/Inline OR Additional context

No response

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[runllm]

The "dataproxy" in Flyte is a service that provides upload locations (e.g., signed URLs) for storing code bundles and other artifacts, which is essential for operations like flyte run and flyte deploy. The v2 implementation involves adding a CreateUploadLocation API, configuration for the dataproxy, and a service that can be started independently (e.g., on port 8088). The dataproxy enables secure, managed uploads to the Flyte blob store, supporting features like content length validation and integration with storage backends such as MinIO or S3. Example usage and setup instructions are provided in the referenced PR, including local development and testing steps with MinIO as the storage backend. The dataproxy is required for uploading artifacts and is being integrated into the Flyte SDK and cloud components for v2 support. There is no mention of alternatives in the provided sources, and the implementation is actively being developed for v2 compatibility.

For more details on the dataproxy implementation, setup, and usage, see the open PR and related documentation. Would you like a step-by-step breakdown of the implementation or example code for using the dataproxy?

Sources:

• Feat: add data proxy service (PR #6840)

---

This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @RunLLM to ask follow-up questions. See runllm.com for more!