Skip to content

Commit 8807be5

Browse files
authored
Update Technical Report (#268)
1 parent 3f29a78 commit 8807be5

File tree

3 files changed

+8
-2
lines changed

3 files changed

+8
-2
lines changed

README.md

+1-1
Original file line numberDiff line numberDiff line change
@@ -253,6 +253,6 @@ Please cite the repo if you use the code in this repo.
253253
title = {Couler: Unified Machine Learning Workflow Optimization in Cloud},
254254
year = {2023},
255255
publisher = {GitHub},
256-
howpublished = {\url{https://github.com/couler-proj/couler}}.
256+
howpublished = {\url{https://github.com/couler-proj/couler/tree/master/docs/Technical-Report-of-Couler}}.
257257
}
258258
```
+7-1
Original file line numberDiff line numberDiff line change
@@ -1 +1,7 @@
1-
In this technical report, we delve into "Couler: Optimizing Machine Learning Workflows in Cloud", a framework designed to streamline the construction and execution of machine learning workflows. The report is segmented into three comprehensive chapters: Unified Programming Model, Implementation and Running Example.
1+
Machine Learning (ML) has become ubiquitous, fueling data-driven applications across various organizations. Contrary to the traditional perception of ML in research, ML workflows can be complex, resource-intensive, and time-consuming.
2+
Expanding an ML workflow to encompass a wider range of data infrastructure and data types may lead to larger workloads and increased deployment costs.
3+
Currently, numerous workflow engines are available (with over ten being widely recognized). This variety poses a challenge for end-users in terms of mastering different engine APIs. While efforts have primarily focused on optimizing ML Operations (MLOps) for a specific workflow engine, current methods largely overlook workflow optimization across different engines.
4+
5+
In this work, we design and implement Couler, a system designed for unified ML workflow optimization in the cloud.
6+
Our main insight lies in the ability to generate an ML workflow using natural language (NL) descriptions.
7+
We integrate Large Language Models (LLMs) into workflow generation, and provide a unified programming interface for various workflow engines. This approach alleviates the need to understand various workflow engines' APIs. Moreover, Couler enhances workflow computation efficiency by introducing automated caching at multiple stages, enabling large workflow auto-parallelization and automatic hyperparameters tuning. These enhancements minimize redundant computational costs and improve fault tolerance during deep learning workflow training.

0 commit comments

Comments
 (0)