-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add the quickstart #65
Open
Rishit-dagli
wants to merge
3
commits into
civo:main
Choose a base branch
from
Rishit-dagli:Rishit-dagli-guide
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
3 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,73 @@ | ||
--- | ||
sidebar_position: 2 | ||
title: Get Started with Kubeflow | ||
description: Learn how to get started with Civo's Kubeflow via an example. | ||
--- | ||
|
||
<head> | ||
<title>Kubeflow quickstart | Civo Documentation</title> | ||
</head> | ||
|
||
In this quickstart, focused mainly on Pipelines and Notebooks, we take a look at an example to train and run a machine learning model using Kubeflow. | ||
|
||
## Before you start | ||
|
||
Make sure you can log into your Kubeflow cluster by following the instructions at [Logging into your cluster](kubeflow-dashboard.md#logging-into-your-cluster). | ||
|
||
Make sure you can create a notebook and log into the notebook by following the instructions at [Creating a Kubeflow Notebook](creating-a-new-kubeflow-notebook.md). | ||
|
||
## Run a pipeline | ||
|
||
We will first compile a pipeline DSL that will train a model using the [MNIST dataset](https://ieeexplore.ieee.org/document/726791). Notice that this pipeline mainly has the following coponents which are run: hyperparameter tuning with Katib, creating a new volume for training, run a training job and finally serve the model using KServe. | ||
|
||
Now [create a new notebook instance](creating-a-new-kubeflow-notebook.md) and run the following commands: | ||
|
||
```bash | ||
git clone | ||
cd | ||
pip install -r requirements.txt | ||
python mnist-example.py | ||
``` | ||
|
||
This produces for us a `mnist-example.yaml` file that we will use to run our pipeline. | ||
|
||
We now use [Pipelines](kubeflow-dashboard.md/) and use the pipeline configuration we generated in the previous step to define the pipeline. Once you do so, you should create a new [experiment](kubeflow-dashboard.md/) and then trigger a run for the pipeline by going to the Run tab, clicking on the "create a run" button and choosing the pipeline you just created. | ||
|
||
<img src={require('./images/mnist-pipeline.png').default} style={{"background-color":"white"}} /> | ||
|
||
This would trigger a Kubeflow pipeline run and we would see the pipeline run in the Pipelines dashboard. Once the pipeline run is complete, we can see the model saved in the `end-to-end-pipeline-{ID}-model` volume we created earlier. | ||
|
||
## Predict using the model | ||
|
||
We can now use the model we trained to make predictions. We will use the [KFServing](kubeflow-dashboard.md/) component of Kubeflow to serve our model. You could now modify this code and run it in the Notebook instance you created to predict using the Serving endpoint. | ||
|
||
```python | ||
import numpy as np | ||
from PIL import Image | ||
import requests | ||
|
||
name = "kfaas-docs" | ||
namesapce = "my-profile" | ||
image_url = "https://raw.githubusercontent.com/kubeflow/katib/master/examples/v1beta1/kubeflow-pipelines/images/9.bmp" | ||
image = Image.open(requests.get(image_url, stream=True).raw) | ||
data = ( | ||
np.array(image.convert("L").resize((28, 28))) | ||
.astype(np.float) | ||
.reshape(-1, 28, 28, 1) | ||
) | ||
data_formatted = np.array2string( | ||
data, separator=",", formatter={"float": lambda x: "%.1f" % x} | ||
) | ||
json_request = '{{ "instances" : {} }}'.format(data_formatted) | ||
|
||
url = "http://{}-predictor-default.{}.svc.cluster.local/v1/models/{}:predict".format( | ||
name, namespace, name | ||
) | ||
response = requests.post(url, data=json_request) | ||
|
||
print("Prediction for the image") | ||
display(image) | ||
print(response.json()) | ||
Rishit-dagli marked this conversation as resolved.
Show resolved
Hide resolved
|
||
``` | ||
|
||
You should see the image and the JSON response our prediction endpoint returns for the image. |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
git clone is missing a repo address?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah yes, on this I had the code pushed to the civo gitlab on the link I shared in this pr, and I wanted to ask what would the best avenue to make a public repo by Civo would be?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah yes, on this I had the code pushed to the civo gitlab on the link I shared in this pr, and I wanted to ask what would the best avenue to make a public repo by Civo would be?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah yes, on this I had the code pushed to the civo gitlab on the link I shared in this pr, and I wanted to ask what would the best avenue to make a public repo by Civo would be?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Any Civo admin would be able to create a repo. I believe the plan was to have an ML examples repo.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Got it, would you or Josh be able to do it, I do not have access to do so
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe Josh has the required creds. I don't.