From 11516fa84983fd0e890dc86320ab104e33357d3d Mon Sep 17 00:00:00 2001 From: Michael Louis Date: Wed, 22 Nov 2023 17:34:45 -0500 Subject: [PATCH 1/9] Added predict documentation but unsure --- cerebrium/environments/initial-setup.mdx | 5 +++++ cerebrium/getting-started/quickstart.mdx | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/cerebrium/environments/initial-setup.mdx b/cerebrium/environments/initial-setup.mdx index 71e1f719..d39bfcf4 100644 --- a/cerebrium/environments/initial-setup.mdx +++ b/cerebrium/environments/initial-setup.mdx @@ -51,6 +51,9 @@ The parameters for your config file are the same as those which you would use as | `min_replicas` | The minimum number of replicas to run. | int | 0 | | `max_replicas` | The maximum number of replicas to scale to. | int | \*plan limit | | `cooldown` | The number of seconds to keep your model warm after each request. It resets after every request | int | 60 | +| `predict-data` | A JSON object for you to test inference on your endpoint when a deployment is made | json | {} | +| `disable-predict` | Whether to disable testing your predict function with input | boolean | false | + ## Config File Example @@ -67,4 +70,6 @@ disable_animation: false python_version: 3.9 min_replicas: 0 max_replicas: 30 +predict_data: '{"prompt": "The quick brown fox jumps over the lazy dog"}' +disable-predict: false ``` diff --git a/cerebrium/getting-started/quickstart.mdx b/cerebrium/getting-started/quickstart.mdx index 5c298f23..7eded3d2 100644 --- a/cerebrium/getting-started/quickstart.mdx +++ b/cerebrium/getting-started/quickstart.mdx @@ -32,7 +32,7 @@ def predict(item, run_id, logger): # Do something with parameters from item - return {"key": "value} + return {"key": "value"} ``` The Item class is where you define the parameters your model receives as well as their type. Item needs to inherit from BaseModel which uses Pydantic to validate request schemas. From 5cb9a667aa7417f2628770226484f4da1f8034cc Mon Sep 17 00:00:00 2001 From: milo157 Date: Wed, 22 Nov 2023 22:35:07 +0000 Subject: [PATCH 2/9] Prettified Code! --- cerebrium/environments/initial-setup.mdx | 1 - 1 file changed, 1 deletion(-) diff --git a/cerebrium/environments/initial-setup.mdx b/cerebrium/environments/initial-setup.mdx index d39bfcf4..9f9f208f 100644 --- a/cerebrium/environments/initial-setup.mdx +++ b/cerebrium/environments/initial-setup.mdx @@ -54,7 +54,6 @@ The parameters for your config file are the same as those which you would use as | `predict-data` | A JSON object for you to test inference on your endpoint when a deployment is made | json | {} | | `disable-predict` | Whether to disable testing your predict function with input | boolean | false | - ## Config File Example ```yaml From 7de3889f401eb3d86c1b2262dcd5203484be1528 Mon Sep 17 00:00:00 2001 From: Michael Katsoulis Date: Thu, 23 Nov 2023 09:09:51 +0000 Subject: [PATCH 3/9] Update initial-setup.mdx --- cerebrium/environments/initial-setup.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cerebrium/environments/initial-setup.mdx b/cerebrium/environments/initial-setup.mdx index 9f9f208f..7e5c6835 100644 --- a/cerebrium/environments/initial-setup.mdx +++ b/cerebrium/environments/initial-setup.mdx @@ -70,5 +70,5 @@ python_version: 3.9 min_replicas: 0 max_replicas: 30 predict_data: '{"prompt": "The quick brown fox jumps over the lazy dog"}' -disable-predict: false +disable_predict: false # You can set this to true if you'd like to override the predict-on-build functionality. ``` From cbe4a7fddaa987c00bc018c91b9bf792b4c841eb Mon Sep 17 00:00:00 2001 From: Michael Katsoulis Date: Thu, 23 Nov 2023 09:11:37 +0000 Subject: [PATCH 4/9] Minor edits --- cerebrium/environments/initial-setup.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/cerebrium/environments/initial-setup.mdx b/cerebrium/environments/initial-setup.mdx index 7e5c6835..a213c526 100644 --- a/cerebrium/environments/initial-setup.mdx +++ b/cerebrium/environments/initial-setup.mdx @@ -51,8 +51,8 @@ The parameters for your config file are the same as those which you would use as | `min_replicas` | The minimum number of replicas to run. | int | 0 | | `max_replicas` | The maximum number of replicas to scale to. | int | \*plan limit | | `cooldown` | The number of seconds to keep your model warm after each request. It resets after every request | int | 60 | -| `predict-data` | A JSON object for you to test inference on your endpoint when a deployment is made | json | {} | -| `disable-predict` | Whether to disable testing your predict function with input | boolean | false | +| `predict_data` | A JSON object for you to test inference on your endpoint when a deployment is made | json | {} | +| `disable_predict` | Whether to disable testing your predict function with input [Optional] | boolean | false | ## Config File Example From 6323ab4ad7162945ee60d7bc4be7a5fc75aa04f6 Mon Sep 17 00:00:00 2001 From: Michael Katsoulis Date: Thu, 23 Nov 2023 14:36:09 +0000 Subject: [PATCH 5/9] Update initial-setup.mdx --- cerebrium/environments/initial-setup.mdx | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/cerebrium/environments/initial-setup.mdx b/cerebrium/environments/initial-setup.mdx index a213c526..578dcce9 100644 --- a/cerebrium/environments/initial-setup.mdx +++ b/cerebrium/environments/initial-setup.mdx @@ -50,9 +50,9 @@ The parameters for your config file are the same as those which you would use as | `python_version` | The Python version you would like to run | float | 3.9 | | `min_replicas` | The minimum number of replicas to run. | int | 0 | | `max_replicas` | The maximum number of replicas to scale to. | int | \*plan limit | -| `cooldown` | The number of seconds to keep your model warm after each request. It resets after every request | int | 60 | -| `predict_data` | A JSON object for you to test inference on your endpoint when a deployment is made | json | {} | -| `disable_predict` | Whether to disable testing your predict function with input [Optional] | boolean | false | +| `cooldown` | The number of seconds to keep your model warm after each request. It resets after every request.| int | 60 | +| `predict_data` | A JSON object for you to test inference on your endpoint when a deployment is made. | json | {} | +| `disable_predict` | Whether to disable testing your predict function with input. [Optional] | boolean | false | ## Config File Example @@ -69,6 +69,6 @@ disable_animation: false python_version: 3.9 min_replicas: 0 max_replicas: 30 -predict_data: '{"prompt": "The quick brown fox jumps over the lazy dog"}' +predict_data: '{"prompt": "Here is some example predict data from your config.yaml which will be used to test your predict function on build."}' disable_predict: false # You can set this to true if you'd like to override the predict-on-build functionality. ``` From 07ede29a9cb2ceada7196aa350164bda55a6c2ca Mon Sep 17 00:00:00 2001 From: Katsie011 Date: Thu, 23 Nov 2023 14:36:26 +0000 Subject: [PATCH 6/9] Prettified Code! --- cerebrium/environments/initial-setup.mdx | 36 ++++++++++++------------ 1 file changed, 18 insertions(+), 18 deletions(-) diff --git a/cerebrium/environments/initial-setup.mdx b/cerebrium/environments/initial-setup.mdx index 578dcce9..88a072b4 100644 --- a/cerebrium/environments/initial-setup.mdx +++ b/cerebrium/environments/initial-setup.mdx @@ -35,24 +35,24 @@ Your config file can be named anything you want and can be placed anywhere on yo The parameters for your config file are the same as those which you would use as flags for a normal `cerebrium deploy` command. They're tabulated below for your convenience: -| Parameter | Description | Type | Default | -| ------------------- | ----------------------------------------------------------------------------------------------- | ------- | ------------------------------------------------------------------ | -| `name` | Name of the deployment | string | | -| `api_key` | API key for the deployment | string | not included for safety | -| `hardware` | Hardware to use for the deployment | string | GPU | -| `gpu_count` | The number of GPUs to specify | int | 2 | -| `cpu` | The number of CPU cores to use | int | 2 | -| `memory` | The amount of Memory to use in GB | int | 14.5 | -| `log_level` | Log level for the deployment | string | INFO | -| `include` | Local iles to include in the deployment | string | '[./*, main.py, requirements.txt, pkglist.txt, conda_pkglist.txt]' | -| `exclude` | Local Files to exclude from the deployment | string | '[./.*, ./__*]' | -| `disable_animation` | Whether to disable the animation in the logs. | boolean | false | -| `python_version` | The Python version you would like to run | float | 3.9 | -| `min_replicas` | The minimum number of replicas to run. | int | 0 | -| `max_replicas` | The maximum number of replicas to scale to. | int | \*plan limit | -| `cooldown` | The number of seconds to keep your model warm after each request. It resets after every request.| int | 60 | -| `predict_data` | A JSON object for you to test inference on your endpoint when a deployment is made. | json | {} | -| `disable_predict` | Whether to disable testing your predict function with input. [Optional] | boolean | false | +| Parameter | Description | Type | Default | +| ------------------- | ------------------------------------------------------------------------------------------------ | ------- | ------------------------------------------------------------------ | +| `name` | Name of the deployment | string | | +| `api_key` | API key for the deployment | string | not included for safety | +| `hardware` | Hardware to use for the deployment | string | GPU | +| `gpu_count` | The number of GPUs to specify | int | 2 | +| `cpu` | The number of CPU cores to use | int | 2 | +| `memory` | The amount of Memory to use in GB | int | 14.5 | +| `log_level` | Log level for the deployment | string | INFO | +| `include` | Local iles to include in the deployment | string | '[./*, main.py, requirements.txt, pkglist.txt, conda_pkglist.txt]' | +| `exclude` | Local Files to exclude from the deployment | string | '[./.*, ./__*]' | +| `disable_animation` | Whether to disable the animation in the logs. | boolean | false | +| `python_version` | The Python version you would like to run | float | 3.9 | +| `min_replicas` | The minimum number of replicas to run. | int | 0 | +| `max_replicas` | The maximum number of replicas to scale to. | int | \*plan limit | +| `cooldown` | The number of seconds to keep your model warm after each request. It resets after every request. | int | 60 | +| `predict_data` | A JSON object for you to test inference on your endpoint when a deployment is made. | json | {} | +| `disable_predict` | Whether to disable testing your predict function with input. [Optional] | boolean | false | ## Config File Example From 2d11d91524a676a87985ff5e3fd441ffea76de0f Mon Sep 17 00:00:00 2001 From: Michael Katsoulis Date: Thu, 23 Nov 2023 15:03:55 +0000 Subject: [PATCH 7/9] Add info on the predict_data to the quickstart --- cerebrium/getting-started/quickstart.mdx | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/cerebrium/getting-started/quickstart.mdx b/cerebrium/getting-started/quickstart.mdx index 7eded3d2..821ad97e 100644 --- a/cerebrium/getting-started/quickstart.mdx +++ b/cerebrium/getting-started/quickstart.mdx @@ -48,6 +48,11 @@ You need to define a function with the name **predict** which receives 3 params: As long as your **main.py** contains the above you can write any other Python code. Import classes, add other functions etc. + +Take note of the parameters you've defined in your `Item` class. +These are the parameters that you will pass to your model when you make an API call to your model endpoint. You can define as many parameters as you like and name them as you see fit. Just make sure to update the `predict_data` in your **config.yaml** so that you can test your model with some sample data. Otherwise, disable testing by setting `disable_predict` to `true`. + + ### Deploy model Then navigate to where your model code (specifically your `main.py`) is located and run the following command: @@ -79,3 +84,5 @@ Below are some links outlining some of the more advanced functionality that Cort - [Persistent Storage](../data-sharing-storage/persistent-storage): Store model weights and files locally for faster access. - [Long Running Tasks](../deployments/long-running-tasks): Execute long running tasks in the background. - [Streaming](../endpoints/streaming): Stream output live back to your endpoint + + From 3a3180ea7c3d22e891073066692c5ad6a7958d5b Mon Sep 17 00:00:00 2001 From: Katsie011 Date: Thu, 23 Nov 2023 15:04:14 +0000 Subject: [PATCH 8/9] Prettified Code! --- cerebrium/getting-started/quickstart.mdx | 4 ---- 1 file changed, 4 deletions(-) diff --git a/cerebrium/getting-started/quickstart.mdx b/cerebrium/getting-started/quickstart.mdx index 821ad97e..889d9412 100644 --- a/cerebrium/getting-started/quickstart.mdx +++ b/cerebrium/getting-started/quickstart.mdx @@ -48,11 +48,9 @@ You need to define a function with the name **predict** which receives 3 params: As long as your **main.py** contains the above you can write any other Python code. Import classes, add other functions etc. - Take note of the parameters you've defined in your `Item` class. These are the parameters that you will pass to your model when you make an API call to your model endpoint. You can define as many parameters as you like and name them as you see fit. Just make sure to update the `predict_data` in your **config.yaml** so that you can test your model with some sample data. Otherwise, disable testing by setting `disable_predict` to `true`. - ### Deploy model Then navigate to where your model code (specifically your `main.py`) is located and run the following command: @@ -84,5 +82,3 @@ Below are some links outlining some of the more advanced functionality that Cort - [Persistent Storage](../data-sharing-storage/persistent-storage): Store model weights and files locally for faster access. - [Long Running Tasks](../deployments/long-running-tasks): Execute long running tasks in the background. - [Streaming](../endpoints/streaming): Stream output live back to your endpoint - - From 04e446bd7f68ff3a32054466aa3d6e849ba99fd8 Mon Sep 17 00:00:00 2001 From: Katsie011 Date: Thu, 23 Nov 2023 15:30:03 +0000 Subject: [PATCH 9/9] Prettified Code! --- cerebrium/environments/initial-setup.mdx | 1 - 1 file changed, 1 deletion(-) diff --git a/cerebrium/environments/initial-setup.mdx b/cerebrium/environments/initial-setup.mdx index 8ba005ed..ae2704c6 100644 --- a/cerebrium/environments/initial-setup.mdx +++ b/cerebrium/environments/initial-setup.mdx @@ -35,7 +35,6 @@ Your config file can be named anything you want and can be placed anywhere on yo The parameters for your config file are the same as those which you would use as flags for a normal `cerebrium deploy` command. They're tabulated below for your convenience: - | Parameter | Description | Type | Default | | ------------------- | ------------------------------------------------------------------------------------------------ | ------- | ------------------------------------------------------------------ | | `name` | Name of the deployment | string | |