Skip to content

Commit

Permalink
Merge branch 'kats/cer-2624-toml-config-docs' of https://github.com/C…
Browse files Browse the repository at this point in the history 
…erebriumAI/documentation into kats/cer-2624-toml-config-docs
  • Loading branch information
@Katsie011
Katsie011 committed Dec 7, 2023
2 parents 96edb41 + 9ac34be commit 34ef00a
Show file tree
Hide file tree
Showing 2 changed files with 26 additions and 29 deletions.
53 changes: 25 additions & 28 deletions cerebrium/environments/initial-setup.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -32,34 +32,31 @@ Your config file can be named anything you want and can be placed anywhere on yo
The parameters for your config file are the same as those which you would use as flags for a normal `cerebrium deploy` command. They're tabulated below for your convenience:
| Section | Parameter | Description | Type | Default |
| --- | --- | --- | --- | --- |
| `cerebrium.build` | A section for all the parameters governing your cortex builds | | | |
| | `predict_data` | The data to use to test your predict function on build. This is the same as the payload in a inference call | string | '{"prompt": "Here is some example predict data for your cerebrium.toml which will be used to test your predict function on build."}' |
| | `force_rebuild` | Whether to force a rebuild of your deployment | boolean | false |
| | `disable_animation` | Whether to disable the animation in the logs. | boolean | false |
| | `log_level` | Log level for the deployment | string | INFO |
| | `disable_deployment_confirmation` | Whether to disable the pre-deployment confirmation prompt | boolean | false |
| `cerebrium.deployment` | All the parameters related to the lifetime of your deployment live here. | | |
| | `python_version` | The Python version you would like to run | float | 3.9 |
| | `include` | Local files to include in the deployment | string | '[./*, main.py]' |
| | `exclude` | Local Files to exclude from the deployment | string | '[./.*, ./__*]' |
| `cerebrium.hardware` | Select the specifics for the machine you would like to run here. | | |
| | `gpu` | The GPU you would like to use. | string | AMPERE_A5000 |
| | `cpu` | The number of CPU cores to use | int | 2 |
| | `memory` | The amount of Memory to use in GB | float | 14.5 |
| | `gpu_count` | The number of GPUs to specify | int | 2 |
| `cerebrium.scaling` | All the parameters related to the auto scaling of your deployment when live are placed here. | | |
| | `min_replicas` | The minimum number of replicas to run. | int | 0 |
| | `max_replicas` | The maximum number of replicas to scale to. | int | \*plan limit |
| | `cooldown` | The number of seconds to keep your model warm after each request. It resets after every request ends. | int | 60 |
| `cerebrium.requirements` | All the parameters related to the packages you would like to install on your deployment are placed here. | | |
| | `pip` | The pip packages you would like to install. In the format 'module' = 'version_constraints' | dict (toml) | |
| | `conda` | The conda packages you would like to install. In the format 'module' = 'version_constraints' | dict (toml) | |
| | `apt` | The apt packages you would like to install. | list (toml) | |
| Section | Parameter | Description | Type | Default |
| ------------------------ | -------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------ |
| `cerebrium.build` | A section for all the parameters governing your cortex builds | | | |
| | `predict_data` | The data to use to test your predict function on build. This is the same as the payload in a inference call | string | '{"prompt": "Here is some example predict data for your cerebrium.toml which will be used to test your predict function on build."}' |
| | `force_rebuild` | Whether to force a rebuild of your deployment | boolean | false |
| | `disable_animation` | Whether to disable the animation in the logs. | boolean | false |
| | `log_level` | Log level for the deployment | string | INFO |
| | `disable_deployment_confirmation` | Whether to disable the pre-deployment confirmation prompt | boolean | false |
| `cerebrium.deployment` | All the parameters related to the lifetime of your deployment live here. | | |
| | `python_version` | The Python version you would like to run | float | 3.9 |
| | `include` | Local files to include in the deployment | string | '[./*, main.py]' |
| | `exclude` | Local Files to exclude from the deployment | string | '[./.*, ./__*]' |
| `cerebrium.hardware` | Select the specifics for the machine you would like to run here. | | |
| | `gpu` | The GPU you would like to use. | string | AMPERE_A5000 |
| | `cpu` | The number of CPU cores to use | int | 2 |
| | `memory` | The amount of Memory to use in GB | float | 14.5 |
| | `gpu_count` | The number of GPUs to specify | int | 2 |
| `cerebrium.scaling` | All the parameters related to the auto scaling of your deployment when live are placed here. | | |
| | `min_replicas` | The minimum number of replicas to run. | int | 0 |
| | `max_replicas` | The maximum number of replicas to scale to. | int | \*plan limit |
| | `cooldown` | The number of seconds to keep your model warm after each request. It resets after every request ends. | int | 60 |
| `cerebrium.requirements` | All the parameters related to the packages you would like to install on your deployment are placed here. | | |
| | `pip` | The pip packages you would like to install. In the format 'module' = 'version_constraints' | dict (toml) | |
| | `conda` | The conda packages you would like to install. In the format 'module' = 'version_constraints' | dict (toml) | |
| | `apt` | The apt packages you would like to install. | list (toml) | |
## Config File Example
Expand Down
2 changes: 1 addition & 1 deletion cerebrium/misc/faster-model-loading.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ In this section below, we'll show you how to use **Tensorizer** to load your mod

### Installation

Add the following to your `[cerebrium.requirements.pip]` in your *cerebrium.toml* file to install **Tensorizer** in your deployment:
Add the following to your `[cerebrium.requirements.pip]` in your _cerebrium.toml_ file to install **Tensorizer** in your deployment:

```txt
tensorizer = ">=2.7.0"
Expand Down

0 comments on commit 34ef00a

Please sign in to comment.