From 5efa981b4d04ac925f34c83d9f9b159f56e28fac Mon Sep 17 00:00:00 2001 From: Katsie011 Date: Thu, 7 Dec 2023 08:10:38 +0000 Subject: [PATCH] Prettified Code! --- cerebrium/environments/initial-setup.mdx | 55 +++++++++++------------- examples/langchain.mdx | 1 - examples/sdxl.mdx | 1 - examples/transcribe-whisper.mdx | 2 +- 4 files changed, 27 insertions(+), 32 deletions(-) diff --git a/cerebrium/environments/initial-setup.mdx b/cerebrium/environments/initial-setup.mdx index 9e4b2177..79b54cee 100644 --- a/cerebrium/environments/initial-setup.mdx +++ b/cerebrium/environments/initial-setup.mdx @@ -32,39 +32,36 @@ Your config file can be named anything you want and can be placed anywhere on yo The parameters for your config file are the same as those which you would use as flags for a normal `cerebrium deploy` command. They're tabulated below for your convenience: -| Section | Parameter | Description | Type | Default | -| --- | --- | --- | --- | --- | -| `cerebrium.build` | A section for all the parameters governing your cortex builds | | | | -| | `predict_data` | The data to use to test your predict function on build. This is the same as the payload in a inference call | string | '{"prompt": "Here is some example predict data for your cerebrium.toml which will be used to test your predict function on build."}' | -| | `force_rebuild` | Whether to force a rebuild of your deployment | boolean | false | -| | `disable_animation` | Whether to disable the animation in the logs. | boolean | false | -| | `log_level` | Log level for the deployment | string | INFO | -| | `disable_deployment_confirmation` | Whether to disable the pre-deployment confirmation prompt | boolean | false | -| `cerebrium.deployment` | All the parameters related to the lifetime of your deployment live here. | | | -| | `python_version` | The Python version you would like to run | float | 3.9 | -| | `include` | Local files to include in the deployment | string | '[./*, main.py, requirements.txt, pkglist.txt, conda_pkglist.txt]' | -| | `exclude` | Local Files to exclude from the deployment | string | '[./.*, ./__*]' | -| `cerebrium.hardware` | Select the specifics for the machine you would like to run here. | | | -| | `gpu` | The GPU you would like to use. | string | AMPERE_A5000 | -| | `cpu` | The number of CPU cores to use | int | 2 | -| | `memory` | The amount of Memory to use in GB | float | 14.5 | -| | `gpu_count` | The number of GPUs to specify | int | 2 | -| `cerebrium.scaling` | All the parameters related to the auto scaling of your deployment when live are placed here. | | | -| | `min_replicas` | The minimum number of replicas to run. | int | 0 | -| | `max_replicas` | The maximum number of replicas to scale to. | int | \*plan limit | -| | `cooldown` | The number of seconds to keep your model warm after each request. It resets after every request ends. | int | 60 | -| `cerebrium.requirements` | All the parameters related to the packages you would like to install on your deployment are placed here. | | | -| | `pip` | The pip packages you would like to install. In the format 'module' = 'version_constraints' | dict (toml) | | -| | `conda` | The conda packages you would like to install. In the format 'module' = 'version_constraints' | dict (toml) | | -| | `apt` | The apt packages you would like to install. | list (toml) | | - - - +| Section | Parameter | Description | Type | Default | +| ------------------------ | -------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------- | ----------- | ------------------------------------------------------------------------------------------------------------------------------------ | +| `cerebrium.build` | A section for all the parameters governing your cortex builds | | | | +| | `predict_data` | The data to use to test your predict function on build. This is the same as the payload in a inference call | string | '{"prompt": "Here is some example predict data for your cerebrium.toml which will be used to test your predict function on build."}' | +| | `force_rebuild` | Whether to force a rebuild of your deployment | boolean | false | +| | `disable_animation` | Whether to disable the animation in the logs. | boolean | false | +| | `log_level` | Log level for the deployment | string | INFO | +| | `disable_deployment_confirmation` | Whether to disable the pre-deployment confirmation prompt | boolean | false | +| `cerebrium.deployment` | All the parameters related to the lifetime of your deployment live here. | | | +| | `python_version` | The Python version you would like to run | float | 3.9 | +| | `include` | Local files to include in the deployment | string | '[./*, main.py, requirements.txt, pkglist.txt, conda_pkglist.txt]' | +| | `exclude` | Local Files to exclude from the deployment | string | '[./.*, ./__*]' | +| `cerebrium.hardware` | Select the specifics for the machine you would like to run here. | | | +| | `gpu` | The GPU you would like to use. | string | AMPERE_A5000 | +| | `cpu` | The number of CPU cores to use | int | 2 | +| | `memory` | The amount of Memory to use in GB | float | 14.5 | +| | `gpu_count` | The number of GPUs to specify | int | 2 | +| `cerebrium.scaling` | All the parameters related to the auto scaling of your deployment when live are placed here. | | | +| | `min_replicas` | The minimum number of replicas to run. | int | 0 | +| | `max_replicas` | The maximum number of replicas to scale to. | int | \*plan limit | +| | `cooldown` | The number of seconds to keep your model warm after each request. It resets after every request ends. | int | 60 | +| `cerebrium.requirements` | All the parameters related to the packages you would like to install on your deployment are placed here. | | | +| | `pip` | The pip packages you would like to install. In the format 'module' = 'version_constraints' | dict (toml) | | +| | `conda` | The conda packages you would like to install. In the format 'module' = 'version_constraints' | dict (toml) | | +| | `apt` | The apt packages you would like to install. | list (toml) | | ## Config File Example ```toml -# This file was automatically generated by Cerebrium as a starting point for your project. +# This file was automatically generated by Cerebrium as a starting point for your project. # You can edit it as you wish. # If you would like to learn more about your Cerebrium config, please visit https://docs.cerebrium.ai/cerebrium/environments/initial-setup#config-file-example diff --git a/examples/langchain.mdx b/examples/langchain.mdx index ff4fa821..a3a3886e 100644 --- a/examples/langchain.mdx +++ b/examples/langchain.mdx @@ -148,7 +148,6 @@ We then integrate Langchain with a Cerebrium deployed endpoint to answer questio Your cerebrium.toml file is where you can set your compute/environment. Please make sure that the hardware you specify is a AMPERE_A5000, and that you have enough memory (RAM) on your instance to run the models. You cerebrium.toml file should look like: - ```toml [cerebrium.build] diff --git a/examples/sdxl.mdx b/examples/sdxl.mdx index 65a91f80..ff293ca2 100644 --- a/examples/sdxl.mdx +++ b/examples/sdxl.mdx @@ -104,7 +104,6 @@ def predict(item, run_id, logger): Your cerebrium.toml file is where you can set your compute/environment. Please make sure that the hardware you specify is a AMPERE_A5000 and that you have enough memory (RAM) on your instance to run the models. You cerebrium.toml file should look like: - ```toml [cerebrium.build] diff --git a/examples/transcribe-whisper.mdx b/examples/transcribe-whisper.mdx index 94331d5b..f9af07b2 100644 --- a/examples/transcribe-whisper.mdx +++ b/examples/transcribe-whisper.mdx @@ -123,7 +123,6 @@ In our predict function, which only runs on inference requests, we simply create Your cerebrium.toml file is where you can set your compute/environment. Please make sure that the hardware you specify is a AMPERE_A5000 and that you have enough memory (RAM) on your instance to run the models. You cerebrium.toml file should look like: - ```toml [cerebrium.build] @@ -157,6 +156,7 @@ openai-whisper [cerebrium.requirements.conda] ``` + To deploy the model use the following command: ```bash