Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs : Update installation-deployments.md #706

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
177 changes: 133 additions & 44 deletions docs/installation-deployments.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,17 @@ Check out the [API docs](https://portkey.ai/docs/welcome/make-your-first-request
## Local Deployment

1. Do [NPM](#node) or [Bun](#bun) Install
2. Run a [Node.js Server](https://github.com/Portkey-AI/gateway/blob/main/docs/installation-deployments.md#run-a-nodejs-server)
2. Run a [Node.js Server](#nodejs-server)
3. Deploy on [App Stack](#deploy-to-app-stack)
4. Deploy on [Cloudflare Workers](https://github.com/Portkey-AI/gateway/blob/main/docs/installation-deployments.md#deploy-to-cloudflare-workers)
5. Deploy using [Docker](https://github.com/Portkey-AI/gateway/blob/main/docs/installation-deployments.md#deploy-using-docker)
6. Deploy using [Docker Compose](https://github.com/Portkey-AI/gateway/blob/main/docs/installation-deployments.md#deploy-using-docker-compose)
4. Deploy on [Cloudflare Workers](#cloudflare-workers)
5. Deploy using [Docker](#docker)
6. Deploy using [Docker Compose](#docker-compose)
7. Deploy on [Replit](#replit)
8. Deploy on [Zeabur](https://github.com/Portkey-AI/gateway/blob/main/docs/installation-deployments.md#deploy-to-zeabur)
8. Deploy on [Zeabur](#zeabur)
9. Deploy with [Supabase Functions](#supabase-functions)
10. Deploy using [Fastly](#fastly)

---

### Node

Expand All @@ -39,7 +43,35 @@ $ bunx @portkey-ai/gateway

<br>

# Deploy to App Stack
---

### NodeJS Server

1. Clone the Repository

```sh
git clone https://github.com/portkey-ai/gateway
```

2. Install the NPM Dependencies

```sh
cd gateway
npm i
npm run build
```

3. Run the Server

```sh
node build/start-server.js
```

<br>

---

### Deploy to App Stack
F5 Distributed Cloud
1. [Create an App Stack Site](https://docs.cloud.f5.com/docs/how-to/site-management/create-voltstack-site)

Expand Down Expand Up @@ -94,7 +126,8 @@ curl --request POST \
--data '{"metadata": {"name": "$DISTRIBUTED_CLOUD_SERVICE_NAME","namespace": "$DISTRIBUTED_CLOUD_NAMESPACE","labels": {},"annotations": {},"description": "","disable": false},"spec": {"origin_servers": [{"k8s_service": {"service_name": "$DISTRIBUTED_CLOUD_SERVICE_NAME.$DISTRIBUTED_CLOUD_APP_STACK_NAMESPACE","site_locator": {"site": {"tenant": "$DISTRIBUTED_CLOUD_TENANT_ID","namespace": "system","name": "$DISTRIBUTED_CLOUD_APP_STACK_SITE"}},"inside_network": {}},"labels": {}}],"no_tls": {},"port": 8787,"same_as_endpoint_port": {},"healthcheck": [],"loadbalancer_algorithm": "LB_OVERRIDE","endpoint_selection": "LOCAL_PREFERRED","advanced_options": null}}'
```
or [use the UI](https://docs.cloud.f5.com/docs/how-to/app-networking/origin-pools)
5. Create an HTTP Load Balancer, including header injection of Portkey provider and credentials

6. Create an HTTP Load Balancer, including header injection of Portkey provider and credentials
```shell
curl --request POST \
--url https://$DISTRIBUTED_CLOUD_TENANT.console.ves.volterra.io/api/config/namespaces/$DISTRIBUTED_CLOUD_NAMESPACE/http_loadbalancers \
Expand All @@ -103,7 +136,8 @@ curl --request POST \
--data '{"metadata": {"name": "$DISTRIBUTED_CLOUD_SERVICE_NAME","namespace": "$DISTRIBUTED_CLOUD_NAMESPACE","labels": {},"annotations": {},"description": "","disable": false},"spec": {"domains": ["$PORTKEY_GATEWAY_FQDN"],"https_auto_cert": {"http_redirect": true,"add_hsts": false,"tls_config": {"default_security": {}},"no_mtls": {},"default_header": {},"enable_path_normalize": {},"port": 443,"non_default_loadbalancer": {},"header_transformation_type": {"default_header_transformation": {}},"connection_idle_timeout": 120000,"http_protocol_options": {"http_protocol_enable_v1_v2": {}}},"advertise_on_public_default_vip": {},"default_route_pools": [{"pool": {"tenant": "$DISTRIBUTED_CLOUD_TENANT_ID","namespace": "$DISTRIBUTED_CLOUD_NAMESPACE","name": "$DISTRIBUTED_CLOUD_SERVICE_NAME"},"weight": 1,"priority": 1,"endpoint_subsets": {}}],"origin_server_subset_rule_list": null,"routes": [],"cors_policy": null,"disable_waf": {},"add_location": true,"no_challenge": {},"more_option": {"request_headers_to_add": [{"name": "x-portkey-provider","value": "$PORTKEY_PROVIDER","append": false},{"name": "Authorization","value": "Bearer $PORTKEY_PROVIDER_AUTH_TOKEN","append": false}],"request_headers_to_remove": [],"response_headers_to_add": [],"response_headers_to_remove": [],"max_request_header_size": 60,"buffer_policy": null,"compression_params": null,"custom_errors": {},"javascript_info": null,"jwt": [],"idle_timeout": 30000,"disable_default_error_pages": false,"cookies_to_modify": []},"user_id_client_ip": {},"disable_rate_limit": {},"malicious_user_mitigation": null,"waf_exclusion_rules": [],"data_guard_rules": [],"blocked_clients": [],"trusted_clients": [],"api_protection_rules": null,"ddos_mitigation_rules": [],"service_policies_from_namespace": {},"round_robin": {},"disable_trust_client_ip_headers": {},"disable_ddos_detection": {},"disable_malicious_user_detection": {},"disable_api_discovery": {},"disable_bot_defense": {},"disable_api_definition": {},"disable_ip_reputation": {},"disable_client_side_defense": {},"csrf_policy": null,"graphql_rules": [],"protected_cookies": [],"host_name": "","dns_info": [],"internet_vip_info": [],"system_default_timeouts": {},"jwt_validation": null,"disable_threat_intelligence": {},"l7_ddos_action_default": {},}}'
```
or [use the UI](https://docs.cloud.f5.com/docs/how-to/app-networking/http-load-balancer)
6. Test the service

7. Test the service
```shell
curl --request POST \
--url https://$PORTKEY_GATEWAY_FQDN/v1/chat/completions \
Expand Down Expand Up @@ -137,6 +171,7 @@ in addition to the response headers, you should get a response body like
}
```

---

### Cloudflare Workers

Expand All @@ -159,33 +194,10 @@ npm install
npm run deploy
```

For more details, refer to [Cloudflare Workers official](https://developers.cloudflare.com/workers/).
<br>

### NodeJS Server

1. Clone the Repository

```sh
git clone https://github.com/portkey-ai/gateway
```

2. Install the NPM Dependencies

```sh
cd gateway
npm i
npm run build
```

<br>

3. Run the Server

```sh
node build/start-server.js
```

<br>
---

### Docker

Expand All @@ -199,6 +211,8 @@ For more information on the Docker image, check [here](https://hub.docker.com/r/

<br>

---

### Docker Compose

1. Download Compose File from the Repository:
Expand All @@ -214,50 +228,125 @@ docker compose up -d
```
> The service is now running and listening on port 8787

For more details, refer to [Docker Compose official](https://docs.docker.com/compose/).
<br>

---

### Replit

[![Deploy on Replit](https://replit.com/badge?caption=Deploy%20on%20Replit)](https://replit.com/@portkey/AI-Gateway?v=1)

<br>
---

### Zeabur

[![Deploy on Zeabur](https://zeabur.com/button.svg)](https://zeabur.com/templates/RU38E3)

<br>
---

### Vercel
### Supabase Functions

Docs to be written, please help!
1. Clone the Repository:

```sh
git clone https://github.com/portkey-ai/gateway
```

2. Set up a Supabase Account and create a new project (in browser).
3. Install the Supabase CLI, login using `supabase login
` and Initialize, follow [guides](https://supabase.com/docs/guides/local-development)
4. Write Your Supabase Function in:
```sh
supabase/functions/your-function-file/index.ts
```
5. Deploy the Function (for windows add `npx` in start):
```sh
supabase functions deploy your-function-file
```
6. Now, its ready to Test, Monitor and Manage Functions (Using url from supabase dashboard):
- Similar to `https://your-project-id.supabase.co/functions/v1/your-function-file`

Refer to [Supabase's official documentation](https://supabase.com/docs) for more details.

<br>

---

### Fastly

Docs to be written, please help!
1. Create / login to your fastly account (in browser).

2. Clone the Gateway Repository:

```sh
git clone https://github.com/portkey-ai/gateway
```

3. Install Fastly’s CLI:

```sh
npm install fastly-cli
```

4. Log in to Fastly:
```sh
fastly login
```

5. Create a New Fastly Service:
```sh
fastly compute init
```

6. Write Your Edge Function Code in `src/main.rs` or `src/index.js`

7. Set up the token locally (token is located on `Account > User(left bottom most) > API Token`)
```sh
fastly profile create
```
Then follow the steps there to add token.

8. Deploy to Fastly
```sh
fastly compute publish
```
9. Now, its ready to Test, Monitor and Manage Functions (Using url from fastly dashboard):
- Similar to `your-new-project.edgecompute.app`

For more details, refer to [Fastly’s official](https://www.fastly.com/products/edge-compute).

<br>

### AWS Lambda
---

### Vercel

Docs to be written, please help!
- Docs to be written, please help!
- Make sure you've deployed and tested the same.

<br>

### Lambda@edge
---

### AWS Lambda

Docs to be written, please help!
- Docs to be written, please help!
- Make sure you've deployed and tested the same.

<br>

### Supabase Functions
---

### Lambda edge

Docs to be written, please help!
- Docs to be written, please help!
- Make sure you've deployed and tested the same.

<br>

---

## Enterprise Deployment
Make your AI app more <ins>reliable</ins> and <ins>forward compatible</ins>, while ensuring complete <ins>data security</ins> and <ins>privacy</ins>.

Expand Down
Loading