Skip to content

DiUS/aisha

BranchesTags
 
 

Repository files navigation

For Dius Devs

Current state 23/8/2024: This repo is deployed (deployed via CDK from local - see instructions below), you can create an account and sign in at https://d11scinx7hb6tv.cloudfront.net/

However chatting does not seem to work. Receiving An error occurred while responding.

Logs from chatting are available in Cloud watch under the AishaWebsocket log group

Prerequisites

  • AWS Access - Dius GenAI account (012654631482) - ask on sys-admin slack channel

Deploying

  • Install npm packages
cd cdk
npm ci
  • Install AWS CDK if you haven't already got it
npm i -g aws-cdk
  • (You can probably skip this step as this should already be done for Aisha). Before deploying the CDK, you will need to work with Bootstrap once for the region you are deploying to. In this example, we will deploy to the ap-southeast-2 region. Please replace your account id into <account id>.
cdk bootstrap aws://<account id>/ap-southeast-2

Make sure you have a profile setup in ~/.aws/credentials. You can get access keys from the AWS start page https://d-97677dca1f.awsapps.com/start/#/ to paste into there. It looks something like this:

[012654631482_AWSAdministratorAccess]
aws_access_key_id=xxx
aws_secret_access_key=xxx
aws_session_token=xxx
region=ap-southeast-2

Deploy using your profile setup above.

cdk deploy --require-approval never --all --profile 012654631482_AWSAdministratorAccess
  • You will get output similar to the following. The URL of the web app will be output in AishaBedrockChatStack.FrontendURL, so please access it from your browser.
 ✅  AishaBedrockChatStack

✨  Deployment time: 78.57s

Outputs:
AishaBedrockChatStack.AuthUserPoolClientIdXXXXX = xxxxxxx
AishaBedrockChatStack.AuthUserPoolIdXXXXXX = ap-northeast-1_XXXX
AishaBedrockChatStack.BackendApiBackendApiUrlXXXXX = https://xxxxx.execute-api.ap-northeast-1.amazonaws.com
AishaBedrockChatStack.FrontendURL = https://xxxxx.cloudfront.net

AWS ORIGINAL README BELOW

Bedrock Claude Chat

Warning

If you are using old version (e.g. v0.4.x) and wish to use the latest version, refer migration guide. Without any care, ALL DATA IN Aurora cluster WILL BE DESTROYED, and NO LONGER USERS CANNOT USE EXISTING BOTS WITH KNOWLEDGE AND CREATE NEW BOTS.

This repository is a sample chatbot using the Anthropic company's LLM Claude, one of the foundational models provided by Amazon Bedrock for generative AI.

Watch Overview and Installation on YouTube

Overview

Basic Conversation

Not only text but also images are available with Anthropic's Claude 3. Currently we support Haiku, Sonnet and Opus.

Bot Personalization

Add your own instruction and give external knowledge as URL or files (a.k.a RAG). The bot can be shared among application users. The customized bot also can be published as stand-alone API (See the detail).

Important

For governance reasons, only allowed users are able to create customized bots. To allow the creation of customized bots, the user must be a member of group called CreatingBotAllowed, which can be set up via the management console > Amazon Cognito User pools or aws cli. Note that the user pool id can be referred by accessing CloudFormation > AishaBedrockChatStack > Outputs > AuthUserPoolIdxxxx.

Administrator dashboard

Administrator dashboard

Analyze usage for each user / bot on administrator dashboard. detail

LLM-powered Agent

LLM-powered Agent

By using the Agent functionality, your chatbot can automatically handle more complex tasks. For example, to answer a user's question, the Agent can retrieve necessary information from external tools or break down the task into multiple steps for processing.

📚 Supported Languages

  • English 💬
  • 日本語 💬 (ドキュメントはこちら)
  • 한국어 💬
  • 中文 💬
  • Français 💬
  • Deutsch 💬
  • Español 💬
  • Italian 💬

🚀 Super-easy Deployment

  • In the ap-southeast-2 region, open Bedrock Model access > Manage model access > Check Anthropic / Claude 3 Haiku, Anthropic / Claude 3 Sonnet, Anthropic / Claude 3.5 Sonnet and Cohere / Embed Multilingual then Save changes.
Screenshot

  • Open CloudShell at the region where you want to deploy
  • Run deployment via following commands. If you want to specify the version to deploy or need to apply security policies, please specify the appropriate parameters from Optional Parameters.
git clone https://github.com/aws-samples/bedrock-claude-chat.git
cd bedrock-claude-chat
chmod +x bin.sh
./bin.sh
  • You will be asked if a new user or using v1. If you are not a continuing user from v0, please enter y.

Optional Parameters

You can specify the following parameters during deployment to enhance security and customization:

  • --disable-self-register: Disable self-registration (default: enabled). If this flag is set, you will need to create all users on cognito and it will not allow users to self register their accounts.
  • --ipv4-ranges: Comma-separated list of allowed IPv4 ranges. (default: allow all ipv4 addresses)
  • --ipv6-ranges: Comma-separated list of allowed IPv6 ranges. (default: allow all ipv6 addresses)
  • --disable-ipv6: Disable connections over IPv6. (default: enabled)
  • --allowed-signup-email-domains: Comma-separated list of allowed email domains for sign-up. (default: no domain restriction)
  • --bedrock-region: Define the region where bedrock is available. (default: ap-southeast-2)
  • --version: The version of Bedrock Claude Chat to deploy. (default: latest version in development)

Example command with parameters:

./bin.sh --disable-self-register --ipv4-ranges "192.0.2.0/25,192.0.2.128/25" --ipv6-ranges "2001:db8:1:2::/64,2001:db8:1:3::/64" --allowed-signup-email-domains "example.com,anotherexample.com" --bedrock-region "us-west-2" --version "v1.2.6"
  • After about 35 minutes, you will get the following output, which you can access from your browser
Frontend URL: https://xxxxxxxxx.cloudfront.net

The sign-up screen will appear as shown above, where you can register your email and log in.

Important

Without setting the optional parameter, this deployment method allows anyone who knows the URL to sign up. For production use, it is strongly recommended to add IP address restrictions and disable self-signup to mitigate security risks (you can define allowed-signup-email-domains to restrict users so that only email addresses from your company’s domain can sign up). Use both ipv4-ranges and ipv6-ranges for IP address restrictions, and disable self-signup by using disable-self-register when executing ./bin.

Tip

If the Frontend URL does not appear or Bedrock Claude Chat does not work properly, it may be a problem with the latest version. In this case, please add --version "v1.2.6" to the parameters and try deployment again.

Architecture

It's an architecture built on AWS managed services, eliminating the need for infrastructure management. Utilizing Amazon Bedrock, there's no need to communicate with APIs outside of AWS. This enables deploying scalable, reliable, and secure applications.

Deploy using CDK

TODOs

  • Default guardrail does not contain the context grounding as this is not available in the cdk yet (Eventhough it exists for Cloud Formation templates). Need to add this manually and create a new version.

Super-easy Deployment uses AWS CodeBuild to perform deployment by CDK internally. This section describes the procedure for deploying directly with CDK.

  • Please have UNIX, Docker and a Node.js runtime environment. If not, you can also use Cloud9

Important

If there is insufficient storage space in the local environment during deployment, CDK bootstrapping may result in an error. If you are running in Cloud9 etc., we recommend expanding the volume size of the instance before deploying.

  • Clone this repository
git clone https://github.com/aws-samples/bedrock-claude-chat
  • Install npm packages
cd bedrock-claude-chat
cd cdk
npm ci

npm i -g aws-cdk
  • Before deploying the CDK, you will need to work with Bootstrap once for the region you are deploying to. In this example, we will deploy to the ap-southeast-2 region. Please replace your account id into <account id>.
cdk bootstrap aws://<account id>/ap-southeast-2

  • If necessary, edit the following entries in cdk.json if necessary.

    • bedrockRegion: Region where Bedrock is available. NOTE: Bedrock does NOT support all regions for now.
    • allowedIpV4AddressRanges, allowedIpV6AddressRanges: Allowed IP Address range.

  • Deploy this sample project

cdk deploy --require-approval never --all
  • You will get output similar to the following. The URL of the web app will be output in AishaBedrockChatStack.FrontendURL, so please access it from your browser.
 ✅  AishaBedrockChatStack

✨  Deployment time: 78.57s

Outputs:
AishaBedrockChatStack.AuthUserPoolClientIdXXXXX = xxxxxxx
AishaBedrockChatStack.AuthUserPoolIdXXXXXX = ap-northeast-1_XXXX
AishaBedrockChatStack.BackendApiBackendApiUrlXXXXX = https://xxxxx.execute-api.ap-northeast-1.amazonaws.com
AishaBedrockChatStack.FrontendURL = https://xxxxx.cloudfront.net

Others

Configure Mistral models support

Update enableMistral to true in cdk.json, and run cdk deploy.

...
  "enableMistral": true,

Important

This project focus on Anthropic Claude models, the Mistral models are limited supported. For example, prompt examples are based on Claude models. This is a Mistral-only option, once you toggled to enable Mistral models, you can only use Mistral models for all the chat features, NOT both Claude and Mistral models.

Configure default text generation

Users can adjust the text generation parameters from the custom bot creation screen. If the bot is not used, the default parameters set in config.py will be used.

DEFAULT_GENERATION_CONFIG = {
    "max_tokens": 2000,
    "top_k": 250,
    "top_p": 0.999,
    "temperature": 0.6,
    "stop_sequences": ["Human: ", "Assistant: "],
}

Remove resources

If using cli and CDK, please cdk destroy. If not, access CloudFormation and then delete AishaBedrockChatStack and FrontendWafStack manually. Please note that FrontendWafStack is in ap-southeast-2 region.

Stopping Vector DB for RAG

By setting cdk.json in the following CRON format, you can stop and restart Aurora Serverless resources created by the VectorStore construct. Applying this setting can reduce operating costs. By default, Aurora Serverless is always running. Note that it will be executed in UTC time.

...
"rdbSchedules": {
  "stop": {
    "minute": "50",
    "hour": "10",
    "day": "*",
    "month": "*",
    "year": "*"
  },
  "start": {
    "minute": "40",
    "hour": "2",
    "day": "*",
    "month": "*",
    "year": "*"
  }
}

Language Settings

This asset automatically detects the language using i18next-browser-languageDetector. You can switch languages from the application menu. Alternatively, you can use Query String to set the language as shown below.

https://example.com?lng=ja

Disable self sign up

This sample has self sign up enabled by default. To disable self sign up, open cdk.json and switch selfSignUpEnabled as false. If you configure external identity provider, the value will be ignored and automatically disabled.

Restrict Domains for Sign-Up Email Addresses

By default, this sample does not restrict the domains for sign-up email addresses. To allow sign-ups only from specific domains, open cdk.json and specify the domains as a list in allowedSignUpEmailDomains.

"allowedSignUpEmailDomains": ["example.com"],

Customize Number of NAT Gateway

By default, this sample deploys 2 NAT gateways, but you can change the number of NAT gateways if you don't need 2 NAT gateways to reduce costs. Open cdk.json and change this parameter 'number of NAT gateways'.

"natgatewayCount": 2

External Identity Provider

This sample supports external identity provider. Currently we support Google and custom OIDC provider.

Add new users to groups automatically

This sample has the following groups to give permissions to users:

If you want newly created users to automatically join groups, you can specify them in cdk.json.

"autoJoinUserGroups": ["CreatingBotAllowed"],

By default, newly created users will be joined to the CreatingBotAllowed group.

Local Development

See LOCAL DEVELOPMENT.

Contribution

Thank you for considering contributing to this repository! We welcome bug fixes, language translations (i18n), feature enhancements, agent tools, and other improvements.

For feature enhancements and other improvements, before creating a Pull Request, we would greatly appreciate it if you could create a Feature Request Issue to discuss the implementation approach and details. For bug fixes and language translations (i18n), proceed with creating a Pull Request directly.

Please also take a look at the following guidelines before contributing:

RAG (Retrieval Augmented Generation)

See here.

Contacts

🏆 Significant Contributors

Contributors

bedrock claude chat contributors

License

This library is licensed under the MIT-0 License. See the LICENSE file.

About

DiUS Aisha

Resources

Readme

License

MIT-0 license

Code of conduct

Code of conduct
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published

Languages

  • TypeScript 54.7%
  • Python 43.3%
  • JavaScript 1.2%
  • Shell 0.4%
  • Dockerfile 0.3%
  • HTML 0.1%