A repo for a document chatbot that uses the Falcon model
The key advantage with this implementation is that no data ever leaves your AWS account. The model is hosted in a SageMaker endpoint in your account and all inference requests will be sent to that endpoint.
- Install the required packages for this application with
pip install -r requirements.txt. To avoid conflicts with existing python dependencies, it is best to do so in a virtual environment:
$python3 -m venv .venv
$source .venv/bin/activate
$pip3 install -r requirements.txt - You will need a SageMaker endpoint with the Falcon model deployed in your account. If you don't have, one you can use this notebook to deploy it in your account.
- (optional) Amend the chatbot.py file so that it points to your endpoint (variable
endpoint_name) - Run the app with
streamlit run chatbot.py - Upload a text file (e.g. Amazon's Q1 results)
- Start chatting 🤗
If you want to run Streamlit apps directly in SM Studio, you can do so with command streamlit run chatbot.py --server.port 6006. Once the app has started you can go to https://<YOUR_STUDIO_ID>.studio.<YOUR_REGION>.sagemaker.aws/jupyter/default/proxy/6006/ to launch the app.
Another way to use this application is creating and running a docker image using the Dockerfile.
You can create the docker image with this command
docker build --tag chatbot_image .
Then you can run the application with the following command:
docker run -v ~/.aws:/root/.aws -p 8501:8501 chatbot_image
The flag -v allows you to bind mount your AWS credentials file from your host into the Docker container. This command tells Docker to mount the ~/.aws directory from your host machine into the /root/.aws directory in the Docker container. AWS SDKs will look for the credentials file in this location.
The format is -p <host_port>:<container_port>. If your application inside Docker listens on port 8501, for instance, and you want to access it via port 8501 on your host machine, you'd use -p 8501:8501.
Access Your Application:
If your Docker container is running a web application, you should now be able to access it in a web browser at localhost:8501 (or whatever host port you used).