Skip to content

Commit

Permalink
Basic auth messaging
Browse files Browse the repository at this point in the history
  • Loading branch information
hadley committed Feb 3, 2025
1 parent 629166e commit ffce6e8
Showing 1 changed file with 12 additions and 2 deletions.
14 changes: 12 additions & 2 deletions README.Rmd
Original file line number Diff line number Diff line change
Expand Up @@ -52,16 +52,26 @@ ellmer supports a wide variety of model providers:
* Snowflake Cortex: `chat_snowflake()` and `chat_cortex_analyst()`.
* VLLM: `chat_vllm()`.

## Model choice
### Provider/model choice

If you're using ellmer inside your organisation, you'll be limited to what your IT department allows, which is likely to be one provided by a big cloud provider, e.g. `chat_azure()`, `chat_bedrock()`, `chat_databricks()`, or `chat_snowflake()`. If you're using ellmer for your own exploration, you'll have a lot more freedom, so we have a few recommendations to help you get started:
If you're using ellmer inside your organisation, you'll be limited to what your IT department allows, which is likely to be one provided by a big cloud provider, e.g. `chat_azure()`, `chat_bedrock()`, `chat_databricks()`, or `chat_snowflake()`.

If you're using ellmer for your own exploration, you'll have a lot more freedom, so we have a few recommendations to help you get started:

- `chat_openai()` or `chat_claude()` are good places to start. `chat_openai()` defaults to **GPT-4o**, but you can use `model = "gpt-4o-mini"` for a cheaper, lower-quality model, or `model = "o1-mini"` for more complex reasoning. `chat_claude()` is also good; it defaults to **Claude 3.5 Sonnet**, which we have found to be particularly good at writing code.

- `chat_gemini()` is great for large prompts because it has a much larger context window than other models. It allows up to 1 million tokens, compared to Claude 3.5 Sonnet's 200k and GPT-4o's 128k. It also comes with a generous free tier (with the downside that [your data is used](https://ai.google.dev/gemini-api/terms#unpaid-services) to improve the model).

- `chat_ollama()`, which uses [Ollama](https://ollama.com), allows you to run models on your own computer. While the biggest models you can run locally aren't as good as the state of the art hosted models, they don't share your data and are effectively free.

### Authentication

Authentication works a little differently depending on whether you're using a personal or organisational account. If you're using a personal account, you will need to get an API key from the provider and save it in an environment variable. When deploying, you'll need to make sure If you deploy to another system, you'll need to ensure that this environment variable is also available in your deployment environment.

If you're using an organisational account and you're using Posit Workbench and/or Posit Connect, ellmer integrates seamlessly with their credential management. For the supported endpoints (currently `chat_azure()`, `chat_bedrock()`, `chat_databricks()`, `chat_snowflake()`), this means that auth will just work in Workbench, without you having to worry about it (if it doesn't, you'll need to talk to your server admin so that they can set it up). When deploying to Connect, you'll need to make a choice: do you want to use a service account or viewer-based credentials.

If you're not using Posit products, you'll need to work with your system administrator. ellmer has been designed to work with a variety of auth mechanisms, and we're happy to add more if needed.

## Using ellmer

You can work with ellmer in several different ways, depending on whether you are working interactively or programmatically. They all start with creating a new chat object:
Expand Down

0 comments on commit ffce6e8

Please sign in to comment.