Skip to content

Commit

Permalink
Let people init OpenAI with client and tokenizer
Browse files Browse the repository at this point in the history
  • Loading branch information
RobinPicard authored and rlouf committed Mar 7, 2024
1 parent 4775309 commit 9fd0f46
Show file tree
Hide file tree
Showing 7 changed files with 184 additions and 391 deletions.
27 changes: 22 additions & 5 deletions docs/reference/models/openai.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# Generate text with the OpenAI API
# Generate text with the OpenAI and compatible APIs

!!! Installation

Expand All @@ -16,16 +16,33 @@ print(type(model))
# OpenAI
```

Outlines also supports Azure OpenAI models:

It is possible to pass a system message to the model when initializing it:

```python
```
from outlines import models
model = models.openai("gpt-4", system_prompt="You are a useful assistant")
model = models.azure_openai(
api_version="2023-07-01-preview",
azure_endpoint="https://example-endpoint.openai.azure.com",
)
```

More generally, you can use any API client compatible with the OpenAI interface by passing an instance of the client, a configuration, and optionally the corresponding tokenizer (if you want to be able to use `outlines.generate.choice`):

```
from openai import AsyncOpenAI
import tiktoken
from outlines.models.openai import OpenAI, OpenAIConfig
config = OpenAIConfig(model="gpt-4")
client = AsyncOpenAI()
tokenizer = tiktoken.encoding_for_model("gpt-4")
model = OpenAI(client, config, tokenizer)
```

This message will be used for every subsequent use of the model:

## Monitoring API use

Expand Down
2 changes: 0 additions & 2 deletions outlines/models/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,10 @@
"""
from typing import Union

from .azure import AzureOpenAI, azure_openai
from .exllamav2 import ExLlamaV2Model, exl2
from .llamacpp import LlamaCpp, llamacpp
from .mamba import Mamba, mamba
from .openai import OpenAI, openai
from .openai_compatible import OpenAICompatibleAPI, openai_compatible_api
from .transformers import Transformers, transformers

LogitsGenerator = Union[Transformers, LlamaCpp, ExLlamaV2Model, Mamba]
119 changes: 0 additions & 119 deletions outlines/models/azure.py

This file was deleted.

Loading

0 comments on commit 9fd0f46

Please sign in to comment.