A TypeScript / JavaScript library for estimating the number of tokens an OpenAI chat completion request will use.
Estimating token usage for chat completions isn't quite as easy as it sounds.
For regular chat messages, you need to consider how the messages are formatted by OpenAI when they're provided to the model, as they don't simply dump the JSON messages they receive via the API into the model.
For function calling, things are even more complex, as the OpenAPI-style function definitions get rewritten into TypeScript type definitions.
This library handles both of those cases, as well as a minor adjustment needed for handling the results of function calling. tiktoken is used to do the tokenization.
import { promptTokensEstimate } from "openai-chat-tokens"
const estimate = promptTokensEstimate({
messages: [
{ role: "system", content: "These aren't the droids you're looking for" },
{ role: "user", content: "You can go about your business. Move along." },
],
functions: [
{
name: "activate_hyperdrive",
description: "Activate the hyperdrive",
parameters: {
type: "object",
properties: {
destination: { type: "string" },
},
},
},
],
})
Built in TypeScript, tested with Vitest.
pnpm install
pnpm test
When adding new test cases or debugging token count mismatches, it can be helpful to validate the estimated tokens in the tests against the live OpenAI API. To do this:
- Set up the
OPENAI_API_KEY
environment variable with a live API key - Add
validate: true
to one of the test examples, or setvalidateAll
totrue
intoken-counts.test.ts
, then run the tests
- "Counting tokens for chat completions API calls" in OpenAI's "How to count tokens with tiktoken" notebook
- A post about counting function call tokens on the OpenAI forum.