Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the handling of AI clients and API keys #9

Open
jtpio opened this issue Oct 25, 2024 · 2 comments
Open

Improve the handling of AI clients and API keys #9

jtpio opened this issue Oct 25, 2024 · 2 comments

Comments

@jtpio
Copy link
Member

jtpio commented Oct 25, 2024

We should look into improving how the clients are instantiated, and the API keys provided:

  • do we need to instantiate both a MistralAI and ChatMistralAI clients?
  • maybe there should be one plugin to provide the AI clients, that other plugins would consume
  • offer a way to use in set the API key without going through the settings system, since the settings are stored in the browser in JupyterLite

Originally posted by @jtpio in #6 (comment)

@brichet
Copy link
Collaborator

brichet commented Oct 25, 2024

  • do we need to instantiate both a MistralAI and ChatMistralAI clients?

They seem to be intended for different uses, respectively completion and chat.

From the API of ChatMistralAI and MistralAI:

  • Class ChatMistralAI
    Mistral AI chat model integration.

  • Class MistralAI
    MistralAI completions LLM.

@pmkrawczyk
Copy link

I am serving Jupyterlite to multiple users who don't have API key. Would there be a way to add config for optional backend API endpoints that would take care of actual communication with LLM providers using a single API key?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants