Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add VertexAI extension #4428

Closed
wants to merge 4 commits into from
Closed

Add VertexAI extension #4428

wants to merge 4 commits into from

Conversation

Lash-L
Copy link

@Lash-L Lash-L commented Jan 9, 2025

Describe Your Changes

  • This adds a new extension for VertexAI
  • I am currently facing a glitch where the models are disabled on each close and reopen until I open up the settings page(Don't have to change anything or even go to the VertexAI settings page), but I'm hoping this is something a dev can walk me through fixing - as there is likely some check happening that determines if a model should be enabled or not.

Relates to:
#3786

Self Checklist

  • Added relevant comments, esp in complex areas
  • Updated docs (for bug fixes / features)
  • Created issues for follow-up changes or refactoring needed

@louis-jan
Copy link
Contributor

Hi @Lash-L, thanks a lot for your great work on adding VertexAI remote extension support. Since PR #4364, we've removed support for separated remote extensions and moved all work to cortex.cpp. There are many reasons for that transition, such as:

  • Any chat/completion request from extensions can be directed to a single endpoint (cortex.cpp server endpoint). E.g. adding tools from extensions would just need an OpenAI sdk setup to only one single endpoint, instead of checking every single extension engine to construct request accordingly.
  • Consistent interface across extensions, applications, and tools. Every single request would just need model and engine name. So that developers don't have to pull extension settings such as API Key or URL to construct a request.
  • Easier to build out a shared OpenAI client instance to be shared across extensions.
  • Simplify app GUI code base so that can be scaled to multiple platforms such as web and mobile.
  • Enable users to setup a new remote provider without requiring new extension setup (via GUI).

You can check out more here
#3786
and here
#4251

Even this PR will be closed but we will use it as a reference for future remote providers' support. Thank you so much for the contribution

@louis-jan louis-jan closed this Jan 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants