roadmap: Jan Local Engine Management can pick Version and Variant

[imtuyethan]

Problem Statement

https://cortex.so/api-reference#tag/engines/get/v1/engines/{name}
Jan needs a UI to manage Cortex.cpp engines based on the following API endpoints:

• List installed engines
• Install/Uninstall engines
• Load/Unload engines
• Get/Set default engine variant
• Update engine

Tasklist

Jan Tasks

• planning: Migrate llama.cpp Vulkan support to use Engine Management APIs #3913
• feat: update inference-cortex-extension to work with the new engine management of cortex.cpp #4025
• feat: Implement UI for Local Engine Management #4250
• feat: pre-packaged engine variants and installation directory #4337

Cortex Backend Tasks

• Implementation: epic: Improve Cortex Engine Management cortex.cpp#1416
• Enhancement: feat: Engines API endpoints should be consistent cortex.cpp#1684
• engines update: planning: cortex engines update  cortex.cpp#1638

Feature Idea

https://cortex.so/api-reference#tag/engines/get/v1/engines/{name}

Engine Management

• Version
• Update: Update engine to latest version of current default variant
• Set Default: Ability to set default variant for each engine type
• Show installed engines with:
  • Engine type (llama-cpp, onnxruntime, tensorrt-llm)
  • Variant name (e.g., mac-arm64, linux-amd64-avx)
• Only one engine can be loaded at a time
• Handle errors gracefully
• Install: Allow installing specific engine variant/version
• Uninstall: Remove engine variant