planning: Cortex API supports basic /assistants (Jan status quo equivalent)

[dan-menlo]

Goal

• Cortex should support the Assistants API and implement underlying data structures
  • cortex.db has Assistants table
  • assistant.yaml
• Linked: roadmap: Migrate Jan's legacy Assistants to Cortex /assistants API jan#3909

[dan-menlo]

@namchuai @louis-jan For migration of Assistant, I think we can scope it down into two things:

Legacy "Assistants"

• For legacy assistants, I think we should follow the strategy in /threads
• v1 assistants should be handled by v1 thread.json assistants, and not exposed via API

Legacy Retrieval Tool

I would like to understand the possible approaches for us to handle this:

• Option 1: Maintain as a legacy Retrieval tool (eg have a legacy API handler in Cortex)
• Option 2: Migrate files, vector DB to Cortex?

I think it is easier for us to scope this to having legacy handlers and focus specifically on migration. With system stability, it is the possible for us to use v2 API handlers to build proper APIs, which will handle new Threads, Assistants and Tools

[dan-menlo]

@louis-jan @namchuai While this ticket is for a v1 of Assistants, I think we should project forward to the likely roadmap for Assistants:

• Are we able to implement Assistants using an existing framework?
  • e.g. Assistants have type, e.g. Langchain or LlamaIndex
  • e.g. Assistants implement Tools differently based on type, e.g. https://python.langchain.com/v0.1/docs/modules/agents/

We should evaluate this; if it is too messy or difficult to align with an OpenAI-compatible API, I am open with also just going with our own (e.g. Runs, etc).

[TC117]

Hi @namchuai, can you take a look at assistants endpoint

I tried with threads but that didn’t work either

I did check and Jan have an assistants folder and I think Jan query assistant endpoint for this folder not from thread

curl -X 'GET' \
  'http://127.0.0.1:1337/v1/assistants' \
  -H 'accept: application/json'

Repsponse:
{
  "object": "list",
  "data": [
    {
      "avatar": "",
      "id": "jan",
      "object": "assistant",
      "created_at": 1733377512678,
      "name": "Jan",
      "description": "A default assistant that can use all downloaded models",
      "model": "*",
      "instructions": "",
      "tools": [
        {
          "type": "retrieval",
          "enabled": false,
          "useTimeWeightedRetriever": false,
          "settings": {
            "top_k": 2,
            "chunk_size": 1024,
            "chunk_overlap": 64,
            "retrieval_template": "Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.\n----------------\nCONTEXT: {CONTEXT}\n----------------\nQUESTION: {QUESTION}\n----------------\nHelpful Answer:"
          }
        }
      ],
      "file_ids": []
    }
  ]
}

[TC117]

Create new assistant

[TC117]

But cant get assistant with ID