Jan: Data Migrations

[dan-homebrew]

Goals

• Jan's Data Structures will (inevitably) change
• We need to have a structured way of handling Version Upgrades, to avoid user's files getting corrupted

Scope

• Legacy file-based data structures (e.g. assistant.json)
• Future sqlite-based data persistence
• How do we deal with cortex.cpp? (should be handled by Cortex's own Data Migration strategy)
• Logs, etc

[freelerobot]

@dan-homebrew is it 2 separate migrations?

1. /jan -> /.cortexcpp: Migrating models into cortex.cpp data structure (file based)

2. /jan -> /.cortex-platform: Migrating stateful data (threads, assistant) into sql databased (db based)

3. /jan -> /jan: themes, UI stuff, stay the same?

[marknguyen1302]

hi @dan-homebrew , about migrating the legacy model. I think we have 2 options here: migrating the legacy model to the new model.json or supporting both the legacy and new model in the cortex extension.
From my point of view, here are the pros and cons of both approaches:

• migrating the legacy model to the new model.json:
  • props:
    • less complexity in our codebase
    • easier to debug when users have issues since we only have 1 model structure
  • cons:
    • we have to deprecate the Nitro extension
    • a risk when migrating the old model.json of the user to the new one ( have to be careful about transactions: eg, if the process/machine crash in the middle of the process, we have to continue without data loss, redundant data)
• supporting both the legacy and new model in the cortex extension:
  • props:
    • reduce the risk since we don't modify the user data
    • we can keep the Nitro extension and deprecate it later if needed
    • user can install the app without any additional process (migration process)
  • cons:
    • more complexity since we now have 2 structures of the model.json ( they're not too different as far as I know)
    • harder to debug the issue since we need to know what the model.json users using

cc @louis-jan , please cmiiw

[freelerobot]

migrating the legacy model to the new model.json
@marknguyen1302 do you mean YAML? The new format is a flattened YAML

[dan-homebrew]

@marknguyen1302 @louis-jan Can I check my understanding:

"Legacy" scope is essentially:

• ~/jan/models folder structure
• model.json config
• GGUF models are the same, so either Cortex's llama.cpp or Nitro's llama.cpp can run it

My high-level understanding is that we have gone the route of Option 2, i.e. inference-cortex-extension supports both the legacy folder structure, and model.json:

• @marknguyen1302 Can you link me to the actual code you implement this? (is there a PR?)
• I recall discussion on Cortex Platform having a "Files extension", in addition to a "DB extension" - is this correct?
• Do we make any changes to the legacy folder structure, or do we leave things as-is?

[marknguyen1302]

ah sorry @0xSage , i meant YAML

[marknguyen1302]

hi @dan-homebrew , i think your understanding is correct.

• "Can you link me to the actual code you implement this?". Louis already implemented it. you can checkout branch feat/cortex-extension-engine, then look at the getModelsMetadata in extensions/model-extension/src/index.ts
• yeah it's correct, we gonna support the file system as a database in the Cortex Platform
• I think we will leave the legacy folder structure

[dan-homebrew]

I am refactoring this discussion to focus on Jan's Data Migrations strategy:

• Jan's Data Structures will (inevitably) change
• We need to have a structured way of handling Version Upgrades, to avoid user's files getting corrupted