bug: insufficient handling of insufficient memory

[jlfranklin]

Jan version

0.5.5

Describe the Bug

When there is insufficient memory to run the model, lot of errors are thrown into the logs, and the returned text is complete gibberish.

Jan should stop the model and say something like, "sorry, my brain is full."

Steps to Reproduce

1. Clean install of Jan on an 8GB Macbook Air M1
2. Load Qwen Chat 7B
3. Ask it anything.

Screenshots / Logs

app.log

What is your OS?

• MacOS
• Windows
• Linux

[imtuyethan]

cortex.cpp team is working on this

[freelerobot]

Needed: proper error handling when:

1. user attempts to load a model too big to fit in available memory
2. error message, e.g.: unable to load model due to insufficient system memory. xx needed. xx available.

Feel free to reassign @vansangpfiev and move to a different sprint

[dan-menlo]

Related to #1165

[gabrielle-ong]

Linked to #1108 to recommend models based on hardware