fix: 4171 - Model loading gets stuck on stop

[louis-jan]

Describe Your Changes

This PR aims to update the cortex.cpp version to address a couple of issues, including rounded float values and stopping the model after loading.

Also, to cancel the pending model start quest, as there’s a known server-side issue where the model couldn’t be stopped mid-load.

I’ve noticed that enabling cont_batching and increasing the number of parallel operations can slightly improve the user experience, especially when the model is generating requests in the background. (There’s an issue where the server can’t stop a running inference). So that better to add these settings into cortex for advanced usage for LLM enthusiast, there will be options and enhanced on the settings UX, this is the first step of mapping engine parameters into Jan Settings.

Fixes Issues

• bug: Model starting takes extremely long  #4171
• bug: After stopping inference, stop icon button gets stuck #4176
• bug: Model parameters display unrounded decimal values #4087

Changes made

The code changes include:

1. Version Update:

   • The version in version.txt is updated from 1.0.4-rc4 to 1.0.4-rc5.

2. Settings Update:

   • The default_settings.json has been modified to include new settings (cont_batching, caching_enabled, cache_type, use_mmap) and updated descriptions and default values for various settings. The placeholder and value fields are also adjusted.

3. Refactoring Constants:

   • In rollup.config.ts and global.d.ts, DEFAULT_SETTINGS is renamed to SETTINGS.

4. New Enum for Settings:

   • An enum Settings is created in index.ts to manage different available settings.

5. Settings Handling:

   • New properties added to the class JanInferenceCortexExtension for default settings values.
   • Adding a method onSettingUpdate to handle updates to settings.
   • Register settings on load and use these settings during model operations in loadModel.

6. Abort Controller:

   • Introduces abort controllers for managing requests associated with models and handles cleanup after operations.

7. Enhancements in UI:

   • In ErrorMessage, LoadModelError, and TextMessage components, CSS classes are adjusted for capitalization and overflow behavior to improve UI consistency and presentation.

These changes encompass enhancements in functionality (settings management, abort controllers) and user interface adjustments.

[github-actions]

Barecheck - Code coverage report

Total: 69.32%

Your code coverage diff: 0.00% ▴

Uncovered files and lines

File	Lines
web/containers/ErrorMessage/index.tsx	38, 40-41, 44
web/screens/Thread/ThreadCenterPanel/LoadModelError/index.tsx	17-21, 23-24, 35-37, 40, 57
web/screens/Thread/ThreadCenterPanel/TextMessage/index.tsx	28-31, 33-34, 36-37, 40-41