Skip to content

Add offline SQLite backend#175

Draft
cfuselli wants to merge 8 commits intomasterfrom
sqlite-offline
Draft

Add offline SQLite backend#175
cfuselli wants to merge 8 commits intomasterfrom
sqlite-offline

Conversation

@cfuselli
Copy link
Member

This pull request introduces offline (SQLite-based) backend support to utilix, allowing core database and GridFS operations to work seamlessly when disconnected from the main MongoDB service. The main changes involve integrating the new sqlite_backend and updating the relevant classes and methods to use it when configured.

Offline backend integration:

  • Added imports for sqlite_backend and its key classes/functions (OfflineGridFS, OfflineSQLiteCollection, SQLiteConfig, _load_sqlite_config) in utilix/__init__.py, utilix/mongo_storage.py, and utilix/rundb.py to enable offline support throughout the package. [1] [2] [3]

GridFS (file storage) offline support:

  • Updated the GridFsBase class in utilix/mongo_storage.py to initialize and use OfflineGridFS when offline mode is detected, including offline-safe overrides for file listing, config existence checks, and file downloads. This ensures file storage and retrieval continue to work in offline scenarios. [1] [2]

RunDB (database) offline support:

  • Modified utilix/rundb.py to provide an _sqlite_collection function and update xent_collection to use the offline SQLite backend if enabled, allowing database queries to fallback to the local SQLite database when offline.

General improvements and error handling:

  • Added robust error handling for configuration loading and type checking for cache directories, improving reliability and user feedback when initializing offline backends.

Consistency and code reuse:

  • Ensured consistent use of the offline backend throughout the codebase, with shared logic for configuration loading and backend selection in both file and database operations. [1] [2] [3]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant