What's Changed
-
audio upload extension with gdrive credentials
-
FIX: API parameters
-
UPDATE: tmp files cleanup n code refactored as per review
-
REFACTOR: minor refactor as per review
-
REFACTOR: minor update as per review
-
UPDATE: gdrive sync logic
-
REFACTOR: code update as per gdrive and update credential client
-
REFACTOR: validation updated - as per review from CR
-
UPDATE: code has been refactore for UUID for diffrent audio upload sources
-
REFACTOR: updated code as per review
-
Update documentation and configuration to reflect the transition from 'friend-backend' to 'chronicle-backend' across various files, including setup instructions, Docker configurations, and service logs.
-
Update test script to use docker-compose-test.yml for all test-related operations
-
Added standard MIT license
-
Fix/cleanup model (#219)
-
refactor memory
-
add config
-
docstring
-
more cleanup
-
code quality
-
code quality
-
unused return
-
DOTTED GET
-
Refactor Docker and CI configurations
- Removed the creation of
memory_config.yamlfrom the CI workflow to streamline the process. - Updated Docker Compose files to mount
config.ymlfor model registry and memory settings in both services. - Added new dependencies for Google API clients in
uv.lockto support upcoming features.
- Update configuration files for model providers and Docker setup
- Changed LLM, embedding, and STT providers in
config.ymlto OpenAI and Deepgram. - Removed read-only flag from
config.ymlin Docker Compose files to allow UI configuration saving. - Updated memory configuration endpoint to accept plain text for YAML input.
- Update transcription job handling to format speaker IDs
- Changed variable name from
speaker_nametospeaker_idfor clarity. - Added logic to convert integer speaker IDs from Deepgram to string format for consistent speaker labeling.
- Remove loading of backend .env file in test environment setup
- Eliminated the code that loads the .env file from the backends/advanced directory, simplifying the environment configuration for tests.
- Enhance configuration management and setup wizard
- Updated README to reflect the new setup wizard process.
- Added functionality to load and save
config.ymlin the setup wizard, including default configurations for LLM and memory providers. - Improved user feedback during configuration updates, including success messages for configuration file updates.
- Enabled backup of existing
config.ymlbefore saving changes.
- Enhance HTTPS configuration in setup wizard
- Added functionality to check for existing SERVER_IP in the environment file and prompt the user to reuse or enter a new IP for SSL certificates.
- Improved user prompts for server IP/domain input during HTTPS setup.
- Updated default behavior to use existing IP or localhost based on user input.
- Changed RECORD_ONLY_ENROLLED_SPEAKERS setting in the .env template to false for broader access.
- Add source parameter to audio file writing in websocket controller
- Included a new
sourceparameter with the value "websocket" in the_process_batch_audio_completefunction to enhance audio file context tracking.
-
fix/broken-tests (#230)
-
refactor memory
-
add config
-
docstring
-
more cleanup
-
code quality
-
code quality
-
unused return
-
DOTTED GET
-
Refactor Docker and CI configurations
- Removed the creation of
memory_config.yamlfrom the CI workflow to streamline the process. - Updated Docker Compose files to mount
config.ymlfor model registry and memory settings in both services. - Added new dependencies for Google API clients in
uv.lockto support upcoming features.
- Update configuration files for model providers and Docker setup
- Changed LLM, embedding, and STT providers in
config.ymlto OpenAI and Deepgram. - Removed read-only flag from
config.ymlin Docker Compose files to allow UI configuration saving. - Updated memory configuration endpoint to accept plain text for YAML input.
- Update transcription job handling to format speaker IDs
- Changed variable name from
speaker_nametospeaker_idfor clarity. - Added logic to convert integer speaker IDs from Deepgram to string format for consistent speaker labeling.
- Remove loading of backend .env file in test environment setup
- Eliminated the code that loads the .env file from the backends/advanced directory, simplifying the environment configuration for tests.
- Enhance configuration management and setup wizard
- Updated README to reflect the new setup wizard process.
- Added functionality to load and save
config.ymlin the setup wizard, including default configurations for LLM and memory providers. - Improved user feedback during configuration updates, including success messages for configuration file updates.
- Enabled backup of existing
config.ymlbefore saving changes.
- Enhance HTTPS configuration in setup wizard
- Added functionality to check for existing SERVER_IP in the environment file and prompt the user to reuse or enter a new IP for SSL certificates.
- Improved user prompts for server IP/domain input during HTTPS setup.
- Updated default behavior to use existing IP or localhost based on user input.
- Changed RECORD_ONLY_ENROLLED_SPEAKERS setting in the .env template to false for broader access.
- Add source parameter to audio file writing in websocket controller
- Included a new
sourceparameter with the value "websocket" in the_process_batch_audio_completefunction to enhance audio file context tracking.
- Refactor error handling in system controller and update memory config routes
- Replaced ValueError with HTTPException for better error handling in
save_diarization_settingsandvalidate_memory_configfunctions. - Introduced a new Pydantic model,
MemoryConfigRequest, for validating memory configuration requests in the system routes. - Updated the
validate_memory_configendpoint to accept the new request model, improving input handling and validation.
-
Feat/add obsidian 3 (#233)
-
obsidian support
-
neo4j comment
-
cleanup code
-
unused line
-
unused line
-
Fix MemoryEntry object usage in chat service
-
comment
-
feat(obsidian): add obsidian memory search integration to chat
-
unit test
-
use rq
-
neo4j service
-
typefix
-
test fix
-
cleanup
-
cleanup
-
version changes
-
profile
-
remove unused imports
-
Refactor memory configuration validation endpoints
- Removed the deprecated
validate_memory_config_rawendpoint and replaced it with a new endpoint that accepts plain text for validation. - Updated the existing
validate_memory_configendpoint to clarify that it now accepts JSON input. - Adjusted the API call in the frontend to point to the new validation endpoint.
- Refactor health check model configuration loading
- Updated the health check function to load model configuration from the models registry instead of the root config.
- Improved error handling by logging warnings when model configuration loading fails.
-
Update .gitignore to exclude all files in app/ios and app/android directories (#238)
-
fix: Copy full source code in speaker-recognition Dockerfile (#243)
Adds COPY src/ src/ step after dependency installation to ensure all source files are available in the Docker image. This improves build caching while ensuring complete source code is present.
-
Enhance configuration management and add new setup scripts (#235)
-
Enhance configuration management and add new setup scripts
- Updated .gitignore to include config.yml and its template.
- Added config.yml.template for default configuration settings.
- Introduced restart.sh script for service management.
- Enhanced services.py to load config.yml and check for Obsidian/Neo4j integration.
- Updated wizard.py to prompt for Obsidian/Neo4j configuration during setup and create config.yml from template if it doesn't exist.
- Refactor transcription providers and enhance configuration management
- Updated Docker Compose files to include the new Neo4j service configuration.
- Added support for Obsidian/Neo4j integration in the setup process.
- Refactored transcription providers to utilize a registry-driven approach for Deepgram and Parakeet.
- Enhanced error handling and logging in transcription processes.
- Improved environment variable management in test scripts to prioritize command-line overrides.
- Removed deprecated Parakeet provider implementation and streamlined audio stream workers.
-
Update configuration management and enhance file structure, add test-matrix (#237)
-
Update configuration management and enhance file structure
- Refactored configuration file paths to use a dedicated
config/directory, including updates toconfig.ymland its template. - Modified service scripts to load the new configuration path for
config.yml. - Enhanced
.gitignoreto include the new configuration files and templates. - Updated documentation to reflect changes in configuration file locations and usage.
- Improved setup scripts to ensure proper creation and management of configuration files.
- Added new test configurations for various provider combinations to streamline testing processes.
- Add test requirements and clean up imports in wizard.py
- Introduced a new
test-requirements.txtfile to manage testing dependencies. - Removed redundant import of
shutilinwizard.pyto improve code clarity.
- Add ConfigManager for unified configuration management
- Introduced a new
config_manager.pymodule to handle reading and writing configurations fromconfig.ymland.envfiles, ensuring backward compatibility. - Refactored
ChronicleSetupinbackends/advanced/init.pyto utilizeConfigManagerfor loading and updating configurations, simplifying the setup process. - Removed redundant methods for loading and saving
config.ymldirectly inChronicleSetup, as these are now managed byConfigManager. - Enhanced user feedback during configuration updates, including success messages for changes made to configuration files.
- Refactor transcription provider configuration and enhance setup process
- Updated
.env.templateto clarify speech-to-text configuration and removed deprecated options for Mistral. - Modified
docker-compose.ymlto streamline environment variable management by removing unused Mistral keys. - Enhanced
ChronicleSetupininit.pyto provide clearer user feedback and updated the transcription provider selection process to rely onconfig.yml. - Improved error handling in the websocket controller to determine the transcription provider from the model registry instead of environment variables.
- Updated health check routes to reflect the new method of retrieving the transcription provider from
config.yml. - Adjusted
config.yml.templateto include comments on transcription provider options for better user guidance.
- Enhance ConfigManager with deep merge functionality
- Updated the
update_memory_configmethod to perform a deep merge of updates into the memory configuration, ensuring nested dictionaries are merged correctly. - Added a new
_deep_mergemethod to handle recursive merging of dictionaries, improving configuration management capabilities.
- Refactor run-test.sh and enhance memory extraction tests
- Removed deprecated environment variable handling for TRANSCRIPTION_PROVIDER in
run-test.sh, streamlining the configuration process. - Introduced a new
run-custom.shscript for executing Robot tests with custom configurations, improving test flexibility. - Enhanced memory extraction tests in
audio_keywords.robotandmemory_keywords.robotto include detailed assertions and result handling. - Updated
queue_keywords.robotto fail fast if a job is in a 'failed' state when expecting 'completed', improving error handling. - Refactored
test_env.pyto load environment variables with correct precedence, ensuring better configuration management.
-
unify tests to robot test, add some more clean up
-
Update health check configuration in docker-compose-test.yml (#241)
- Increased the number of retries from 5 to 10 for improved resilience during service readiness checks.
- Extended the start period from 30s to 60s to allow more time for services to initialize before health checks commence.
- Add step to create test configuration file in robot-tests.yml
- Introduced a new step in the GitHub Actions workflow to copy the test configuration file from tests/configs/deepgram-openai.yml to a new config/config.yml.
- Added logging to confirm the creation of the test config file, improving visibility during the test setup process.
-
remove cache step since not required
-
coderabbit comments
-
Refactor ConfigManager error handling for configuration file loading
- Updated the ConfigManager to raise RuntimeError exceptions when the configuration file is not found or is invalid, improving error visibility and user guidance.
- Removed fallback behavior that previously returned the current directory, ensuring users are explicitly informed about missing or invalid configuration files.
- Refactor _find_repo_root method in ConfigManager
- Updated the _find_repo_root method to locate the repository root using the file location instead of searching for config/config.yml, simplifying the logic and improving reliability.
- Removed the previous error handling that raised a RuntimeError if the configuration file was not found, as the new approach assumes config_manager.py is always at the repo root.
-
Enhance speaker recognition service integration and error handling (#245)
-
Enhance speaker recognition service integration and error handling
- Updated
docker-compose-test.ymlto enable speaker recognition in the test environment and added a newspeaker-service-testservice for testing purposes. - Refactored
run-test.shto improve the execution of Robot Framework tests from the repository root. - Enhanced error handling in
speaker_recognition_client.pyto return detailed error messages for connection issues. - Improved error logging in
speaker_jobs.pyto handle and report errors from the speaker recognition service more effectively. - Updated
Dockerfileto copy the full source code after dependencies are cached, ensuring all necessary files are included in the image.
- Remove integration tests workflow and enhance robot tests with HF_TOKEN verification
- Deleted the
integration-tests.ymlworkflow file to streamline CI processes. - Updated
robot-tests.ymlto include verification for the newHF_TOKENsecret, ensuring all required secrets are checked before running tests.
-
Fix key access in system admin tests to use string indexing for speakers data
-
Refactor Robot Framework tests and enhance error handling in memory services
- Removed the creation of the test environment file from the GitHub Actions workflow to streamline setup.
- Updated the Robot Framework tests to utilize a unified test script for improved consistency.
- Enhanced error messages in the MemoryService class to provide more context on connection failures for LLM and vector store providers.
- Added critical checks for API key presence in the OpenAIProvider class to ensure valid credentials are provided before proceeding.
- Adjusted various test setup scripts to use a centralized BACKEND_DIR variable for better maintainability and clarity.
- Refactor test container cleanup in run-robot-tests.sh
- Updated the script to dynamically construct container names from docker-compose services, improving maintainability and reducing hardcoded values.
- Enhanced the cleanup process for stuck test containers by utilizing the COMPOSE_PROJECT_NAME variable.
- Enhance run-robot-tests.sh for improved logging and cleanup
- Set absolute paths for consistent directory references to simplify navigation.
- Capture container logs, status, and resource usage for better debugging.
- Refactor cleanup process to utilize dynamic backend directory references, improving maintainability.
- Ensure proper navigation back to the tests directory after operations.
- Add speaker recognition configuration and update test script defaults
- Introduced speaker recognition settings in config.yml.template, allowing for easy enable/disable and service URL configuration.
- Updated run-robot-tests.sh to use a test-specific configuration file that disables speaker recognition for improved CI performance.
- Modified deepgram-openai.yml to disable speaker recognition during CI tests to enhance execution speed.
- Refactor speaker recognition configuration management
- Updated docker-compose-test.yml to clarify speaker recognition settings, now controlled via config.yml for improved CI performance.
- Enhanced model_registry.py to include a dedicated speaker_recognition field for better configuration handling.
- Modified speaker_recognition_client.py to load configuration from config.yml, allowing for dynamic enabling/disabling of the speaker recognition service based on the configuration.
- Add minimum worker count verification to infrastructure tests
- Introduced a new keyword to verify that the minimum number of workers are registered, enhancing the robustness of health checks.
- Updated the worker count validation test to include a wait mechanism for worker registration, improving test reliability.
- Clarified comments regarding expected worker counts to reflect the distinction between RQ and audio stream workers.
- Update configuration management and enhance model handling
- Added OBSIDIAN_ENABLED configuration to ChronicleSetup for improved feature toggling.
- Introduced speaker_recognition configuration handling in model_registry.py to streamline model loading.
- Refactored imports in deepgram.py to improve clarity and reduce redundancy.
-
Refactor configuration management in wizard and ChronicleSetup (#246)
-
Refactor configuration management in wizard and ChronicleSetup
- Updated wizard.py to read Obsidian/Neo4j configuration from config.yml, enhancing flexibility and error handling.
- Refactored ChronicleSetup to utilize ConfigManager for loading and verifying config.yml, ensuring a single source of truth.
- Improved user feedback for missing configuration files and streamlined the setup process for memory and transcription providers.
- Fix string formatting for error message in ChronicleSetup
Full Changelog: https://github.com/chronicler-ai/chronicle/compare/v0.0.1...v0.1.0