Add structured rotating file logging infrastructure #43
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This pull request introduces centralized, structured, rotating file-based logging for both the Flask web application and the Celery worker. It ensures that logs are persisted across restarts, consistently formatted, and protected from unbounded growth, significantly improving observability and debuggability.
Changes
1. Introduce Centralized Logging Configuration
Issue:
Logging was configured inconsistently across components, with no shared setup between the web application and the Celery worker. Log files could grow indefinitely and were not guaranteed to persist outside containers.
Fix:
Added a new
logging_config.pymodule that provides a reusablesetup_logging()function. This function:RotatingFileHandler(10 MB max size, 5 backups)timestamp level logger module message)2. Enable File-Based Logging for the Web Application
Issue:
Flask and Gunicorn logs were not consistently written to a persistent file, making debugging difficult after container restarts.
Fix:
Updated
app.pyto initialize logging viasetup_logging(app)after loading configuration.This ensures all application-level logs are written to the rotating log file.
3. Enable File-Based Logging for Celery Workers
Issue:
Asynchronous task execution and failures were not reliably logged to disk, limiting visibility into background processing errors.
Fix:
Updated
celeryWorker.pyto apply the same logging configuration during worker startup.Celery task logs now use the shared rotating log file and consistent formatting.
4. Persist Logs via Docker Volumes
Issue:
Logs generated inside containers were not persisted on the host filesystem.
Fix:
Updated
docker-compose.ymlto mount a shared./logsdirectory into both the web and worker containers (/app/logs), ensuring logs survive container restarts.Impact
Observability
All application and background task logs are now persisted, structured, and centrally accessible, making debugging and monitoring significantly easier.
Reliability
Log rotation prevents uncontrolled log growth, reducing the risk of disk exhaustion in long-running deployments.
Maintainability
A single, shared logging configuration ensures consistent behavior across components and simplifies future logging-related changes.