feat: improved caching memory usage

[TeKrop]

Summary by Sourcery

Improve caching memory usage by introducing JSON compression and decompression for API cache values. Optimize search data caching by switching from Redis 'set' to 'hset'. Update Dockerfile to use a different OpenResty image and install additional dependencies. Adjust test cases to reflect changes in cache handling.

New Features:

• Introduce JSON compression and decompression for API cache values to improve memory usage.

Enhancements:

• Switch from using Redis 'set' to 'hset' for search data caching to optimize storage and retrieval.
• Update Dockerfile to use 'openresty/openresty:alpine-fat' and install additional dependencies like 'git' and 'lua-zlib'.

Build:

• Update Dockerfile to use 'openresty/openresty:alpine-fat' and install additional dependencies like 'git' and 'lua-zlib'.

Tests:

• Modify test cases to accommodate changes in cache value handling, specifically for JSON objects.

[sourcery-ai]

Reviewer's Guide by Sourcery

This PR improves memory usage in caching by implementing compression for JSON values and optimizing Redis data structures. The changes include switching from individual keys to hash maps for search data storage and adding zlib compression for API cache values.

Updated class diagram for CacheManager

classDiagram
    class CacheManager {
        +get_api_cache(cache_key: str) dict | list | None
        +update_api_cache(cache_key: str, value: dict | list, expire: int) None
        +get_search_data_cache(data_type: SearchDataType, cache_key: str) str | None
        +update_search_data_cache(search_data: dict) None
        __compress_json_value(value: dict | list) str
        __decompress_json_value(api_cache: str) dict | list
    }
    note for CacheManager "Improved memory usage by compressing JSON values and using hash maps for Redis storage."

Loading

File-Level Changes

Change	Details	Files
Implemented JSON compression for API cache values	Modified return type of get_api_cache to return parsed dict/list instead of raw string Added JSON compression/decompression handling for cache values Updated cache retrieval to handle compressed data	app/cache_manager.py
Optimized search data storage structure in Redis	Switched from individual Redis keys to Redis hash maps for search data Removed explicit timeout setting for search data cache Updated get_search_data_cache to use hget instead of get Modified update_search_data_cache to use hset instead of set	app/cache_manager.py app/config.py
Added zlib compression support in OpenResty	Switched to alpine-fat base image for additional dependencies Added zlib development packages Installed lua-zlib package via luarocks	build/nginx/Dockerfile

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time. You can also use
  this command to specify where the summary should be inserted.

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[sourcery-ai]

Hey @TeKrop - I've reviewed your changes - here's some feedback:

Overall Comments:

• Consider adding a TTL to the Redis hashes for search data to prevent unbounded growth, similar to the previous implementation's search_data_timeout

Here's what I looked at during the review

• 🟡 General issues: 2 issues found
• 🟢 Security: all looks good
• 🟡 Testing: 1 issue found
• 🟢 Complexity: all looks good
• 🟢 Documentation: all looks good

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}