delete expired results in batches to avoid running OOM

[saz]

fixes #430

[Copilot]

Pull Request Overview

This PR modifies the deletion of expired results to process records in batches, thereby reducing the likelihood of running out of memory. It also updates a benchmark test threshold to reflect the changes in deletion performance.

• Adjust benchmark test assertion to allow higher timing.
• Implement batched deletion in the manager to mitigate OOM issues.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
t/integration/benchmark_models.py	Updated test assertion threshold from 1 to 5
django_celery_results/managers.py	Introduced batched deletion with a configurable batch_size

Comments suppressed due to low confidence (2)

t/integration/benchmark_models.py:70

• [nitpick] Consider adding a comment to explain why the threshold was increased from 1 to 5, clarifying the trade-off in benchmark performance due to the batched deletion implementation.

        assert self.benchmark.stats.stats.max < 5

django_celery_results/managers.py:87

• [nitpick] Review the default batch_size value of 100000 to ensure that it is appropriate for production workloads; consider making this parameter externally configurable if necessary.

    def delete_expired(self, expires, batch_size=100000):

[auvipy]

can we also add some unit test for this?

[saz]

@auvipy is there a chance to get a new release with this fix included?

[auvipy]

yes