How to avoid missing a ZIM update without noticing?

[benoit74]

Currently, the scheduler automatically request tasks at the configured interval. However, these tasks might fail. When they do, the scheduler will not request the task again until the configured interval has passed again.

This behavior is meant to avoid repeating again and again the same failing task, which must indeed be avoided.

It however comes with the drawback that if there is an intermittent issue, we might wait long (especially for recipe configured for yearly update) until there is a new schedule.

Monitoring failed tasks should be done with due diligence, but I think it is too easy to forget one failing task / recipe and not update a ZIM for very long.

This cause a significant problem for me: we might have ZIMs which are not updated as requested without rely noticing it. And it is currently very hard to know at a given moment (now) which ZIM / recipe have not been updated as requested.

I see multiple solutions:

1. automatically create Github issues for every failing tasks, so that we have an overview in Github

• the drawback is that the issue itself might get outdated ... so we will need a job to automatically close these issues once the recipe runs again ...

2. create a new filter / view in the Zimfarm with every recipes which have not been updated as requested

• more work, but more precise

3. automatically retry failed recipes more often than the periodic refresh (e.g. once a day / week)

• this is properly the final goal, but it can work only if recipes failing for other than intermittent reasons are properly disabled (not been always the case so far ... but might be a good incentive to do so and avoid clogging the zimfarm with tasks failing forever)

I'm personally in favor of option 3, since it will save manual work globally.