Archive old data

[rstrange-scottlogic]

Currently the database and data textures volume will continue to grow as time goes on.

We would like to check how quickly that is likely to be, and consider a sensible 'archive' solution to stop the linux box just filling up over time.

By archive we mean delete.

This includes the database tables and the data textures volume. We will clear both out in tandem so that they are kept in sync.

The database clears the forecast, in-situ, and data textures tables.

• The database size and data texture storage is used to work out what a relatively sensible limit should be, i.e. 3 months, 6 months etc
• A new daily script is created to archive data older than a configurable setting (default figured out above). This includes
  • Creating the script
  • Dockerising it to run once per day
  • Deploying it to the linux machine
  • Checking logs are correct (as we are unlikely to actually delete anything yet).
• Integration tests are written for this.

EDIT : After analysis we have decided not to bother implementing any archiving at this time, as it does not seem like it's going to be a problem for quite some time : https://github.com/ECMWFCode4Earth/vAirify/wiki/Data-Archive-Considerations
We have still updated the purge script to take account of the extra database table and data textures folders, so if needs be this could be deployed and scheduled at a later date.