Containerise the backend

[eternaltyro]

Containerise the backend infrastructure

Currently, the backend is run on AWS EC2 VMs. The aim of this epic is to completely containerise the backend so that it can be run on AWS ECS or equivalent service.

Aspirational Service Quality Objectives

• Open Standard / vendor neutral

  • Building containers to OCI specs - using buildah / podman
  • Standard libraries and packages

• End-to-end environment security

  • Container security scanning using Quay / Synk
  • Sane practices for injecting secrets
  • Secure container build steps
  • SBOM and license compliance
  • Trufflehog scans to check for leaked credentials

• End-to-end observability

  • Service availability monitoring
  • APM
  • Stack traces

• Seemless Scalability

  • Scale to meet demand. fast.

• Easily maintained

  • Updates, deployments and rollbacks should be easy and fast.

Tasks involved

• Improve docker defaults #5313
• TM not available after starting with docker-compose #5008
• Cannot build Dockerfile.backend with Alpine 3.14 #4918
• [BUG] Cleanup Dockerfiles #5706
• [BUG] Error while login on instance setup using docker-compose #5483
• Cannot use Docker due to tzlocal.utils.ZoneInfoNotFoundError #4906
• Documentation link for postgres docker image is broken #4223
• PostgreSQL data are not persistent #3647
• Log to stderr instead of log directory #3636
• Avoid bind-mounting code on non-dev setups #3633

Pull Requests to review

• Incorporate changes from - Update dependencies (backend) #5642
• Incorporate changes from - Update backend dependency #5463
• Make docker defaults more useful for local testing setup and update documentation #5087
• obsolete Deployment/container tasking manager #4857

[spwoodcock]

I would be happy to contribute a Helm chart if a Kubernetes based deployment is desirable.

But it may be best to do one thing at a time. VM --> Containers --> Orchestration System.

[eternaltyro]

Thank you, @spwoodcock for the kind offer. I agree that we should move one step at a time. Currently, I am focusing on Containers. If you think there is any other quality objective that I missed above, please feel free to add them here and we can discuss where in the roadmap it should go.