-
Notifications
You must be signed in to change notification settings - Fork 99
Description
Is your feature request related to a problem? Please describe.
Since Bacalhau is a distributed system with multiple components, it is critical to have a reliable method for end-to-end testing. Also, it's important that these tests closely resemble a real production environment without relying on mocks.
Additionally, it is important to have an easy way for new users of Bacalhau to quickly experiment with the system on their own local machines, without the need, or the complexity, or the overhead to deploy on a public cloud.
Describe the solution you'd like
The suggested solution is to deploy different components of Bacalhau using Docker Compose, while utilizing the Docker-in-Docker capability.
This solution:
- Allows new users to standup a full deployment of Bacalhau on their own machines within seconds, and experiment with it.
- Gives the ability to write Jobs that uses S3 Input/Output sources by utilizing the Minio component deployed.
- Removes the need to pull/push images from Docker Hub, and instead utilize the Image Registry component deployed.
- Allows to easily and quickly run tests against a real Bacalhau deployment, and quickly iterate and test different setups and scenarios. Please note that Docker compose itself can be put inside a container (using Docker-in-Docker)
- By utilizing Docker networks, it allows us to simulate different scenarios of network failures, latency, and segmentation between Compute Nodes and Requester Nodes.
- Easily scale to multiple requester and compute nodes.
- Allows Bacalhau developers to easily run tests locally before pushing their commits.
- Paves the road for Fuzz testing
- Paves the way for easier testing TLS NATS communication (authentication and authorization)
Describe alternatives you've considered
Alternatives would be to standup different processes on a single host, where each process is a different component of Bacalhau
Additional context**
Few TODOs to keep in mind for later:
- Utilize Sysbox
- Think about Fuzz Tests and best to integrate them
- IPFS support
- Think about how to increase the number or requester nodes and compute nodes when needed to test different scenarios.
- Utilize TestContainers when building the test suite around this setup