Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deploy Bacalhau using Docker Compose #4595

Closed
jamlo opened this issue Oct 7, 2024 · 0 comments · Fixed by #4596
Closed

Deploy Bacalhau using Docker Compose #4595

jamlo opened this issue Oct 7, 2024 · 0 comments · Fixed by #4596
Assignees
Labels
request/new Request: Indicates a new request that has been submitted and awaits initial triage type/enhancement Type: New features or enhancements to existing features

Comments

@jamlo
Copy link
Contributor

jamlo commented Oct 7, 2024

Is your feature request related to a problem? Please describe.

Since Bacalhau is a distributed system with multiple components, it is critical to have a reliable method for end-to-end testing. Also, it's important that these tests closely resemble a real production environment without relying on mocks.

Additionally, it is important to have an easy way for new users of Bacalhau to quickly experiment with the system on their own local machines, without the need, or the complexity, or the overhead to deploy on a public cloud.


Describe the solution you'd like

The suggested solution is to deploy different components of Bacalhau using Docker Compose, while utilizing the Docker-in-Docker capability.

This solution:

  1. Allows new users to standup a full deployment of Bacalhau on their own machines within seconds, and experiment with it.
  2. Gives the ability to write Jobs that uses S3 Input/Output sources by utilizing the Minio component deployed.
  3. Removes the need to pull/push images from Docker Hub, and instead utilize the Image Registry component deployed.
  4. Allows to easily and quickly run tests against a real Bacalhau deployment, and quickly iterate and test different setups and scenarios. Please note that Docker compose itself can be put inside a container (using Docker-in-Docker)
  5. By utilizing Docker networks, it allows us to simulate different scenarios of network failures, latency, and segmentation between Compute Nodes and Requester Nodes.
  6. Easily scale to multiple requester and compute nodes.
  7. Allows Bacalhau developers to easily run tests locally before pushing their commits.
  8. Paves the road for Fuzz testing
  9. Paves the way for easier testing TLS NATS communication (authentication and authorization)

Describe alternatives you've considered

Alternatives would be to standup different processes on a single host, where each process is a different component of Bacalhau


Additional context**

Few TODOs to keep in mind for later:

  1. Utilize Sysbox
  2. Think about Fuzz Tests and best to integrate them
  3. IPFS support
  4. Think about how to increase the number or requester nodes and compute nodes when needed to test different scenarios.
  5. Utilize TestContainers when building the test suite around this setup
@jamlo jamlo added type/enhancement Type: New features or enhancements to existing features request/new Request: Indicates a new request that has been submitted and awaits initial triage labels Oct 7, 2024
@jamlo jamlo self-assigned this Oct 7, 2024
@jamlo jamlo moved this from Inbox to In Progress in Engineering Planning Oct 7, 2024
jamlo added a commit that referenced this issue Oct 7, 2024
jamlo added a commit that referenced this issue Oct 7, 2024
@jamlo jamlo linked a pull request Oct 7, 2024 that will close this issue
jamlo added a commit that referenced this issue Oct 9, 2024
This pull requests addresses issue #4595
@github-project-automation github-project-automation bot moved this from In Progress to Done in Engineering Planning Oct 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
request/new Request: Indicates a new request that has been submitted and awaits initial triage type/enhancement Type: New features or enhancements to existing features
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

1 participant