Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update index.md #27

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

Update index.md #27

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
dcbe6b0
Update index.md
Rajakavitha1 Aug 30, 2024
c27d527
incorporated the changes suggested
Rajakavitha1 Sep 8, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
146 changes: 69 additions & 77 deletions setting-up/workload-onboarding/container/index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,67 +1,34 @@
---
description: How to use the Bacalhau Docker image
description: How to use Bacalhau Docker Image for task management
---

# Bacalhau Docker Image

This documentation explains how to use the Bacalhau Docker image to run tasks and manage them using the Bacalhau client.

This documentation explains how to use the Bacalhau Docker image for task management with Bacalhau client.
## Prerequisites

To get started, you need to install the Bacalhau client (see more information [here](../../../getting-started/installation.md)) and Docker.

## 1. Pull the Bacalhau Docker image

The first step is to pull the Bacalhau Docker image from the [Github container registry](https://github.com/orgs/bacalhau-project/packages/container/package/bacalhau).

```
docker pull ghcr.io/bacalhau-project/bacalhau:latest
```

Expected output:

```shell
latest: Pulling from bacalhau-project/bacalhau
d14ccdd25413: Pull complete
621f190d05c8: Pull complete
Digest: sha256:3cda5619984de9b56c738c50f94188684170f54f7e417f8dcbe74ff8ec8eb434
Status: Downloaded newer image for ghcr.io/bacalhau-project/bacalhau:latest
ghcr.io/bacalhau-project/bacalhau:latest
```

You can also pull a specific version of the image, e.g.:
Install the [Bacalhau CLI in Docker](../../..//getting-started/installation#step-1.1-install-the-bacalhau-cli).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to install the bacalhau client at all? aren't we using the pre-built image?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to install bacalhau client else I kept getting the error: rkodhand@blr-mpe5j ~ % docker run -t ghcr.io/bacalhau-project/bacalhau:latest \ docker run \ --id-only \ --wait \ ubuntu:latest \ -- sh -c 'uname -a && echo "Hello from Docker Bacalhau!"' 12:02:36.828 | INF pkg/repo/fs.go:93 > Initializing repo at '/root/.bacalhau' for environment 'production' publicapi: after posting request: "This endpoint is deprecated. See the migration guide at https://docs.bacalhau.org/references/cli-reference/command-migration for more information"


```bash
docker pull ghcr.io/bacalhau-project/bacalhau:v0.3.16
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are we pinning to such an old version?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exactly that is why these instructions were removed from the guide and only the latest version is retained.

```

{% hint style="warning" %}
Remember that the "latest" tag is just a string. It doesn't refer to the latest version of the Bacalhau client, it refers to an image that has the "latest" tag. Therefore, if your machine has already downloaded the "latest" image, it won't download it again. To force a download, you can use the `--no-cache` flag.
{% endhint %}

## 2. Check version

To check the version of the Bacalhau client, run:
## 1. Check the version of Bacalhau client

```bash
docker run -t ghcr.io/bacalhau-project/bacalhau:latest version
```

Expected Output:
The output is similar to:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is "Expected Output" - the change is too wordy. If there are variables that will change, then just mention them.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just followed the standard documentation guidelines https://developers.google.com/style/code-in-text


```shell
13:38:54.518 | INF pkg/repo/fs.go:81 > Initializing repo at '/root/.bacalhau' for environment 'production'
CLIENT SERVER UPDATE MESSAGE
v1.2.0 v1.2.0
12:00:32.427 | INF pkg/repo/fs.go:93 > Initializing repo at '/root/.bacalhau' for environment 'production'
CLIENT SERVER UPDATE MESSAGE
v1.3.0 v1.4.0
```

## 3. Running a Bacalhau Job
## 2. Run a Bacalhau Job

In the example below, an Ubuntu-based job runs to print the message 'Hello from Docker Bacalhau':
For example to run an Ubuntu-based job that prints the message 'Hello from Docker Bacalhau':
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the former language better.


```shell
docker run -t ghcr.io/bacalhau-project/bacalhau:latest \
docker run \
bacalhau docker run \
--id-only \
--wait \
ubuntu:latest \
Expand All @@ -70,77 +37,102 @@ docker run -t ghcr.io/bacalhau-project/bacalhau:latest \

### Structure of the command

1. `ghcr.io/bacalhau-project/bacalhau:latest` : Name of the Bacalhau Docker image
2. `--id-only`: Output only the job id
3. `--wait`: Wait for the job to finish
4. `ubuntu:latest.` Ubuntu container
5. `--`: Separate Bacalhau parameters from the command to be executed inside the container
6. `sh -c 'uname -a && echo "Hello from Docker Bacalhau!"'`: The command executed inside the container
1. `--id-only`: Output only the job id
2. `--wait`: Wait for the job to finish
3. `ubuntu:latest.` Ubuntu container
4. `--`: Separate Bacalhau parameters from the command to be executed inside the container
5. `sh -c 'uname -a && echo "Hello from Docker Bacalhau!"'`: The command executed inside the container

Let's have a look at the command execution in the terminal:
The command execution in the terminal is similar to:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The former feels friendlier.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Followed the word standard writing guidelines. https://developers.google.com/style/accessibility


```shell
13:53:46.478 | INF pkg/repo/fs.go:81 > Initializing repo at '/root/.bacalhau' for environment 'production'
ab95a5cc-e6b7-40f1-957d-596b02251a66
j-6ffd54b8-e992-498f-9ee9-766ab09d5daa
```

The output you're seeing is in two parts: **The first line:** `13:53:46.478 | INF pkg/repo/fs.go:81 > Initializing repo at '/root/.bacalhau' for environment 'production'` is an informational message indicating the initialization of a repository at the specified directory `('/root/.bacalhau')` for the `production` environment. **The second line:** `ab95a5cc-e6b7-40f1-957d-596b02251a66` is a `job ID`, which represents the result of executing a command inside a Docker container. It can be used to obtain additional information about the executed job or to access the job's results. We store that in an environment variable so that we can reuse it later on (env: `JOB_ID=ab95a5cc-e6b7-40f1-957d-596b02251a66`)
`j-6ffd54b8-e992-498f-9ee9-766ab09d5daa` is a `job ID`, which represents the result of executing a command inside a Docker container. It can be used to obtain additional information about the executed job or to access the job's results. We store that in an environment variable so that we can reuse it later on (env: `JOB_ID=j-6ffd54b8-e992-498f-9ee9-766ab09d5daa`)

To print out the **content of the Job ID**, run the following command:
To print the **content of the Job ID**, execute the following command:

```
docker run -t ghcr.io/bacalhau-project/bacalhau:latest \
describe ab95a5cc-e6b7-40f1-957d-596b02251a66 \
| grep -A 2 "stdout: |"
bacalhau job describe j-6ffd54b8-e992-498f-9ee9-766ab09d5daa
```

Expected Output:
The output is similar to:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not "is similar to"


```shell
stdout: |
Linux fff680719453 6.2.0-1019-gcp #21~22.04.1-Ubuntu SMP Thu Nov 16 18:18:34 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Hello from Docker Bacalhau!
ID = j-6ffd54b8-e992-498f-9ee9-766ab09d5daa
Name = j-6ffd54b8-e992-498f-9ee9-766ab09d5daa
Namespace = default
Type = batch
State = Completed
Count = 1
Created Time = 2024-09-08 14:33:19
Modified Time = 2024-09-08 14:33:20
Version = 0

Summary
Completed = 1

Job History
TIME REV. STATE TOPIC EVENT
2024-09-08 14:33:19 1 Pending Submission Job submitted
2024-09-08 14:33:19 2 Running
2024-09-08 14:33:20 3 Completed

Executions
ID NODE ID STATE DESIRED REV. CREATED MODIFIED COMMENT
e-bd5746b8 n-e002001e Completed Stopped 6 27m21s ago 27m21s ago Accepted job

Execution e-bd5746b8 History
TIME REV. STATE TOPIC EVENT
2024-09-08 14:33:19 1 New
2024-09-08 14:33:19 2 AskForBid
2024-09-08 14:33:19 3 AskForBidAccepted Requesting Node Accepted job
2024-09-08 14:33:19 4 AskForBidAccepted
2024-09-08 14:33:19 5 BidAccepted
2024-09-08 14:33:20 6 Completed

Standard Output
Linux 7d5c3dcc7fc2 6.5.0-1024-gcp #26~22.04.1-Ubuntu SMP Fri Jun 14 18:48:45 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Hello from Docker Bacalhau!

```

## 4. Submit a Job With Output Files
## 3. Submit a Job With Output Files

One inconvenience that you'll see is that you'll need to mount directories into the container to access files. This is because the container is running in a separate environment from your host machine. Let's take a look at the example below:
You always need to mount directories into the container to access files. This is because the container is running in a separate environment from your host machine.

The first part of the example should look familiar, except for the Docker commands.
The first part of this example should look familiar, except for the Docker commands.

```shell
docker run -t ghcr.io/bacalhau-project/bacalhau:latest \
docker run \
bacalhau docker run \
--id-only \
--wait \
--gpu 1 \
ghcr.io/bacalhau-project/examples/stable-diffusion-gpu:0.0.1 -- \
python main.py --o ./outputs --p "A Docker whale and a cod having a conversation about the state of the ocean"
```
Comment on lines +108 to 114
Copy link
Contributor Author

@Rajakavitha1 Rajakavitha1 Sep 8, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This job was not successful for me and I kept getting the error: Error: not enough nodes to run job. requested: 1, available: 3, suitable: 0. • Node QmPLPUUj: node busy with available capacity {CPU: 0.3500000000000001, Memory: 3.7 GB, Disk: 1.7 TB, GPU: 0} and queue capacity {CPU: 1.5, Memory: 5.0 GB, Disk: 16 GB, GPU: 0} • Node QmbxGSsM: node busy with available capacity {CPU: 0.10000000000000009, Memory: 1.7 GB, Disk: 1.7 TB, GPU: 0} and queue capacity {CPU: 1, Memory: 2.0 GB, Disk: 2.8 kB, GPU: 0} • Node n-e002001e: node busy with available capacity {CPU: 0.6000000000000001, Memory: 2.7 GB, Disk: 1.7 TB, GPU: 0}
I am not sure if it was because of the hardware resource requirements at my end. If yes, then may be best to add the hardware requirements in the Prereq section.


When a job is submitted, Bacalhau prints out the related `job_id` (`a46a9aa9-63ef-486a-a2f8-6457d7bafd2e`):
When a job is submitted, Bacalhau prints the related `job_id` (`j-da29a804-3960-4667-b6e5-73f05e120117`):

```shell
09:05:58.434 | INF pkg/repo/fs.go:81 > Initializing repo at '/root/.bacalhau' for environment 'production'
a46a9aa9-63ef-486a-a2f8-6457d7bafd2e
j-da29a804-3960-4667-b6e5-73f05e120117
```

## 5. Checking the State of your Jobs
## 4. Check the State of your Jobs

**Job status**: You can check the status of the job using `bacalhau job list`.

```bash
docker run -t ghcr.io/bacalhau-project/bacalhau:latest \
list $JOB_ID \
bacalhau job list
```

When it says `Completed`, that means the job is done, and we can get the results.
When it reads `Completed`, that means the job is done, and you can get the results.

**Job information**: You can find out more information about your job by using `bacalhau job describe`.

```bash
docker run -t ghcr.io/bacalhau-project/bacalhau:latest \
describe $JOB_ID \
bacalhau job describe j-da29a804-3960-4667-b6e5-73f05e120117
```

**Job download**: You can download your job results directly by using `bacalhau job get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in the `result` directory.
Expand All @@ -149,7 +141,7 @@ docker run -t ghcr.io/bacalhau-project/bacalhau:latest \
bacalhau job get ${JOB_ID} --output-dir result
```

After the download has finished, you should see the following contents in the results directory.
After the download is complete, you should see the following contents in the results directory.

![png](../../../.gitbook/assets/index\_25\_0.png)

Expand Down