Merge branch 'main' into update-list-command

README.md

@@ -5,9 +5,17 @@ coverY: 0
 
 # Welcome to Bacalhau Docs
 
+{% hint style="success" %}
+* Bacalhau is now using [NATS](https://nats.io/). 
+* The support for libp2p and IPFS is deprecated from the version 1.4.0. going forward.
+* The API and the job specifications changed from previous versions.
+
+For more information, check out the release notes.
+{% endhint %}
+
 ## What is Bacalhau?
 
-Bacalhau is a platform for fast, cost efficient, and secure computation by running jobs where the data is generated and stored. With Bacalhau, you can streamline your existing workflows without the need of extensive rewriting by running arbitrary Docker containers and WebAssembly (wasm) images as tasks. This architecture is also referred to as **Compute Over Data** (or CoD). [_Bacalhau_](https://translate.google.com/?sl=pt\&tl=en\&text=bacalhau\&op=translate) _was coined from the Portuguese word for salted Cod fish_.
+Bacalhau is a platform for fast, cost efficient, and secure computation by running jobs where the data is generated and stored. With Bacalhau, you can streamline your existing workflows without the need of extensive rewriting by running arbitrary Docker containers and WebAssembly (WASM) images as tasks. This architecture is also referred to as **Compute Over Data** (or CoD). [_Bacalhau_](https://translate.google.com/?sl=pt\&tl=en\&text=bacalhau\&op=translate) _was coined from the Portuguese word for salted Cod fish_.
 
 Bacalhau seeks to transform data processing for large-scale datasets to improve cost and efficiency, and to open up data processing to larger audiences. Our goals is to create an open, collaborative compute ecosystem that enables unparalleled collaboration. We ([Expanso.io](https://expanso.io)) offer a demo network so you can try out jobs without even installing. Give it a shot!
 

SUMMARY.md

@@ -123,8 +123,8 @@
     * [S3 Source Specification](references/other-specifications/sources/s3.md)
     * [URL Source Specification](references/other-specifications/sources/url.md)
 * [API](references/api/README.md)
+  * [Bacalhau API overview](references/api/index.md)
   * [Agent API](references/api/agent.md)
-  * [Bacalhau API](references/api/index.md)
   * [Jobs API](references/api/jobs.md)
   * [Nodes API](references/api/nodes.md)
 * [CLI](references/cli-reference/README.md)

dev/cli-reference/all-flags.md

@@ -98,7 +98,7 @@ Examples:
   bacalhau create ./job.yaml
 
   # Create a new job from an already executed job
-  bacalhau describe 6e51df50 | bacalhau create -
+  bacalhau job describe 6e51df50 | bacalhau create -
 ```
 
 An example job in YAML format:
@@ -196,7 +196,7 @@ An example UCAN Invocation that runs a WebAssembly job might look like:
 Full description of a job, in yaml format. Use 'bacalhau list' to get a list of all ids. Short form and long form of the job id are accepted.
 
 Usage:
-  bacalhau describe [id] [flags]
+  bacalhau job describe [id] [flags]
 
 Flags:
   -h, --help             help for describe
@@ -209,13 +209,13 @@ Flags:
 ```
 Examples:
   # Describe a job with the full ID
-  bacalhau describe e3f8c209-d683-4a41-b840-f09b88d087b9
+  bacalhau job describe e3f8c209-d683-4a41-b840-f09b88d087b9
 
   # Describe a job with the a shortened ID
-  bacalhau describe 47805f5c
+  bacalhau job describe 47805f5c
 
   # Describe a job and include all server and local events
-  bacalhau describe --include-events b6ad164a
+  bacalhau job describe --include-events b6ad164a
 ```
 
 ## Docker run

dev/debugging/debugging-general.md

@@ -31,7 +31,7 @@ CREATED   ID        JOB                      STATE      VERIFIED  PUBLISHED
 
 ## 2. Inspecting the Status of the Job
 
-When you first suspect that your job has failed, the first thing you should do is inspect the status. The `bacalhau describe $JOB_ID` command presents everything that is known about a job from the perspective of the network.
+When you first suspect that your job has failed, the first thing you should do is inspect the status. The `bacalhau job describe $JOB_ID` command presents everything that is known about a job from the perspective of the network.
 
 Look through the `Shards` of the job and see if any of them have a `State` of `Error`. The `RunOutput` field provides the juicy details of what went wrong.
 

examples/data-engineering/blockchain-etl/index.md

@@ -175,19 +175,19 @@ bacalhau job list --id-filter ${JOB_ID}
 
 When it says `Published` or `Completed`, that means the job is done, and we can get the results.
 
-* **Job information**: You can find out more information about your job by using `bacalhau describe`.
+* **Job information**: You can find out more information about your job by using `bacalhau job describe`.
 
 ```bash
 %%bash
-bacalhau describe ${JOB_ID}
+bacalhau job describe ${JOB_ID}
 ```
 
-* **Job download**: You can download your job results directly by using `bacalhau get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
+* **Job download**: You can download your job results directly by using `bacalhau job get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
 
 ```bash
 %%bash
 rm -rf ./results && mkdir -p ./results # Temporary directory to store the results
-bacalhau get --output-dir ./results ${JOB_ID} # Download the results
+bacalhau job get --output-dir ./results ${JOB_ID} # Download the results
 ```
 
 After the download has finished you should see the following contents in the results directory.
@@ -253,7 +253,7 @@ And then download all the results and merge them into a single directory. This m
 %%bash
 for id in $(cat job_ids.txt); do \
     rm -rf results_$id && mkdir results_$id
-    bacalhau get --output-dir results_$id $id &
+    bacalhau job get --output-dir results_$id $id &
 done
 wait
 ```

examples/data-engineering/csv-to-avro-or-parquet/index.md

@@ -190,7 +190,7 @@ When a job is submitted, Bacalhau prints out the related `job_id`. We store that
 ```
 
 ```
-env: JOB_ID=bacalhau describe 71ecde0e-dac3-4c8d-bf2e-7a92cc54425e
+env: JOB_ID=bacalhau job describe 71ecde0e-dac3-4c8d-bf2e-7a92cc54425e
 ```
 
 ## Checking the State of your Jobs
@@ -211,19 +211,19 @@ bacalhau job list --id-filter={JOB_ID}
 
 When it says `Published` or `Completed`, that means the job is done, and we can get the results.
 
-* **Job information**: You can find out more information about your job by using `bacalhau describe`.
+* **Job information**: You can find out more information about your job by using `bacalhau job describe`.
 
 ```bash
 %%bash
-bacalhau describe {JOB_ID}
+bacalhau job describe {JOB_ID}
 ```
 
-* **Job download**: You can download your job results directly by using `bacalhau get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
+* **Job download**: You can download your job results directly by using `bacalhau job get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
 
 ```bash
 %%bash
 rm -rf results && mkdir -p results
-bacalhau get ${JOB_ID} --output-dir results
+bacalhau job get ${JOB_ID} --output-dir results
 ```
 
 ## Viewing your Job Output

examples/data-engineering/image-processing/index.md

@@ -69,19 +69,19 @@ bacalhau job list --id-filter=${JOB_ID} --no-style
 
 When it says `Published` or `Completed`, that means the job is done, and we can get the results.
 
-* **Job information**: You can find out more information about your job by using `bacalhau describe`.
+* **Job information**: You can find out more information about your job by using `bacalhau job describe`.
 
 ```bash
 %%bash
-bacalhau describe ${JOB_ID}
+bacalhau job describe ${JOB_ID}
 ```
 
-* **Job download**: You can download your job results directly by using `bacalhau get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
+* **Job download**: You can download your job results directly by using `bacalhau job get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
 
 ```bash
 %%bash
 rm -rf results && mkdir results # Temporary directory to store the results
-bacalhau get ${JOB_ID} --output-dir results # Download the results
+bacalhau job get ${JOB_ID} --output-dir results # Download the results
 ```
 
 After the download has finished you should see the following contents in results directory.

examples/data-engineering/index-1.md

@@ -160,17 +160,17 @@ bacalhau job list --id-filter ${JOB_ID}
 
 When it says `Published` or `Completed`, that means the job is done, and we can get the results.
 
-**Job information**: You can find out more information about your job by using `bacalhau describe`.
+**Job information**: You can find out more information about your job by using `bacalhau job describe`.
 
 ```bash
-bacalhau describe ${JOB_ID}
+bacalhau job describe ${JOB_ID}
 ```
 
-**Job download**: You can download your job results directly by using `bacalhau get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory (`results`) and downloaded our job output to be stored in that directory.
+**Job download**: You can download your job results directly by using `bacalhau job get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory (`results`) and downloaded our job output to be stored in that directory.
 
 ```bash
 rm -rf results && mkdir -p results # Temporary directory to store the results
-bacalhau get ${JOB_ID} --output-dir results # Download the results
+bacalhau job get ${JOB_ID} --output-dir results # Download the results
 ```
 
 ## Viewing your Job Output[​](http://localhost:3000/examples/data-engineering/blockchain-etl/#viewing-your-job-output) <a href="#viewing-your-job-output" id="viewing-your-job-output"></a>
@@ -247,7 +247,7 @@ Wait until all of these jobs have been completed. And then download all the resu
 ```bash
 for id in $(cat job_ids.txt); do \
     rm -rf results_$id && mkdir results_$id
-    bacalhau get --output-dir results_$id $id &
+    bacalhau job get --output-dir results_$id $id &
 done
 wait
 ```

examples/data-engineering/index-2.md

@@ -359,17 +359,17 @@ bacalhau job list --id-filter ${JOB_ID}
 
 When it says `Published` or `Completed`, that means the job is done, and we can get the results.
 
-**Job information**: You can find out more information about your job by using `bacalhau describe`.
+**Job information**: You can find out more information about your job by using `bacalhau job describe`.
 
 ```bash
-bacalhau describe ${JOB_ID}
+bacalhau job describe ${JOB_ID}
 ```
 
-**Job download**: You can download your job results directly by using `bacalhau get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory (`results`) and downloaded our job output to be stored in that directory.
+**Job download**: You can download your job results directly by using `bacalhau job get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory (`results`) and downloaded our job output to be stored in that directory.
 
 ```bash
 rm -rf results && mkdir -p results # Temporary directory to store the results
-bacalhau get ${JOB_ID} --output-dir results # Download the results
+bacalhau job get ${JOB_ID} --output-dir results # Download the results
 ```
 
 ## Viewing your Job Output[​](http://localhost:3000/examples/data-engineering/csv-to-avro-or-parquet/#viewing-your-job-output) <a href="#viewing-your-job-output" id="viewing-your-job-output"></a>

examples/data-engineering/index-3.md

@@ -93,17 +93,17 @@ bacalhau job list --id-filter ${JOB_ID}
 
 When it says `Completed`, that means the job is done, and we can get the results.
 
-**Job information**: You can find out more information about your job by using `bacalhau describe`:
+**Job information**: You can find out more information about your job by using `bacalhau job describe`:
 
 ```bash
-bacalhau describe ${JOB_ID}
+bacalhau job describe ${JOB_ID}
 ```
 
-**Job download**: You can download your job results directly by using `bacalhau get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
+**Job download**: You can download your job results directly by using `bacalhau job get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
 
 ```bash
 rm -rf results && mkdir results
-bacalhau get ${JOB_ID} --output-dir results
+bacalhau job get ${JOB_ID} --output-dir results
 ```
 
 ### Display the image[​](http://localhost:3000/examples/data-engineering/image-processing/#display-the-image) <a href="#display-the-image" id="display-the-image"></a>

examples/data-engineering/index-4.md

@@ -254,18 +254,18 @@ bacalhau job list --id-filter ${JOB_ID}
 
 When it says `Published` or `Completed`, that means the job is done, and we can get the results.
 
-**Job information**: You can find out more information about your job by using `bacalhau describe`.
+**Job information**: You can find out more information about your job by using `bacalhau job describe`.
 
 ```bash
-bacalhau describe ${JOB_ID}
+bacalhau job describe ${JOB_ID}
 ```
 
-**Job download**: You can download your job results directly by using `bacalhau get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory (`results`) and downloaded our job output to be stored in that directory.
+**Job download**: You can download your job results directly by using `bacalhau job get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory (`results`) and downloaded our job output to be stored in that directory.
 
 ```bash
 rm -rf results
 mkdir -p ./results # Temporary directory to store the results
-bacalhau get ${JOB_ID} --output-dir ./results # Download the results
+bacalhau job get ${JOB_ID} --output-dir ./results # Download the results
 ```
 
 ## Viewing your Job Output[​](http://localhost:3000/examples/data-engineering/oceanography-conversion/#viewing-your-job-output) <a href="#viewing-your-job-output" id="viewing-your-job-output"></a>

examples/data-engineering/index-5.md

@@ -97,22 +97,22 @@ bacalhau job list --id-filter ${JOB_ID} --no-style
 
 When it says `Published` or `Completed`, that means the job is done, and we can get the results.
 
-**Job information**: You can find out more information about your job by using `bacalhau describe`.
+**Job information**: You can find out more information about your job by using `bacalhau job describe`.
 
 ```bash
-bacalhau describe ${JOB_ID}
+bacalhau job describe ${JOB_ID}
 ```
 
-**Job download**: You can download your job results directly by using `bacalhau get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory (`results`) and downloaded our job output to be stored in that directory.
+**Job download**: You can download your job results directly by using `bacalhau job get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory (`results`) and downloaded our job output to be stored in that directory.
 
 ```bash
 mkdir -p ./results # Temporary directory to store the results
-bacalhau get ${JOB_ID} --output-dir ./results # Download the results
+bacalhau job get ${JOB_ID} --output-dir ./results # Download the results
 ```
 
 ## Viewing your Job Output[​](http://localhost:3000/examples/data-engineering/simple-parallel-workloads/#viewing-your-job-output) <a href="#viewing-your-job-output" id="viewing-your-job-output"></a>
 
-To view the results open the `results/outputs/` folder.&#x20;
+To view the results open the `results/outputs/` folder.
 
 {% file src="../../.gitbook/assets/scaled_Prominent_Late_Gothic_styled_architecture.mp4" %}
 

examples/data-engineering/index.md

@@ -151,17 +151,17 @@ bacalhau job list --id-filter ${JOB_ID}
 
 When it says `Published` or `Completed`, that means the job is done, and we can get the results.
 
-**Job information**: You can find out more information about your job by using `bacalhau describe`.
+**Job information**: You can find out more information about your job by using `bacalhau job describe`.
 
 ```bash
-bacalhau describe ${JOB_ID}
+bacalhau job describe ${JOB_ID}
 ```
 
-**Job download**: You can download your job results directly by using `bacalhau get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
+**Job download**: You can download your job results directly by using `bacalhau job get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
 
 ```bash
 rm -rf results && mkdir -p results
-bacalhau get $JOB_ID --output-dir results
+bacalhau job get $JOB_ID --output-dir results
 ```
 
 ## Viewing your Job Output
@@ -250,17 +250,17 @@ When a job is submitted, Bacalhau prints out the related `job_id`. We store that
 bacalhau job list --id-filter ${JOB_ID} --wide
 ```
 
-**Job information**: You can find out more information about your job by using `bacalhau describe`.
+**Job information**: You can find out more information about your job by using `bacalhau job describe`.
 
 ```bash
-bacalhau describe ${JOB_ID}
+bacalhau job describe ${JOB_ID}
 ```
 
-**Job download**: You can download your job results directly by using `bacalhau get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
+**Job download**: You can download your job results directly by using `bacalhau job get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
 
 ```bash
 rm -rf results && mkdir -p results
-bacalhau get $JOB_ID --output-dir results
+bacalhau job get $JOB_ID --output-dir results
 ```
 
 ## Viewing your Job Output

examples/data-engineering/oceanography-conversion/index.md

@@ -255,20 +255,20 @@ bacalhau job list --id-filter ${JOB_ID}
 
 When it says `Published` or `Completed`, that means the job is done, and we can get the results.
 
-* **Job information**: You can find out more information about your job by using `bacalhau describe`.
+* **Job information**: You can find out more information about your job by using `bacalhau job describe`.
 
 ```bash
 %%bash
-bacalhau describe ${JOB_ID}
+bacalhau job describe ${JOB_ID}
 ```
 
-* **Job download**: You can download your job results directly by using `bacalhau get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
+* **Job download**: You can download your job results directly by using `bacalhau job get`. Alternatively, you can choose to create a directory to store your results. In the command below, we created a directory and downloaded our job output to be stored in that directory.
 
 ```bash
 %%bash
 rm -rf results
 mkdir -p ./results # Temporary directory to store the results
-bacalhau get --output-dir ./results ${JOB_ID} # Download the results
+bacalhau job get --output-dir ./results ${JOB_ID} # Download the results
 ```
 
 ## Viewing your Job Output