SUMMARY.md

Table of contents

• Welcome to the Bacalhau Docs

Getting Started

• How Bacalhau Works
• Installation
• Create Network
• Hardware Setup
• Workload Onboarding
  • Docker Workloads
  • WebAssembly (WASM) Workloads

Setting Up

• Running Nodes
  • Node Onboarding
  • GPU Installation
  • Job selection policy
  • Access Management
  • Node persistence
  • Connect Storage
  • Configuration Management
  • Configuring Transport Level Security
  • Observability
  • Limits and Timeouts
  • Test Network Locally
  • Bacalhau WebUI
• Workload Onboarding
  • Container
    • Docker Workload Onboarding
    • WebAssembly (WASM) Workloads
    • Bacalhau Docker Image
    • How To Work With Custom Containers in Bacalhau
  • Python
    • Building and Running Custom Python Container
    • Running Pandas on Bacalhau
    • Running a Python Script
    • Running Jupyter Notebooks on Bacalhau
    • Scripting Bacalhau with Python
  • R (language)
    • Building and Running your Custom R Containers on Bacalhau
    • Running a Simple R Script on Bacalhau
  • Run CUDA programs on Bacalhau
  • Running a Prolog Script
  • Reading Data from Multiple S3 Buckets using Bacalhau
  • Running Rust programs as WebAssembly (WASM)
  • Generate Synthetic Data using Sparkov Data Generation technique
• Data Ingestion
  • Copy Data from URL to Public Storage
  • Pinning Data
  • Running a Job over S3 data
• Networking Instructions
  • Accessing the Internet from Jobs
• GPU Workloads Setup
• Automatic Update Checking
• Marketplace Deployments
  • Google Cloud Marketplace

Guides

• Write a config.yaml
• Write a SpecConfig

Examples

• Data Engineering
  • Using Bacalhau with DuckDB
  • Ethereum Blockchain Analysis with Ethereum-ETL and Bacalhau
  • Convert CSV To Parquet Or Avro
  • Simple Image Processing
  • Oceanography - Data Conversion
  • Video Processing
• Model Inference
  • EasyOCR (Optical Character Recognition) on Bacalhau
  • Running Inference on Dolly 2.0 Model with Hugging Face
  • Speech Recognition using Whisper
  • Stable Diffusion on a GPU
  • Stable Diffusion on a CPU
  • Object Detection with YOLOv5 on Bacalhau
  • Generate Realistic Images using StyleGAN3 and Bacalhau
  • Stable Diffusion Checkpoint Inference
  • Running Inference on a Model stored on S3
• Model Training
  • Training Pytorch Model with Bacalhau
  • Training Tensorflow Model
  • Stable Diffusion Dreambooth (Finetuning)
• Molecular Dynamics
  • Running BIDS Apps on Bacalhau
  • Coresets On Bacalhau
  • Genomics Data Generation
  • Gromacs for Analysis
  • Molecular Simulation with OpenMM and Bacalhau

References

• Jobs Guide
  • Job Specification
    • Job Types
    • Task Specification
      • Engines
        • Docker Engine Specification
        • WebAssembly (WASM) Engine Specification
      • Publishers
        • IPFS Publisher Specification
        • Local Publisher Specification
        • S3 Publisher Specification
      • Sources
        • IPFS Source Specification
        • Local Source Specification
        • S3 Source Specification
        • URL Source Specification
      • Network Specification
      • Input Source Specification
      • Resources Specification
      • ResultPath Specification
    • Constraint Specification
    • Labels Specification
    • Meta Specification
  • Job Templates
  • Queuing & Timeouts
    • Job Queuing
    • Timeouts Specification
  • Job Results
    • State
• CLI Guide
  • Single CLI commands
    • Agent
      • Agent Overview
      • Agent Alive
      • Agent Node
      • Agent Version
    • Config
      • Config Overview
      • Config Auto-Resources
      • Config Default
      • Config List
      • Config Set
    • Job
      • Job Overview
      • Job Describe
      • Job Exec
      • Job Executions
      • Job History
      • Job List
      • Job Logs
      • Job Run
      • Job Stop
    • Node
      • Node Overview
      • Node Approve
      • Node Delete
      • Node List
      • Node Describe
      • Node Reject
  • CLI Commands Overview
  • Command Migration
• API Guide
  • Bacalhau API overview
  • Best Practices
  • Agent Endpoint
  • Orchestrator Endpoint
  • Migration API
• Node Management
• Authentication & Authorization
• Database Integration
• Debugging
  • Debugging Failed Jobs
  • Debugging Locally
• Open Telemetry in Bacalhau
• Running locally with 'devstack'
• Setting up Dev Environment

Help & FAQ

• Bacalhau FAQs
• Release Notes
• Glossary

Integrations

• Apache Airflow Provider for Bacalhau
• Lilypad
• Bacalhau Python SDK
• Observability for WebAssembly Workloads in Bacalhau

Community

• Social Media
• Style Guide
• Ways to Contribute