Merge pull request #690 from Danil-Grigorev/document-oci

📖 Document OCI source usage, publish and preload subcommands

docs/book/src/02_installation/02_plugin-installation.md

@@ -1,49 +1,3 @@
 # Plugin installation
 
-The `cluster-api-operator` plugin can be installed using krew, the kubectl plugin manager.
-
-## Prerequisites
-
-[krew][] installed on your system. See the krew installation guide for instructions.
-
-[krew]: [https://krew.sigs.k8s.io/docs/user-guide/setup/install/]
-
-## Steps
-
-1. Add the cluster-api-operator plugin index to krew:
-```bash
-kubectl krew index add operator https://github.com/kubernetes-sigs/cluster-api-operator.git
-```
-
-2. Install the cluster-api-operator plugin:
-```bash
-kubectl krew install operator/clusterctl-operator
-```
-
-3. Verify the installation:
-```bash
-kubectl operator
-```
-
-This should print help information for the kubectl operator plugin.
-
-The `cluster-api-operator` plugin is now installed and ready to use with `kubectl`.
-
-### Optionally: installing as a `clusterctl` plugin
-Typically the plugin is installed under `~/.krew/bin/kubectl-operator`, which would be present under your `$PATH` after correct `krew` installation. If you want to use plugin with `clusterctl`, you need to rename this file to be prefixed with `clusterctl-` instead, like so:
-```bash
-cp ~/.krew/bin/kubectl-operator ~/.krew/bin/clusterctl-operator
-```
-
-After that plugin is available to use as a `clusterctl` plugin:
-```bash
-clusterctl operator --help
-```
-
-## Upgrade
-
-To upgrade your plugin with the new release of `cluster-api-operator` you will need to run:
-
-```bash
-kubectl krew upgrade
-```
+Please refer to [plugin installation](../03_topics/03_plugin/01_installation.md) section.

docs/book/src/03_topics/02_configuration/01_air-gapped-environtment.md

@@ -4,18 +4,20 @@ To install Cluster API providers in an air-gapped environment using the operator
 
 1. Configure the operator for an air-gapped environment:
    - Manually fetch and store a helm chart for the operator.
-   - Provide image overrides for the operator in from an accessible image repository.
+   - Provide image overrides for the operator from an accessible image repository.
 2. Configure providers for an air-gapped environment:
-   - Provide fetch configuration for each provider from an accessible location (e.g., an internal GitHub repository) or from pre-created ConfigMaps within the cluster.
+   - Provide fetch configuration for each provider from an accessible location: e.g., an OCI artifact, internal Github/Gitlab repository URL or from pre-created ConfigMaps within the cluster.
    - Provide image overrides for each provider to pull images from an accessible image repository.
 
 **Example Usage:**
 
 As an admin, I need to fetch the Azure provider components from within the cluster because I am working in an air-gapped environment.
 
+### Using ConfigMap
+
 In this example, there is a ConfigMap in the `capz-system` namespace that defines the components and metadata of the provider.
 
-The Azure InfrastructureProvider is configured with a `fetchConfig` specifying the label selector, allowing the operator to determine the available versions of the Azure provider. Since the provider's version is marked as `v1.9.3`, the operator uses the components information from the ConfigMap with matching label to install the Azure provider.
+The Azure InfrastructureProvider is configured with a `fetchConfig` specifying the label selector, allowing the operator to determine the available versions of the Azure provider. Since the provider's version is marked as `v1.9.3`, the operator uses the components information from the ConfigMap with a matching label to install the Azure provider.
 
 ```yaml
 ---
@@ -47,39 +49,145 @@ spec:
         provider-components: azure
 ```
 
-### Situation when manifests do not fit into configmap
+### Using OCI Artifact
+
+OCI artifact files can follow these naming patterns:
+
+- `<registry>/<repository>:<tag>` (e.g., `my-registry.example.com/my-provider:v1.9.3`)
+- `<registry>/<repository>` (e.g., my-registry.example.com/my-provider), in which case the tag is substituted by provider version.
+
+When working with metadata and component files within OCI artifacts, the files stored in the artifact should follow these naming conventions:
+
+- **Metadata Files**:
+  - Default: `metadata.yaml`
+  - Versioned: `fmt.Sprintf("%s-%s-%s-metadata.yaml", p.GetType(), p.GetName(), p.GetSpec().Version)`, Example: `infrastructure-azure-v1.9.3-metadata.yaml`
+
+- **Component Files**:
+  - Default: `components.yaml`
+  - Typed: `fmt.Sprintf("%s-components.yaml", p.GetType())`, Example: `infrastructure-components.yaml`
+  - Versioned: `fmt.Sprintf("%s-%s-%s-components.yaml", p.GetType(), p.GetName(), p.GetSpec().Version)`, Example: `infrastructure-azure-v1.9.3-components.yaml`
+
+Versioned files allow to use single image for hosting multiple provider manifests and versions simultaneously, without overlapping each other.
+
+Typed allow to store multiple provider types inside single image, which is needed for example for `bootstrap` and `control-plane` providers.
+
+Example layout for a `kubeadm` provider may look like:
+- `metadata.yaml`
+- `control-plane-components.yaml`
+- `bootstrap-components.yaml`
+
+To fetch provider components which are stored as an OCI artifact, you can configure `fetchConfig.oci` field to pull them directly from an OCI registry:
+
+```yaml
+apiVersion: operator.cluster.x-k8s.io/v1alpha2
+kind: InfrastructureProvider
+metadata:
+  name: azure
+  namespace: capz-system
+spec:
+  version: v1.9.3
+  configSecret:
+    name: azure-variables
+  fetchConfig:
+    oci: "my-oci-registry.example.com/my-provider:v1.9.3"
+```
+
+## OCI Authentication
+
+To securely authenticate with an OCI registry, environment variables are used for user credentials. The following environment variables are involved:
+
+- **`OCI_USERNAME`**: The username for the OCI registry.
+- **`OCI_PASSWORD`**: The password associated with the username.
+- **`OCI_ACCESS_TOKEN`**: A token used for authentication.
+- **`OCI_REFRESH_TOKEN`**: A refresh token to obtain new access tokens.
+
+### Fetching Provider Components from a secure OCI Registry
+
+To fetch provider components stored as an OCI artifact, you can configure the `fetchConfig.oci` field to pull them directly from an OCI registry. The `configSecret` field references a Kubernetes `Secret` that should contain the necessary OCI credentials (such as username and password, or token), ensuring that sensitive information is securely stored.
+
+Here’s an example of how to configure the `InfrastructureProvider` resource to fetch a specific version of a provider component from an OCI registry:
+
+```yaml
+apiVersion: operator.cluster.x-k8s.io/v1alpha2
+kind: InfrastructureProvider
+metadata:
+  name: azure
+  namespace: capz-system
+spec:
+  version: v1.9.3
+  configSecret:
+    name: azure-variables  # Secret containing the OCI registry credentials
+  fetchConfig:
+    oci: "my-oci-registry.example.com/my-provider:v1.9.3"  # Reference to the OCI artifact (provider)
+```
+
+The reference secret can could contain OCI authentication data:
+
+```yaml
+apiVersion: v1
+kind: Secret
+metadata:
+  name: azure-variables  # Name of the secret referenced in the InfrastructureProvider
+  namespace: capz-system  # Namespace where the secret resides
+type: Opaque
+data:
+  OCI_USERNAME: <secret>
+  OCI_PASSWORD: <secret>
+  OCI_ACCESS_TOKEN: <secret>
+  OCI_REFRESH_TOKEN: <secret>
+```
+
+### Using Github/Gitlab URL
+
+If the provider components are hosted at a specific repository URL, you can use `fetchConfig.url` to retrieve them directly.
+
+```yaml
+apiVersion: operator.cluster.x-k8s.io/v1alpha2
+kind: InfrastructureProvider
+metadata:
+  name: azure
+  namespace: capz-system
+spec:
+  version: v1.9.3
+  configSecret:
+    name: azure-variables
+  fetchConfig:
+    url: "https://my-internal-repo.example.com/providers/azure/v1.9.3.yaml"
+```
+
+## Situation when manifests do not fit into ConfigMap
 
-There is a limit on the [maximum size](https://kubernetes.io/docs/concepts/configuration/configmap/#motivation) of a configmap - 1MiB. If the manifests do not fit into this size, Kubernetes will generate an error and provider installation fail. To avoid this, you can archive the manifests and put them in the configmap that way.
+There is a limit on the [maximum size](https://kubernetes.io/docs/concepts/configuration/configmap/#motivation) of a ConfigMap - 1MiB. If the manifests do not fit into this size, Kubernetes will generate an error and provider installation will fail. To avoid this, you can archive the manifests and put them in the ConfigMap that way.
 
-For example, you have two files: `components.yaml` and `metadata.yaml`. To create a working config map you need:
+For example, you have two files: `components.yaml` and `metadata.yaml`. To create a working ConfigMap, you need:
 
-1. Archive components.yaml using `gzip` cli tool
+1. Archive components.yaml using `gzip` CLI tool:
 
 ```sh
 gzip -c components.yaml > components.gz
 ```
 
-2. Create a configmap manifest from the archived data
+2. Create a ConfigMap manifest from the archived data:
 
 ```sh
 kubectl create configmap v1.9.3 --namespace=capz-system --from-file=components=components.gz --from-file=metadata=metadata.yaml --dry-run=client -o yaml > configmap.yaml
 ```
 
-3. Edit the file by adding "provider.cluster.x-k8s.io/compressed: true" annotation
+3. Edit the file by adding "provider.cluster.x-k8s.io/compressed: true" annotation:
 
 ```sh
 yq eval -i '.metadata.annotations += {"provider.cluster.x-k8s.io/compressed": "true"}' configmap.yaml
 ```
 
-**Note**: without this annotation operator won't be able to determine if the data is compressed or not.
+**Note**: Without this annotation, the operator won't be able to determine if the data is compressed or not.
 
-4. Add labels that will be used to match the configmap in `fetchConfig` section of the provider
+4. Add labels that will be used to match the ConfigMap in the `fetchConfig` section of the provider:
 
 ```sh
 yq eval -i '.metadata.labels += {"my-label": "label-value"}' configmap.yaml
 ```
 
-5. Create a configmap in your kubernetes cluster using kubectl
+5. Create the ConfigMap in your Kubernetes cluster using kubectl:
 
 ```sh
 kubectl create -f configmap.yaml

docs/book/src/03_topics/03_plugin/00.md

@@ -0,0 +1,3 @@
+# Plugin
+
+This section descibes plugin commands with usage and examples

docs/book/src/03_topics/03_plugin/01_installation.md

@@ -0,0 +1,49 @@
+# Plugin installation
+
+The `cluster-api-operator` plugin can be installed using krew, the kubectl plugin manager.
+
+## Prerequisites
+
+[krew][] installed on your system. See the krew installation guide for instructions.
+
+[krew]: [https://krew.sigs.k8s.io/docs/user-guide/setup/install/]
+
+## Steps
+
+1. Add the cluster-api-operator plugin index to krew:
+```bash
+kubectl krew index add operator https://github.com/kubernetes-sigs/cluster-api-operator.git
+```
+
+2. Install the cluster-api-operator plugin:
+```bash
+kubectl krew install operator/clusterctl-operator
+```
+
+3. Verify the installation:
+```bash
+kubectl operator
+```
+
+This should print help information for the kubectl operator plugin.
+
+The `cluster-api-operator` plugin is now installed and ready to use with `kubectl`.
+
+### Optionally: installing as a `clusterctl` plugin
+Typically the plugin is installed under `~/.krew/bin/kubectl-operator`, which would be present under your `$PATH` after correct `krew` installation. If you want to use plugin with `clusterctl`, you need to rename this file to be prefixed with `clusterctl-` instead, like so:
+```bash
+cp ~/.krew/bin/kubectl-operator ~/.krew/bin/clusterctl-operator
+```
+
+After that plugin is available to use as a `clusterctl` plugin:
+```bash
+clusterctl operator --help
+```
+
+## Upgrade
+
+To upgrade your plugin with the new release of `cluster-api-operator` you will need to run:
+
+```bash
+kubectl krew upgrade
+```

docs/book/src/03_topics/03_plugin/02_preload_subcommand.md

@@ -0,0 +1,77 @@
+# Using the `preload` Plugin for Kubernetes Operator
+
+## Overview
+
+The `preload` subcommand allows users to preload provider `ConfigMaps` into a management cluster from an OCI (Open Container Initiative) artifact, known provider source, or URL override. Users can supply any number of provider stings or discover and use existing provider manifests from the cluster.
+
+## Command Syntax
+The basic syntax for using the `preload` command is:
+
+```sh
+kubectl operator preload [flags]
+```
+
+## Flags and Options
+| Flag | Short | Description |
+|------|-------|-------------|
+| `--kubeconfig` | | Path to the kubeconfig file for the source management cluster. Uses default discovery rules if unspecified. |
+| `--existing` | `-e` | Discover all providers in the cluster and prepare `ConfigMap` for each of them. |
+| `--core` | | Specifies the core provider and version (e.g., `cluster-api:v1.1.5`). Defaults to the latest release. |
+| `--infrastructure` | `-i` | Specifies infrastructure providers and versions (e.g., `aws:v0.5.0`). |
+| `--bootstrap` | `-b` | Specifies bootstrap providers and versions (e.g., `kubeadm:v1.1.5`). |
+| `--control-plane` | `-c` | Specifies control plane providers and versions (e.g., `kubeadm:v1.1.5`). |
+| `--ipam` | | Specifies IPAM providers and versions (e.g., `infoblox:v0.0.1`). |
+| `--runtime-extension` | | Specifies runtime extension providers and versions (e.g., `my-extension:v0.0.1`). |
+| `--addon` | | Specifies add-on providers and versions (e.g., `helm:v0.1.0`). |
+| `--target-namespace` | `-n` | Specifies the target namespace where the operator should be deployed. Defaults to `capi-operator-system`. |
+| `--artifact-url` | `-u` | Specifies the URL of the OCI artifact containing component manifests. |
+
+## Examples
+
+### Load CAPI Operator Manifests from an OCI Source
+```sh
+kubectl operator preload --core cluster-api
+```
+This command loads the `cluster-api` core provider manifests into the management cluster. If no version is specified, the latest release is used.
+
+### Load CAPI Operator Manifests from Existing Providers in the Cluster
+```sh
+kubectl operator preload -e
+```
+This command discovers all existing providers in the cluster and prepares ConfigMaps containing their manifests.
+
+### Prepare Provider ConfigMap from OCI for a Specific Infrastructure Provider
+```sh
+kubectl operator preload --infrastructure=aws -u my-registry.example.com/infrastructure-provider
+```
+This command fetches the latest available version of the `aws` infrastructure provider from the specified OCI registry and creates a ConfigMap.
+
+### Prepare Provider ConfigMap with a Specific Version
+```sh
+kubectl operator preload --infrastructure=aws::v2.3.0 -u my-registry.example.com/infrastructure-provider
+```
+This command loads the AWS infrastructure provider version `v2.3.0` from the OCI registry into the default namespace.
+
+### Prepare Provider ConfigMap with a Custom Namespace
+```sh
+kubectl operator preload --infrastructure=aws:custom-namespace -u my-registry.example.com/infrastructure-provider
+```
+This command loads the latest version of the AWS infrastructure provider into the `custom-namespace`.
+
+### Prepare Provider ConfigMap with a Specific Version and Namespace
+```sh
+kubectl operator preload --infrastructure=aws:custom-namespace:v2.3.0 -u my-registry.example.com/infrastructure-provider
+```
+This command loads AWS provider version `v2.3.0` into `custom-namespace`.
+
+### Prepare Provider ConfigMap for Multiple Infrastructure Providers
+```sh
+kubectl operator preload --infrastructure=aws --infrastructure=vsphere -u my-registry.example.com/infrastructure-provider
+```
+This command fetches and loads manifests for both AWS and vSphere infrastructure providers from the OCI registry.
+
+### Prepare Provider ConfigMap with a Custom Target Namespace
+```sh
+kubectl operator preload --infrastructure aws --target-namespace foo -u my-registry.example.com/infrastructure-provider
+```
+This command loads the AWS infrastructure provider into the `foo` namespace, ensuring that the operator uses a customized deployment location.