From 98e1d4ce6b4b382b7e5cafa2e04ae8cd2c06448f Mon Sep 17 00:00:00 2001 From: Danil-Grigorev Date: Mon, 27 Jan 2025 13:13:30 +0100 Subject: [PATCH 1/2] Document OCI source usage, publish and preload subcommands Signed-off-by: Danil-Grigorev --- .../02_installation/02_plugin-installation.md | 48 +------ .../01_air-gapped-environtment.md | 132 ++++++++++++++++-- docs/book/src/03_topics/03_plugin/00.md | 3 + .../03_topics/03_plugin/01_installation.md | 49 +++++++ .../03_plugin/02_preload_subcommand.md | 77 ++++++++++ .../03_plugin/03_publish_subcommand.md | 99 +++++++++++++ 6 files changed, 349 insertions(+), 59 deletions(-) create mode 100644 docs/book/src/03_topics/03_plugin/00.md create mode 100644 docs/book/src/03_topics/03_plugin/01_installation.md create mode 100644 docs/book/src/03_topics/03_plugin/02_preload_subcommand.md create mode 100644 docs/book/src/03_topics/03_plugin/03_publish_subcommand.md diff --git a/docs/book/src/02_installation/02_plugin-installation.md b/docs/book/src/02_installation/02_plugin-installation.md index 357c37deb..8e0ac9735 100644 --- a/docs/book/src/02_installation/02_plugin-installation.md +++ b/docs/book/src/02_installation/02_plugin-installation.md @@ -1,49 +1,3 @@ # Plugin installation -The `cluster-api-operator` plugin can be installed using krew, the kubectl plugin manager. - -## Prerequisites - -[krew][] installed on your system. See the krew installation guide for instructions. - -[krew]: [https://krew.sigs.k8s.io/docs/user-guide/setup/install/] - -## Steps - -1. Add the cluster-api-operator plugin index to krew: -```bash -kubectl krew index add operator https://github.com/kubernetes-sigs/cluster-api-operator.git -``` - -2. Install the cluster-api-operator plugin: -```bash -kubectl krew install operator/clusterctl-operator -``` - -3. Verify the installation: -```bash -kubectl operator -``` - -This should print help information for the kubectl operator plugin. - -The `cluster-api-operator` plugin is now installed and ready to use with `kubectl`. - -### Optionally: installing as a `clusterctl` plugin -Typically the plugin is installed under `~/.krew/bin/kubectl-operator`, which would be present under your `$PATH` after correct `krew` installation. If you want to use plugin with `clusterctl`, you need to rename this file to be prefixed with `clusterctl-` instead, like so: -```bash -cp ~/.krew/bin/kubectl-operator ~/.krew/bin/clusterctl-operator -``` - -After that plugin is available to use as a `clusterctl` plugin: -```bash -clusterctl operator --help -``` - -## Upgrade - -To upgrade your plugin with the new release of `cluster-api-operator` you will need to run: - -```bash -kubectl krew upgrade -``` \ No newline at end of file +Please refer to [plugin installation](../03_topics/03_plugin/01_installation.md) section. diff --git a/docs/book/src/03_topics/02_configuration/01_air-gapped-environtment.md b/docs/book/src/03_topics/02_configuration/01_air-gapped-environtment.md index e9da828f9..c0285e24e 100644 --- a/docs/book/src/03_topics/02_configuration/01_air-gapped-environtment.md +++ b/docs/book/src/03_topics/02_configuration/01_air-gapped-environtment.md @@ -4,18 +4,20 @@ To install Cluster API providers in an air-gapped environment using the operator 1. Configure the operator for an air-gapped environment: - Manually fetch and store a helm chart for the operator. - - Provide image overrides for the operator in from an accessible image repository. + - Provide image overrides for the operator from an accessible image repository. 2. Configure providers for an air-gapped environment: - - Provide fetch configuration for each provider from an accessible location (e.g., an internal GitHub repository) or from pre-created ConfigMaps within the cluster. + - Provide fetch configuration for each provider from an accessible location: e.g., an OCI artifact, internal Github/Gitlab repository URL or from pre-created ConfigMaps within the cluster. - Provide image overrides for each provider to pull images from an accessible image repository. **Example Usage:** As an admin, I need to fetch the Azure provider components from within the cluster because I am working in an air-gapped environment. +### Using ConfigMap + In this example, there is a ConfigMap in the `capz-system` namespace that defines the components and metadata of the provider. -The Azure InfrastructureProvider is configured with a `fetchConfig` specifying the label selector, allowing the operator to determine the available versions of the Azure provider. Since the provider's version is marked as `v1.9.3`, the operator uses the components information from the ConfigMap with matching label to install the Azure provider. +The Azure InfrastructureProvider is configured with a `fetchConfig` specifying the label selector, allowing the operator to determine the available versions of the Azure provider. Since the provider's version is marked as `v1.9.3`, the operator uses the components information from the ConfigMap with a matching label to install the Azure provider. ```yaml --- @@ -47,39 +49,145 @@ spec: provider-components: azure ``` -### Situation when manifests do not fit into configmap +### Using OCI Artifact + +OCI artifact files can follow these naming patterns: + +- `/:` (e.g., `my-registry.example.com/my-provider:v1.9.3`) +- `/` (e.g., my-registry.example.com/my-provider), in which case the tag is substituted by provider version. + +When working with metadata and component files within OCI artifacts, the files stored in the artifact should follow these naming conventions: + +- **Metadata Files**: + - Default: `metadata.yaml` + - Versioned: `fmt.Sprintf("%s-%s-%s-metadata.yaml", p.GetType(), p.GetName(), p.GetSpec().Version)`, Example: `infrastructure-azure-v1.9.3-metadata.yaml` + +- **Component Files**: + - Default: `components.yaml` + - Typed: `fmt.Sprintf("%s-components.yaml", p.GetType())`, Example: `infrastructure-components.yaml` + - Versioned: `fmt.Sprintf("%s-%s-%s-components.yaml", p.GetType(), p.GetName(), p.GetSpec().Version)`, Example: `infrastructure-azure-v1.9.3-components.yaml` + +Versioned files allow to use single image for hosting multiple provider manifests and versions simulaniously, without overlapping each other. + +Typed allow to store multiple provider types inside single image, which is needed for example for `bootstrap` and `control-plane` providers. + +Example layout for a `kubeadm` provider may look like: +- `metadata.yaml` +- `control-plane-components.yaml` +- `bootstrap-components.yaml` + +To fetch provider components which are stored as an OCI artifact, you can configure `fetchConfig.oci` field to pull them directly from an OCI registry: + +```yaml +apiVersion: operator.cluster.x-k8s.io/v1alpha2 +kind: InfrastructureProvider +metadata: + name: azure + namespace: capz-system +spec: + version: v1.9.3 + configSecret: + name: azure-variables + fetchConfig: + oci: "my-oci-registry.example.com/my-provider:v1.9.3" +``` + +## OCI Authentication + +To securely authenticate with an OCI registry, environment variables are used for user credentials. The following environment variables are involved: + +- **`OCI_USERNAME`**: The username for the OCI registry. +- **`OCI_PASSWORD`**: The password associated with the username. +- **`OCI_ACCESS_TOKEN`**: A token used for authentication. +- **`OCI_REFRESH_TOKEN`**: A refresh token to obtain new access tokens. + +### Fetching Provider Components from a secure OCI Registry + +To fetch provider components stored as an OCI artifact, you can configure the `fetchConfig.oci` field to pull them directly from an OCI registry. The `configSecret` field references a Kubernetes `Secret` that should contain the necessary OCI credentials (such as username and password, or token), ensuring that sensitive information is securely stored. + +Here’s an example of how to configure the `InfrastructureProvider` resource to fetch a specific version of a provider component from an OCI registry: + +```yaml +apiVersion: operator.cluster.x-k8s.io/v1alpha2 +kind: InfrastructureProvider +metadata: + name: azure + namespace: capz-system +spec: + version: v1.9.3 + configSecret: + name: azure-variables # Secret containing the OCI registry credentials + fetchConfig: + oci: "my-oci-registry.example.com/my-provider:v1.9.3" # Reference to the OCI artifact (provider) +``` + +The reference secret can could contain OCI authentication data: + +```yaml +apiVersion: v1 +kind: Secret +metadata: + name: azure-variables # Name of the secret referenced in the InfrastructureProvider + namespace: capz-system # Namespace where the secret resides +type: Opaque +data: + OCI_USERNAME: + OCI_PASSWORD: + OCI_ACCESS_TOKEN: + OCI_REFRESH_TOKEN: +``` + +### Using Github/Gitlab URL + +If the provider components are hosted at a specific repository URL, you can use `fetchConfig.url` to retrieve them directly. + +```yaml +apiVersion: operator.cluster.x-k8s.io/v1alpha2 +kind: InfrastructureProvider +metadata: + name: azure + namespace: capz-system +spec: + version: v1.9.3 + configSecret: + name: azure-variables + fetchConfig: + url: "https://my-internal-repo.example.com/providers/azure/v1.9.3.yaml" +``` + +## Situation when manifests do not fit into ConfigMap -There is a limit on the [maximum size](https://kubernetes.io/docs/concepts/configuration/configmap/#motivation) of a configmap - 1MiB. If the manifests do not fit into this size, Kubernetes will generate an error and provider installation fail. To avoid this, you can archive the manifests and put them in the configmap that way. +There is a limit on the [maximum size](https://kubernetes.io/docs/concepts/configuration/configmap/#motivation) of a ConfigMap - 1MiB. If the manifests do not fit into this size, Kubernetes will generate an error and provider installation will fail. To avoid this, you can archive the manifests and put them in the ConfigMap that way. -For example, you have two files: `components.yaml` and `metadata.yaml`. To create a working config map you need: +For example, you have two files: `components.yaml` and `metadata.yaml`. To create a working ConfigMap, you need: -1. Archive components.yaml using `gzip` cli tool +1. Archive components.yaml using `gzip` CLI tool: ```sh gzip -c components.yaml > components.gz ``` -2. Create a configmap manifest from the archived data +2. Create a ConfigMap manifest from the archived data: ```sh kubectl create configmap v1.9.3 --namespace=capz-system --from-file=components=components.gz --from-file=metadata=metadata.yaml --dry-run=client -o yaml > configmap.yaml ``` -3. Edit the file by adding "provider.cluster.x-k8s.io/compressed: true" annotation +3. Edit the file by adding "provider.cluster.x-k8s.io/compressed: true" annotation: ```sh yq eval -i '.metadata.annotations += {"provider.cluster.x-k8s.io/compressed": "true"}' configmap.yaml ``` -**Note**: without this annotation operator won't be able to determine if the data is compressed or not. +**Note**: Without this annotation, the operator won't be able to determine if the data is compressed or not. -4. Add labels that will be used to match the configmap in `fetchConfig` section of the provider +4. Add labels that will be used to match the ConfigMap in the `fetchConfig` section of the provider: ```sh yq eval -i '.metadata.labels += {"my-label": "label-value"}' configmap.yaml ``` -5. Create a configmap in your kubernetes cluster using kubectl +5. Create the ConfigMap in your Kubernetes cluster using kubectl: ```sh kubectl create -f configmap.yaml diff --git a/docs/book/src/03_topics/03_plugin/00.md b/docs/book/src/03_topics/03_plugin/00.md new file mode 100644 index 000000000..08253dd18 --- /dev/null +++ b/docs/book/src/03_topics/03_plugin/00.md @@ -0,0 +1,3 @@ +# Plugin + +This section descibes plugin commands with usage and examples \ No newline at end of file diff --git a/docs/book/src/03_topics/03_plugin/01_installation.md b/docs/book/src/03_topics/03_plugin/01_installation.md new file mode 100644 index 000000000..357c37deb --- /dev/null +++ b/docs/book/src/03_topics/03_plugin/01_installation.md @@ -0,0 +1,49 @@ +# Plugin installation + +The `cluster-api-operator` plugin can be installed using krew, the kubectl plugin manager. + +## Prerequisites + +[krew][] installed on your system. See the krew installation guide for instructions. + +[krew]: [https://krew.sigs.k8s.io/docs/user-guide/setup/install/] + +## Steps + +1. Add the cluster-api-operator plugin index to krew: +```bash +kubectl krew index add operator https://github.com/kubernetes-sigs/cluster-api-operator.git +``` + +2. Install the cluster-api-operator plugin: +```bash +kubectl krew install operator/clusterctl-operator +``` + +3. Verify the installation: +```bash +kubectl operator +``` + +This should print help information for the kubectl operator plugin. + +The `cluster-api-operator` plugin is now installed and ready to use with `kubectl`. + +### Optionally: installing as a `clusterctl` plugin +Typically the plugin is installed under `~/.krew/bin/kubectl-operator`, which would be present under your `$PATH` after correct `krew` installation. If you want to use plugin with `clusterctl`, you need to rename this file to be prefixed with `clusterctl-` instead, like so: +```bash +cp ~/.krew/bin/kubectl-operator ~/.krew/bin/clusterctl-operator +``` + +After that plugin is available to use as a `clusterctl` plugin: +```bash +clusterctl operator --help +``` + +## Upgrade + +To upgrade your plugin with the new release of `cluster-api-operator` you will need to run: + +```bash +kubectl krew upgrade +``` \ No newline at end of file diff --git a/docs/book/src/03_topics/03_plugin/02_preload_subcommand.md b/docs/book/src/03_topics/03_plugin/02_preload_subcommand.md new file mode 100644 index 000000000..78230b914 --- /dev/null +++ b/docs/book/src/03_topics/03_plugin/02_preload_subcommand.md @@ -0,0 +1,77 @@ +# Using the `preload` Plugin for Kubernetes Operator + +## Overview + +The `preload` subcommand allows users to preload provider `ConfigMaps` into a management cluster from an OCI (Open Container Initiative) artifact, known provider source, or URL override. User can supply any number of provider stings or discover and use existing provider manifests from the cluster. + +## Command Syntax +The basic syntax for using the `preload` command is: + +```sh +kubectl operator preload [flags] +``` + +## Flags and Options +| Flag | Short | Description | +|------|-------|-------------| +| `--kubeconfig` | | Path to the kubeconfig file for the source management cluster. Uses default discovery rules if unspecified. | +| `--existing` | `-e` | Discover all providers in the cluster and prepare `ConfigMap` for each of them. | +| `--core` | | Specifies the core provider and version (e.g., `cluster-api:v1.1.5`). Defaults to the latest release. | +| `--infrastructure` | `-i` | Specifies infrastructure providers and versions (e.g., `aws:v0.5.0`). | +| `--bootstrap` | `-b` | Specifies bootstrap providers and versions (e.g., `kubeadm:v1.1.5`). | +| `--control-plane` | `-c` | Specifies control plane providers and versions (e.g., `kubeadm:v1.1.5`). | +| `--ipam` | | Specifies IPAM providers and versions (e.g., `infoblox:v0.0.1`). | +| `--runtime-extension` | | Specifies runtime extension providers and versions (e.g., `my-extension:v0.0.1`). | +| `--addon` | | Specifies add-on providers and versions (e.g., `helm:v0.1.0`). | +| `--target-namespace` | `-n` | Specifies the target namespace where the operator should be deployed. Defaults to `capi-operator-system`. | +| `--artifact-url` | `-u` | Specifies the URL of the OCI artifact containing component manifests. | + +## Examples + +### Load CAPI Operator Manifests from an OCI Source +```sh +kubectl operator preload --core cluster-api +``` +This command loads the `cluster-api` core provider manifests into the management cluster. If no version is specified, the latest release is used. + +### Load CAPI Operator Manifests from Existing Providers in the Cluster +```sh +kubectl operator preload -e +``` +This command discovers all existing providers in the cluster and prepares ConfigMaps containing their manifests. + +### Prepare Provider ConfigMap from OCI for a Specific Infrastructure Provider +```sh +kubectl operator preload --infrastructure=aws -u my-registry.example.com/infrastructure-provider +``` +This command fetches the latest available version of the `aws` infrastructure provider from the specified OCI registry and creates a ConfigMap. + +### Prepare Provider ConfigMap with a Specific Version +```sh +kubectl operator preload --infrastructure=aws::v2.3.0 -u my-registry.example.com/infrastructure-provider +``` +This command loads the AWS infrastructure provider version `v2.3.0` from the OCI registry into the default namespace. + +### Prepare Provider ConfigMap with a Custom Namespace +```sh +kubectl operator preload --infrastructure=aws:custom-namespace -u my-registry.example.com/infrastructure-provider +``` +This command loads the latest version of the AWS infrastructure provider into the `custom-namespace`. + +### Prepare Provider ConfigMap with a Specific Version and Namespace +```sh +kubectl operator preload --infrastructure=aws:custom-namespace:v2.3.0 -u my-registry.example.com/infrastructure-provider +``` +This command loads AWS provider version `v2.3.0` into `custom-namespace`. + +### Prepare Provider ConfigMap for Multiple Infrastructure Providers +```sh +kubectl operator preload --infrastructure=aws --infrastructure=vsphere -u my-registry.example.com/infrastructure-provider +``` +This command fetches and loads manifests for both AWS and vSphere infrastructure providers from the OCI registry. + +### Prepare Provider ConfigMap with a Custom Target Namespace +```sh +kubectl operator preload --infrastructure aws --target-namespace foo -u my-registry.example.com/infrastructure-provider +``` +This command loads the AWS infrastructure provider into the `foo` namespace, ensuring that the operator uses a customized deployment location. diff --git a/docs/book/src/03_topics/03_plugin/03_publish_subcommand.md b/docs/book/src/03_topics/03_plugin/03_publish_subcommand.md new file mode 100644 index 000000000..8e2d59682 --- /dev/null +++ b/docs/book/src/03_topics/03_plugin/03_publish_subcommand.md @@ -0,0 +1,99 @@ +# Using the `publish` Subcommand + +The `publish` subcommand allows you to publish provider manifests to an OCI registry by constructing an OCI artifact from the provided directory and/or files and pushing it to the specified registry. + +## Usage + +```bash +kubectl operator publish [OPTIONS] +``` + +## Options + +| Flag | Short | Description | +|------------------|--------|---------------------------------------------------------------------------------------------------| +| `--artifact-url` | `-u` | The URL of the OCI artifact to collect component manifests from. This includes the registry and optionally a version/tag. **Example**: `ttl.sh/${IMAGE_NAME}:5m` | +| `--dir` | `-d` | The directory containing the provider manifests. The default is the current directory (`.`). **Example**: `manifests` | +| `--file` | `-f` | A list of specific manifest files to include in the OCI artifact. You can specify one or more files. **Example**: `metadata.yaml`, `infrastructure-components.yaml` | + +## Examples + +### Publish provider manifests from a directory to the OCI registry +This command publishes all files in the `manifests` directory to the OCI registry specified in the `-u` option: +```bash +kubectl operator publish -u ttl.sh/${IMAGE_NAME}:5m -d manifests +``` + +### Publish specific manifest files to the OCI registry +This command publishes the `metadata.yaml` and `infrastructure-components.yaml` files to the OCI registry: +```bash +kubectl operator publish -u ttl.sh/${IMAGE_NAME}:5m -f metadata.yaml -f infrastructure-components.yaml +``` + +### Publish with both directory and specific files +This command combines both the directory (`manifests`) and the custom files (`metadata.yaml`, `infrastructure-components.yaml`): +```bash +kubectl operator publish -u ttl.sh/${IMAGE_NAME}:5m -d manifests -f metadata.yaml -f infrastructure-components.yaml +``` + +## Publishing Multiple Providers and Versions in an OCI Image + +This example demonstrates how to publish three different providers (`control-plane kubeadm`, `bootstrap kubeadm`, and `infrastructure docker`) along with their versioned metadata and components files into a **single OCI image**. Each provider has two versions (`v1.9.3` and `v1.9.4`), and the corresponding metadata and components files follow versioned naming conventions. + +The following layout for the directory can be used: + +```bash +manifests/ +├── control-plane-kubeadm-v1.9.3-metadata.yaml +├── control-plane-kubeadm-v1.9.3-components.yaml +├── bootstrap-kubeadm-v1.9.3-metadata.yaml +├── bootstrap-kubeadm-v1.9.3-components.yaml +├── infrastructure-docker-v1.9.3-metadata.yaml +├── infrastructure-docker-v1.9.3-components.yaml +├── control-plane-kubeadm-v1.9.4-metadata.yaml +├── control-plane-kubeadm-v1.9.4-components.yaml +├── bootstrap-kubeadm-v1.9.4-metadata.yaml +├── bootstrap-kubeadm-v1.9.4-components.yaml +└── infrastructure-docker-v1.9.4-metadata.yaml +└── infrastructure-docker-v1.9.4-components.yaml +``` + +```bash +capioperator publish -u my-registry.example.com/providers:latest -d manifests \ +``` + +This will publish both versions (`v1.9.3` and `v1.9.4`) of each provider into single OCI image, and each version will have its corresponding metadata and component files. + +### Publish with authentication +If authentication is required for the OCI registry, you can specify credentials using environment variables: +```bash +export OCI_USERNAME=myusername +export OCI_PASSWORD=mypassword +kubectl operator publish -u ttl.sh/${IMAGE_NAME}:5m -d manifests +``` + +## OCI Authentication + +To securely authenticate with an OCI registry, the `publish` subcommand relies on environment variables for user credentials. The following environment variables are used: + +- **`OCI_USERNAME`**: The username for the OCI registry. +- **`OCI_PASSWORD`**: The password associated with the username. +- **`OCI_ACCESS_TOKEN`**: A token used for authentication. +- **`OCI_REFRESH_TOKEN`**: A refresh token to obtain new access tokens. + +### Example of Setting Up OCI Authentication + +1. Set the environment variables with your OCI credentials: + +```bash +export OCI_USERNAME=myusername +export OCI_PASSWORD=mypassword +``` + +2. Run the `publish` command, which will automatically use the credentials: + +```bash +kubectl operator publish -u my-oci-registry.com/${IMAGE_NAME}:v0.0.1 -d manifests +``` + +This allows the `publish` subcommand to authenticate to the OCI registry without requiring you to manually input the credentials. \ No newline at end of file From a7f01b3f0829a302edb5605da1e3102051a70ec5 Mon Sep 17 00:00:00 2001 From: Danil Grigorev Date: Tue, 28 Jan 2025 08:54:10 +0100 Subject: [PATCH 2/2] Spelling fixes Co-authored-by: Furkat Gofurov --- .../03_topics/02_configuration/01_air-gapped-environtment.md | 2 +- docs/book/src/03_topics/03_plugin/02_preload_subcommand.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/book/src/03_topics/02_configuration/01_air-gapped-environtment.md b/docs/book/src/03_topics/02_configuration/01_air-gapped-environtment.md index c0285e24e..c8177db1a 100644 --- a/docs/book/src/03_topics/02_configuration/01_air-gapped-environtment.md +++ b/docs/book/src/03_topics/02_configuration/01_air-gapped-environtment.md @@ -67,7 +67,7 @@ When working with metadata and component files within OCI artifacts, the files s - Typed: `fmt.Sprintf("%s-components.yaml", p.GetType())`, Example: `infrastructure-components.yaml` - Versioned: `fmt.Sprintf("%s-%s-%s-components.yaml", p.GetType(), p.GetName(), p.GetSpec().Version)`, Example: `infrastructure-azure-v1.9.3-components.yaml` -Versioned files allow to use single image for hosting multiple provider manifests and versions simulaniously, without overlapping each other. +Versioned files allow to use single image for hosting multiple provider manifests and versions simultaneously, without overlapping each other. Typed allow to store multiple provider types inside single image, which is needed for example for `bootstrap` and `control-plane` providers. diff --git a/docs/book/src/03_topics/03_plugin/02_preload_subcommand.md b/docs/book/src/03_topics/03_plugin/02_preload_subcommand.md index 78230b914..ebbe8780f 100644 --- a/docs/book/src/03_topics/03_plugin/02_preload_subcommand.md +++ b/docs/book/src/03_topics/03_plugin/02_preload_subcommand.md @@ -2,7 +2,7 @@ ## Overview -The `preload` subcommand allows users to preload provider `ConfigMaps` into a management cluster from an OCI (Open Container Initiative) artifact, known provider source, or URL override. User can supply any number of provider stings or discover and use existing provider manifests from the cluster. +The `preload` subcommand allows users to preload provider `ConfigMaps` into a management cluster from an OCI (Open Container Initiative) artifact, known provider source, or URL override. Users can supply any number of provider stings or discover and use existing provider manifests from the cluster. ## Command Syntax The basic syntax for using the `preload` command is: