-
Notifications
You must be signed in to change notification settings - Fork 259
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
draft: add namespace scoped operator mode
This adds automation and docs for restricting the operator scope from cluster wide to namespace restricted. Signed-off-by: NymanRobin <[email protected]>
- Loading branch information
1 parent
efae71e
commit b3a3c6e
Showing
14 changed files
with
228 additions
and
36 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| apiVersion: kustomize.config.k8s.io/v1beta1 | ||
| kind: Kustomization | ||
|
|
||
| bases: | ||
| - ../../default | ||
|
|
||
|
|
||
| patches: | ||
| - patch: | | ||
| # Add a namespace to watch | ||
| - op: replace | ||
| path: /kind | ||
| value: RoleBinding | ||
| - op: replace | ||
| path: /roleRef/kind | ||
| value: Role | ||
| - op: add | ||
| path: /metadata/namespace | ||
| value: single-ns-bmh | ||
| target: | ||
| group: rbac.authorization.k8s.io | ||
| kind: ClusterRoleBinding | ||
| name: manager-rolebinding | ||
|
|
||
| patchesStrategicMerge: | ||
| - namespaced-manager-patch.yaml |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| apiVersion: apps/v1 | ||
| kind: Deployment | ||
| metadata: | ||
| name: controller-manager | ||
| namespace: system | ||
| spec: | ||
| template: | ||
| spec: | ||
| containers: | ||
| - name: manager | ||
| env: | ||
| - name: WATCH_NAMESPACE | ||
| value: "single-ns-bmh" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| kind: RoleBinding | ||
| metadata: | ||
| name: manager-rolebinding | ||
| namespace: $(WATCH_NAMESPACE) # Use the variable in the namespace | ||
| roleRef: | ||
| apiGroup: rbac.authorization.k8s.io | ||
| kind: Role | ||
| name: manager-role | ||
| subjects: | ||
| - kind: ServiceAccount | ||
| name: controller-manager | ||
| namespace: $(WATCH_NAMESPACE) # Use the variable in the namespace |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| # How to restrict BMO scope to a single namespace! | ||
|
|
||
| The guide is based on the instructions in this documentation | ||
| https://sdk.operatorframework.io/docs/building-operators/golang/operator-scope/ | ||
|
|
||
| ## To generate manifests for namespace scoped BMO | ||
|
|
||
| To generate namespace-scoped manifests, run the `manifests-kustomize-namespaced` | ||
| make target. This will create manifests configured for the namespace | ||
| `single-ns-bmh`. For further details on the steps involved, or if you | ||
| prefer to manually achieve similar results, please refer to the continuation | ||
| of the documentation. | ||
|
|
||
| ## Watching resources in specific Namespaces | ||
|
|
||
| When setting up the manager, you can use the environment variable | ||
| `WATCH_NAMESPACE` to restrict the operator to a specific namespace. If | ||
| `WATCH_NAMESPACE` is unset or set to an empty string, the operator will | ||
| monitor all namespaces. To limit it to a specific namespace, set | ||
| `WATCH_NAMESPACE` to that namespace. | ||
|
|
||
| For example, to configure the operator to watch the same namespace where | ||
| it is deployed, update the `config/base/manager.yaml` file. Add the | ||
| following configuration under `spec.template.spec.containers.env`: | ||
|
|
||
| ```yaml | ||
| - name: WATCH_NAMESPACE | ||
| valueFrom: | ||
| fieldRef: | ||
| fieldPath: metadata.namespace | ||
| ``` | ||
| ## Restricting Roles and permissions | ||
| When BMO is restricted to a single namespace, the RBAC permissions need | ||
| to be updated accordingly. Instead of using `ClusterRole`, you will use | ||
| `Role`. | ||
|
|
||
| The `Role` is defined in the file `config/base/rbac/role_ns.yaml`. This | ||
| file is auto-generated based on Kubebuilder RBAC markers, specifically those | ||
| in `<some>_controller.go`. The default namespace marking is set to `""`, | ||
| which results in a `ClusterRole`. To restrict it to a specific namespace, | ||
| update this value accordingly. | ||
|
|
||
| You can automatically update the Kubebuilder RBAC markers by running: | ||
|
|
||
| ```bash | ||
| python update_kubebuilder_rbac.py controllers/metal3.io/ your-namespace | ||
| ``` | ||
|
|
||
| The first argument specifies the directory to search, and the second is | ||
| the new namespace. To revert the change, simply set the namespace to `""`. | ||
|
|
||
| After updating the markers, generate the new manifests by running: | ||
|
|
||
| ```bash | ||
| make manifests | ||
| ``` | ||
|
|
||
| Ensure that `config/base/rbac/role_ns.yaml` has been updated to a `Role`. | ||
|
|
||
| Due to limitations in Kubebuilder generation, the `RoleBinding` will not | ||
| be updated automatically. However, a Kustomization overlay is provided to | ||
| replace `ClusterRoleBinding` with `RoleBinding`. This overlay can be found | ||
| in `config/overlays/namespaced`. | ||
|
|
||
| Alternatively, you can manually update `config/base/rbac/role_binding.yaml` | ||
| to achieve the desired outcome. Below is an example of how to modify the | ||
| `role_binding.yaml` file: | ||
|
|
||
| ```yaml: | ||
| apiVersion: rbac.authorization.k8s.io/v1 | ||
| kind: RoleBinding | ||
| metadata: | ||
| name: manager-rolebinding | ||
| namespace: your-namespace | ||
| roleRef: | ||
| apiGroup: rbac.authorization.k8s.io | ||
| kind: Role | ||
| name: manager-role | ||
| namespace: your-namespace | ||
| subjects: | ||
| - kind: ServiceAccount | ||
| name: controller-manager | ||
| namespace: system | ||
| ``` | ||
| Replace `your-namespace` and other fields as necessary to match your | ||
| specific configuration. | ||
|
|
||
| After this you can run `make manifests-kustomize` to get correct RoleBinding generated | ||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| import os | ||
| import re | ||
| import sys | ||
|
|
||
| def update_rbac_markers(directory, namespace): | ||
| rbac_pattern = re.compile(r'(\+kubebuilder:rbac:.*?namespace=")(.*?)(".*)') | ||
|
|
||
| # If the namespace is an empty string, it should look like `namespace=""` | ||
| new_namespace = namespace if namespace else "" | ||
|
|
||
| for root, _, files in os.walk(directory): | ||
| for file in files: | ||
| file_path = os.path.join(root, file) | ||
| if not file.endswith(".go"): | ||
| continue | ||
|
|
||
| with open(file_path, 'r') as f: | ||
| content = f.read() | ||
|
|
||
| new_content, count = rbac_pattern.subn(rf'\1{new_namespace}\3', content) | ||
|
|
||
| if count > 0: | ||
| with open(file_path, 'w') as f: | ||
| f.write(new_content) | ||
| print(f"Updated {count} RBAC markers in {file_path}") | ||
|
|
||
| if __name__ == "__main__": | ||
| if len(sys.argv) != 3: | ||
| print("Usage: python update_kubebuilder_rbac.py <directory> <namespace>") | ||
| sys.exit(1) | ||
|
|
||
| directory = sys.argv[1] | ||
| namespace = sys.argv[2] | ||
|
|
||
| update_rbac_markers(directory, namespace) |