Operator deploy options and master format?

[erikgb]

Current status

Currently trivy-operator supports the following deploy options:

1. Helm
2. Static YAML (somehow modular; multiple files)
3. Static YAML (single file)

In addition, there is also support for kustomize deploy based on the single-file static YAML.

To have single source of truth (DRY), which I think is crucial for maintainers/contributors, the Helm chart is considered as master and static resources are generated by rendering the chart with predefined values.

Challenges

When using controller-gen to generate CRDs and operator RBAC (#204) from code makers, this will generate static resources. For CRDs this works ok with Helm, since CRDs are not usually not templated with Helm. Draft PR: #232. But when generating ClusterRole/Role from code markers, it seems hard/impossible add Helm templating to the generated resources - which is unfortunate, but could be acceptable. Draft PR: #215.

Generating static resources from Helm chart also seems like an uncommon thing to do, and can be confusing for contributors. Example: #195. This has also been discussed in #207, that discusses how to keep generated (slave) files up-to-date.

Proposed solution

To better support resources generated from code markers, I suggest establishing kustomize as the new master format for deploying trivy-operator. To not reinvent the wheel, I suggest using the kubebuilder config structure as inspiration: https://github.com/kubernetes-sigs/kubebuilder/tree/master/testdata/project-v3/config.

This should work well for the static YAML deploy options, as we can easily generate YAML using kustomize - thus avoiding hacky scripts for this task. Or leave this up to the users to do themselves....

The question is what to do with the Helm deploy option? We could try out Helmify to generate the Helm chart from the kustomization. This will probably introduce breaking changes in the chart when migrating, which is not optimal. But I think we should try to avoid maintaining resources in multiple places....

WDYT?

[NissesSenap]

All above sounds very reasonable.
About breaking changes, I will write the same thing here as I did in the starboard discussion.

I think the trivy-operator (starboard-operator) got multiple issues in it. I think it's better if we run fast and destroy things for a while to be able to move as quick as possible when it makes sense.
In short I think the breaking changes that helmify would create shouldn't be to much of an issue as long as we document how to migrate to the new helm chart

Great idea @erikgb