Porch PackageVariant Kptfile pipeline editing feature

[johnbelamaric]

Part of the PV design in #36 - see https://github.com/GoogleContainerTools/kpt/blob/main/docs/design-docs/08-package-variant.md#krm-function-pipeline1

[SimonTheLeg]

Short update on this. In order to merge kptdev/kpt#3925 in an elegant way, the following two upstream contributions are required:

• Add DeepCopy for ResourceMeta and ObjectMeta kubernetes-sigs/kustomize#5145
• update dependencies to k8s.io v0.27.2 kubernetes-sigs/cli-utils#619

[SimonTheLeg]

So short update on this. It turns out the kubernetes-sigs cli-utils is dependent on another upstream component, controller-runtime, which then is dependent on a release of apimachinery. This brings our total upstream "mergetrain" to this:

• Add DeepCopy for ResourceMeta and ObjectMeta kubernetes-sigs/kustomize#5145
• update dependencies to k8s.io v0.27.2 kubernetes-sigs/cli-utils#619
• ✨ upgrade to k8s 0.27.1 kubernetes-sigs/controller-runtime#2189
• a release of apimachinery that includes this commit kubernetes/apimachinery@c2d59b0

Long story short: At this point it is difficult to assess how long it will take for upstream to finish. And I would like to avoid stepping even deeper into the rabbit hole.

Therefore, tomorrow I will work on addressing the open topics from @johnbelamaric's review. This then will either be using it without deepcopy or I go into Mortens idea of the kptfile being a string.

[johnbelamaric]

Any updates here, @SimonTheLeg ?

[SimonTheLeg]

Unfortunately not. I am really struggling with the editing via kyaml. I have it now to a point where it is possible to append to an existing map and it will keep the seqIndentStyle, but now it is struggling with appending if the list was empty before. In a nutshell: either I am missing something quite obvious or this is really complicated to do. In any case: I'll give it another shot today and will push what I have at the end of the day. And then we can have a look either after sig-automation today or pkg-mgmt subproject tomorrow

[johnbelamaric]

Are you using the kpt function SDK? It makes this a lot easier, once you understand it. Take a look at how I did the ConfigMap changes in the package context code, or later today I can provide an example another PR I am working on.

[SimonTheLeg]

So I have pushed something now that is working using kyaml that has the same features as the solution before. I'll have to add the validators and idempotency next.

---

Are you using the kpt function SDK?

That is a good point. I was not yet. I had a look now, but the main challenges for me so far were the reverse appending to a yaml sequence and keeping the comments. And at least from a first glance I don't think kpt fn sdk helps with that. Also in the packageContextCode I noticed:

• I am not sure if cm.Get into a map[string]string will keep the comments. I guess we might need to add a test for this as well
• https://github.com/GoogleContainerTools/kpt/blob/main/porch/controllers/packagevariants/pkg/controllers/packagevariant/packagevariant_controller.go#L768 This will change yamlSeqStyle to the compact one instead of keeping the original style.

[johnbelamaric]

Yeah, good point about the comments. @kispaljr I believe has an update to the SDK that preserves comments better. We can get that integrated at some point. I am OK with forgoing that and treating it as an SDK bug that needs fixing, for now. In other words, I would just go ahead and use the SDK for now. You are right the way I do it in the link is not the best, for these reasons, and also should be fixed. I am more interested in getting the critical path going; we can perfect it later (though we should capture it in an issue).

[johnbelamaric]

This PR may be helpful to you as well: kptdev/kpt#3940