Add a command to list flux resource that are orphaned / lack a ownerRef

[ecerulm]

From https://cloud-native.slack.com/archives/CLAJ40HV3/p1689680529331029?thread_ts=1689678469.471389&cid=CLAJ40HV3

we've had it come up in conversation a few times recently that it would be nice to have a command that you can run, that looks for any resources that are not in flux, that don't have an ownerRef so weren't created by some other operator, and puts them on a list for you to check for whether that resource should be in there or not
we'd probably like that list to be empty but there can be things installed on the cluster by the platform (cloud) that are not accounted with Flux or ownerRef at all, so we can't just search and destroy them

Scenario :

You create a resource with flux create helmrelease .... then forget to add the resource to git. So you endup with a flux get helm-release xxxx with no corresponding file in git.

It would be nice to have a flux list-orphaned command that list all flux resources that were created from a flux create xxxx command and not from a git file.

[kingdonb]

With awareness that almost every cluster is going to have some unmanaged resources, so this list will not be empty in the typical case, I envision this working with a namespace filter. Running flux list-orphaned would run with the flux-system namespace by default, so it would show you any resources that you accidentally created with flux create not using --export, or any such test resources that have not been cleaned up yet, but only in that namespace.

You could call list-orphaned -A to see all namespaces, bearing in mind that things like CNI, cluster control plane, static pods in kube-system will all show up on this list and that typically they are managed through a different process. (Edit: I see you wrote in your ask that it would list all Flux resources... I imagined that it would list all resources of any kind, namespaces, serviceaccounts, if they don't have Flux origins, you would like to know about them if they are unmanaged, so they can be update to manage using GitOps. The OwnerRef is to exclude resources that are not managed by any operator, and wouldn't likely appear on most any Flux resources.)

Next I'd wish to have an exclude-list capability to make the output of list-orphaned -A more useful, since I'd like to alert on the presence of resources that are unmanaged and unexpected, but not those we know about and have made exceptions for. That config pulls from a file or from the command line parameters, preferably a file IMO, list of resources that are expected to be unmanaged.

Alternatively maybe you'd add an annotation like flux-exclude to resources that Flux should not list here, and give list-orphaned a --show-excluded flag to override the annotation and show them anyway? I don't know what's the best way to handle this from an indexing point of view. (We wouldn't want to retrieve all the resources that might be unmanaged just to find out if they have an annotation... but if we are checking for ownerRef then that ship has already sailed.)

It's definitely worth some thought put into this before we release any changes. I think there are probably some edge cases I didn't consider in the bits above. But we don't need all those bells and whistles if it just helps you track down orphaned Flux resources, since bottom line every Flux resource should definitely be kept in Git unless you are not doing GitOps, and Flux should help you do GitOps.