Allow Optional Kustomizations for Paths Which Don't Exist in Source

[isugimpy]

This is kinda a bizarre feature proposal, but I'm hoping other people think it would have merit.

When running a fleet at scale, one of the problems we've been fighting with is consistency across multiple clusters. Ideally, we want to deploy Flux resources to all clusters that look more or less the same, but can have different results depending on the specific cluster, sometimes including no result at all. To that end, I propose a spec.optional field on the Kustomization kind.

To give a concrete example: One of the things we deploy using Flux is Kyverno policies. The set of policies deployed to any given cluster is generally going to be based on the context of that cluster's purpose, environment, etc. In some cases, specific clusters by name have unique policies that must be deployed to exactly that cluster. We maintain a centralized policy repo which contains all of these policies, and have a GitRepository object which points at it. Ideally, we'd deploy a "cluster-specific-policy" Kustomization to all clusters, which would point to a subdirectory in the GitRepository which matches the name of that cluster. However, because we don't have subdirectories for every cluster, because there aren't policies unique to every cluster, that Kustomization will be in a failed state, eventually causing alarms. What I'd like to see is that if spec.optional=true, the Kustomization treats the absence of the directory referred to by spec.path as a valid success state on reconciliation. This would functionally be very similar to the optional property on the pod spec when referring to ConfigMaps.

Is this something that would be considered desirable and acceptable? I'd be happy to look into writing the relevant PR myself if it might be accepted.