Support HTTP endpoint in Kusutomization

[laozc]

We're working on a solution based on Kustomization controller and resources to leverage the inventory and wait capability.
We would like to create a controller to intercept, filter and alter the content (possiblity provide extra signing) before they're actually landing on the cluster after reconcilation.

Per the API definition, currently either GitRepository or Bucket is required when a Kustomization resource reconciles.
https://fluxcd.io/docs/components/kustomize/kustomization/#source-reference

It basically requires us to set up either a Git repository or S3 bucket (via Minio) inside the cluster which we're trying not to do so to keep our design simple.

Another attampt to go through Azure Blob or Git Repsitory outside the cluster may introduce extra latency since the manifest needs to go outside the cluster and then back in. Also, it might need extra measure to protect the data if credentials or other secret data are involved.

I'm wondering if the Kustomization could be able to support an arbitary HTTP URL which points to an internal service.
This may potentially enable the Kustomization controller to work with any controller inside the cluster.

---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: webapp
  namespace: apps
spec:
  interval: 60m0s # detect drift and undo kubectl edits every hour
  wait: true # wait for all applied resources to become ready
  timeout: 3m0s # give up waiting after three minutes
  retryInterval: 2m0s # retry every two minutes on apply or waiting failures
  prune: true # remove stale resources from cluster
  force: true # recreate resources on immutable fields changes
  targetNamespace: apps # set the namespace for all resources
  sourceRef:
    kind: Service
    name: myservice
    namespace: apps
  path: "./deploy/production"

Or a Secret denoting the endpoint would work as well

---
apiVersion: v1
kind: Secret
metadata:
  name: mysecret
  namespace: apps
type: Opaque
data:
  endpoint: some base64 # http://myservice.myns.cluster.local
  username: some base64 # optional
  password: some base64 # optional
  clientCertificate: some base64 # optional
---
apiVersion: kustomize.toolkit.fluxcd.io/v1beta2
kind: Kustomization
metadata:
  name: webapp
  namespace: apps
...
  sourceRef:
    kind: Secret
    name: mysecret
    namespace: apps
  path: "./deploy/production"

[kingdonb]

This was a follow-up to the issue report, answered at:

fluxcd/source-controller#808

In the current state, it's not possible to merge that way. I'm not sure what substantively has changed in the discussion as you propose it now, but thanks for the clarification and moving this to the more appropriate venue, where we can get it in shape to propose an RFC.