sourcegraph-executor-k8s pod affinity configuration

[loujar]

Description

The sourcegraph-executor-k8s chart provides an option to configure the Executor pod's node affinity & anti-affinity configuration via the executor.ConfigMap.yaml Config Map:

  EXECUTOR_KUBERNETES_POD_AFFINITY: "{{ .Values.executor.kubernetesJob.pod.affinity }}"
  EXECUTOR_KUBERNETES_POD_ANTI_AFFINITY: "{{ .Values.executor.kubernetesJob.pod.antiAffinity }}"

However, the current template for this Config Map does not account for parsing the JSON structure of these values, resulting in the following helm error:

Example values.override.yaml:

executor:
  kubernetesJob:
    pod:
      affinity: '[{"labelSelector": {"matchExpressions": [{"key": "app","operator": "In","values": ["executor-batches"]}]},"topologyKey": "kubernetes.io/hostname"}]'

helm template output:

$ helm template sourcegraph/sourcegraph-executor-k8s --values=values.override.yaml
Error: YAML parse error on sourcegraph-executor-k8s/templates/executor.ConfigMap.yaml: error converting YAML to JSON: yaml: line 31: did not find expected key

Target Executor Config Map pod affinity configuration:

The Executor Config Map can be manually configured like so:

apiVersion: v1
data:
  EXECUTOR_KUBERNETES_POD_AFFINITY: '[{"labelSelector": {"matchExpressions": [{"key": "app","operator": "In", "values": ["executor-batches"]}]}, "topologyKey": "kubernetes.io/hostname"}]'
kind: ConfigMap
metadata:
  name: executor-batches-codeintel

Expected Executor pod affinity configuration:

Once configured as described above, the k8s Executor will spawn k8s Jobs with the following (abbreviated) Pod definition with functional affinity configuration:

apiVersion: v1
kind: Pod
metadata:
  name: sg-executor-job--4-step.kubernetes.step.0.pre-gz7v7
spec:
  affinity:
    podAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: app
            operator: In
            values:
            - executor-batches

Related Executor configuration documentation

[loujar]

This can be accomplished by escaping all double quote marks like so:

executor:
  kubernetesJob:
    pod:
      affinity: '[{\"labelSelector\": {\"matchExpressions\": [{\"key\": \"app\",\"operator\": \"In\",\"values\": [\"executor-batches\"]}]},\"topologyKey\": \"kubernetes.io/hostname\"}]'

Resulting podAffinity configuration in the k8s executor pod manifest:

apiVersion: v1
kind: Pod
metadata:
  name: sg-executor-job--6-t4mvg
spec:
  affinity:
    podAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: app
            operator: In
            values:
            - executor-batches
        topologyKey: kubernetes.io/hostname
 ...