dind and private registry job container pull...

[keisari-ch]

Hi,

I'd like to setup an ARC environment providing container execution only. Basically each job in a workflow would call the necessary docker image to do the steps actions. Coming from a gitlab environment, i'm pretty hard coded here, the goal having complete rootless containers in the end.

Until now, i have an almost working setup, but the runner just doesnt manage to call the image described in the container key of the workflow because of authentication issues. I understand we can edit the template for the runner, providing an imagePullSecrets, but it seems this only is used for the dind and runner image call. However, id like this imagePullSecrets information to be used to pull the image mentioned in the workflow.

I guess the dind image is the one calling for the docker pull. Is there a way to pass the imagePullSecrets to the dind container without having to rebuild the image and store the docker config in it ?

Template

I overlay this on the default generated one

#@ load("@ytt:overlay", "overlay")

#@overlay/match by=overlay.subset({"kind": "AutoscalingRunnerSet"}),missing_ok=True
---
spec:
  #@overlay/match missing_ok=True
  listenerTemplate:
    spec:
      #@overlay/replace
      containers:
      - name: listener
        securityContext:
          runAsUser: 1000
          runAsNonRoot: true
          allowPrivilegeEscalation: false
          capabilities:
            drop: [ "ALL" ]
          seccompProfile:
            type: RuntimeDefault
      nodeSelector:
        node-type: runner
      tolerations:
      - effect: NoSchedule
        key: kubernetes.azure.com/scalesetpriority
        operator: Equal
        value: spot
  template:
    spec:
      #@overlay/replace
      initContainers:
      - name: init-dind-externals
        image: ghcr.io/actions/actions-runner:latest
        command: ["cp"]
        args: ["-r", "-v", "/home/runner/externals/.", "/home/runner/tmpDir/"]
        volumeMounts:
          - name: dind-externals
            mountPath: /home/runner/tmpDir
      #@overlay/replace
      containers:
      - name: runner
        command: 
          - /home/runner/run.sh
        image: 
          ghcr.io/actions/actions-runner:latest
        env:
          - name: DOCKER_HOST
            value: unix:///var/run/docker.sock
          - name: RUNNER_WAIT_FOR_DOCKER_IN_SECONDS
            value: "120"
        volumeMounts:
          - name: work
            mountPath: /home/runner/_work
          - name: dind-sock
            mountPath: /var/run
      - name: dind
        image: docker:dind
        args:
          - dockerd
          - --host=unix:///var/run/docker.sock
          - --group=$(DOCKER_GROUP_GID)
        env:
          - name: DOCKER_GROUP_GID
            value: "123"
        securityContext:
          privileged: true
        volumeMounts:
          - name: work
            mountPath: /home/runner/_work
          - name: dind-sock
            mountPath: /var/run
          - name: dind-externals
            mountPath: /home/runner/externals
      #@overlay/match missing_ok=True
      imagePullSecrets:
      - name: acr-secret
      restartPolicy: Never
      serviceAccountName: arc-runner-itplat-gha-rs-no-permission
      #@overlay/replace
      volumes:
      - name: dind-sock
        emptyDir: {}
      - name: dind-externals
        emptyDir: {}
      - name: work
        emptyDir: {}
        
      #@overlay/match missing_ok=True
      nodeSelector:
        node-type: runner

      #@overlay/match missing_ok=True
      tolerations:
        - effect: NoSchedule
          key: kubernetes.azure.com/scalesetpriority
          operator: Equal
          value: spot

=> the secret acr-secret is correct, tested and logged in correctly to the azure contrainer registry.

Workflow Example

jobs:
  terraform-plan:
    name: Terraform Plan
    runs-on: arc-runner-teamA
    container: acrteama.azurecr.io/actions/iactools:latest

Logged

Checking docker version
  /usr/bin/docker version --format '{{.Server.APIVersion}}'
  '1.46'
  Docker daemon API version: '1.46'
  /usr/bin/docker version --format '{{.Client.APIVersion}}'
  '1.44'
  Docker client API version: '1.44'
Clean up resources from previous jobs
  /usr/bin/docker ps --all --quiet --no-trunc --filter "label=eeccb0"
  /usr/bin/docker network prune --force --filter "label=eeccb0"
Create local container network
  /usr/bin/docker network create --label eeccb0 github_network_74b1e8[2](https://github.com/org/azure-network/actions/runs/9972361437/job/27760328636#step:2:2)00f3d449ea0b19e0c33915833
  7044d6e4c17c2d6c67fe301ebbe78444ef2d26c78a51492095c921ea5fde5dea
Starting job container
  /usr/bin/docker pull acrsnsfitplat.azurecr.io/actions/itplat-tools:latest
  Error response from daemon: Head "https://acrteama.azurecr.io/v2/actions/iactools/manifests/latest": unauthorized: {"errors":[{"code":"UNAUTHORIZED","message":"authentication required, visit https://aka.ms/acr/authorization for more information."}]}
  Warning: Docker pull failed with exit code 1, back off 5.57[3](https://github.com/org/azure-network/actions/runs/9972361437/job/27760328636#step:2:3) seconds before retry.
  /usr/bin/docker pull acrsnsfitplat.azurecr.io/actions/itplat-tools:latest
  Error response from daemon: Head "https://acrteama.azurecr.io/v2/actions/iactools/manifests/latest": unauthorized: {"errors":[{"code":"UNAUTHORIZED","message":"authentication required, visit https://aka.ms/acr/authorization for more information."}]}
  Warning: Docker pull failed with exit code 1, back off [5](https://github.com/org/azure-network/actions/runs/9972361437/job/27760328636#step:2:5).813 seconds before retry.
  /usr/bin/docker pull acrsnsfitplat.azurecr.io/actions/itplat-tools:latest
  Error response from daemon: Head "https://acrteama.azurecr.io/v2/actions/iactools/manifests/latest": unauthorized: {"errors":[{"code":"UNAUTHORIZED","message":"authentication required, visit https://aka.ms/acr/authorization for more information."}]}
  Error: Docker pull failed with exit code 1

Any help greatly appreciated :/

[keisari-ch]

holy cow.

https://docs.github.com/en/actions/using-jobs/running-jobs-in-a-container#example-defining-credentials-for-a-container-registry