Issue with shared libraries (libcrypto, libssl) causing symbolic link loop during Docker build using custom Kaniko image

[kraghupathi]

Issue with shared libraries (libcrypto, libssl) causing symbolic link loop during Docker build using custom Kaniko image

Description:
I'm encountering a problem while building a Docker image using a custom Kaniko image. When using the custom-built Kaniko image, the build process fails with the following errors related to shared libraries (libcrypto.so.3, libssl.so.3), indicating a symbolic link loop:

INFO[0008] Running: [/bin/sh -c apk update] Error loading shared library libcrypto.so.3: Symbolic link loop (needed by /usr/sbin/apk) Error loading shared library libssl.so.3: Symbolic link loop (needed by /lib/libapk.so.2.14.0) Error loading shared library libcrypto.so.3: Symbolic link loop (needed by /lib/libapk.so.2.14.0) Error relocating /lib/libapk.so.2.14.0: SSL_get1_peer_certificate: symbol not found Error relocating /lib/libapk.so.2.14.0: SSL_CTX_use_certificate_chain_file: symbol not found Error relocating /lib/libapk.so.2.14.0: OPENSSL_init_ssl: symbol not found Error relocating /lib/libapk.so.2.14.0: SSL_free: symbol not found Error relocating /lib/libapk.so.2.14.0: SSL_CTX_set_verify: symbol not found Error relocating /lib/libapk.so.2.14.0: EVP_get_digestbyname: symbol not found Error relocating /lib/libapk.so.2.14.0: SSL_shutdown: symbol not found

However, if I directly use the base gcr.io/kaniko-project/executor:debug image, everything works fine. I believe the problem might be due to how libraries are being copied or linked in the custom-built image (registry.gitlab.com/test:9.2 is a rhel ubi image).

1. below is my custom kaniko image dockerfile

`FROM gcr.io/kaniko-project/executor:debug as kaniko
COPY certs/ca.pem kaniko/ssl/certs/ca-certificates.crt

FROM registry.gitlab.com/test:9.2
RUN rm -rf /var/mail
COPY --from=kaniko /kaniko /kaniko
ENV DOCKER_CONFIG /kaniko/.docker/
ENV DOCKER_CREDENTIAL_GCR_CONFIG /kaniko/.config/gcloud/docker_credential_gcr_config.json
`
2. Create another Dockerfile using the custom-built Kaniko image (from Step 1):

FROM docker:26.1.2 RUN apk update CMD ["echo", "This is a 'Purpose-Built Container', It is not meant to be ran this way. Please review the documentation on usage."]

Can someone help me here? Is this the right way to build a custom Kaniko image? Do custom Kaniko image builds work in this manner, or are there any suggestions to resolve the issue?

[vladaurosh]

Did you check readme file?
kaniko is meant to be run as an image: gcr.io/kaniko-project/executor. We do not recommend running the kaniko executor binary in another image, as it might not work as you expect - see [Known Issues](https://github.com/GoogleContainerTools/kaniko#known-issues).

[halradaideh]

this is ugly, but it works

Thanks to this
actions/actions-runner-controller#3687 (comment)

dockerfile

# Base images
FROM debian:bookworm-slim AS debian
RUN apt update; apt install -y curl jq git ca-certificates

FROM gcr.io/kaniko-project/executor:debug

# Copy binaries from Debian
COPY --from=debian /usr/bin/jq /usr/bin/curl /usr/bin/git* /bin/bash /usr/local/bin/
COPY --from=debian /usr/lib/git-core /usr/lib/git-core
COPY --from=debian /usr/share/git-core /usr/share/git-core

# Copy all required libraries from Debian
COPY --from=debian /lib/x86_64-linux-gnu/ /lib/x86_64-linux-gnu/
COPY --from=debian /usr/lib/x86_64-linux-gnu/ /usr/lib/x86_64-linux-gnu/
COPY --from=debian /lib64/ /lib64/
COPY --from=debian /etc/ssl/ /etc/ssl/

# Lie about the container being Debian to make ARC behave nicely
COPY --from=debian /etc/os-release /etc/os-release

# Set working directory
WORKDIR /workspace
RUN mkdir /root

ENV LD_LIBRARY_PATH=/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu:/lib64

# Define the default entrypoint
ENTRYPOINT ["/kaniko/executor"]

and github action

    - name: Run Kaniko Executor
      shell: sh
      env:
        GIT_USERNAME: ${{ github.actor }}
        GIT_PASSWORD: ${{ inputs.ghtoken }}
        KANIKO_CACHE_ARGS: "--cache=true --cache-copy-layers=true --cache-ttl=24h"
      run: |
        DESTINATIONS=""
        for tag in $(echo "${{ env.TAGS }}" | tr ',' ' '); do
          DESTINATIONS="$DESTINATIONS --destination=$tag"
        done

        # remove built image dynamic libraries since kaniko is too sensitive 
        # https://github.com/GoogleContainerTools/kaniko#known-issues
        # Define targets as a space-separated list
        targets="/usr/local/bin/jq /usr/local/bin/curl /usr/local/bin/git /usr/local/bin/bash /usr/lib/git-core /usr/share/git-core /lib/x86_64-linux-gnu/ /usr/lib/x86_64-linux-gnu/ /lib64/ /etc/ssl/ /etc/os-release"
        
        # Move all targets to /tmp
        for target in $targets; do
            mkdir -p /tmp/$(dirname $target)
            mv "$target" "/tmp/$target"
        done

        /kaniko/executor \
          --context="${{ github.repositoryUrl }}#${{ github.ref }}#${{ github.sha }}" \
          --context-sub-path="${{ inputs.context }}" \
          --dockerfile="${{ inputs.file }}" \
          $DESTINATIONS \
          --skip-tls-verify \
          --verbosity=info \
          ${{ inputs.build_args }} \
          ${{ env.KANIKO_CACHE_ARGS }} \
          --log-timestamp

        # Restore targets back to their original locations, i don't care just want the action to finish
        for target in $targets; do
            mv "/tmp/$target" "$target" || true
        done