Analyse images builded from Docker Buildx remote driver

[Momotoculteur]

Question

Hello all !

I have a kubernetes which run a Docker buildkit rootless daemon (See that docker buildx, but currently i made my own chart here)

I want to have that scenario in my Jenkins pipeline :

1. Build a docker image via Buildx cli (contact the daemon and send build instructions, which actually works )
2. Scan that image via Trivy (which not work)
3. Push my image if i have no CVE to my registry (which works)

I already try to add --load args in my command. I runs these commands in pipeline :

docker buildx create --name testBuilder --driver=remote tcp://buildkitd.docker-buildkit.svc.cluster.local:1234 --use --bootstrap
docker buildx build -f Dockerfile -o type=docker,dest=- --load -t image:1.0.0
trivy image momo:1.0.0

That should be possible in my opinion, but i think i'm not good with docker context/builder.. :(

Target

None

Scanner

None

Output Format

None

Mode

None

Operating System

No response

Version

No response

[knqyf263]

Did you try --docker-host?

[Momotoculteur]

I already tried to create context with following command in order to let trivy make a docker pull image to scan it :

docker context create myctx --docker host=tcp://buildkitd.docker-buildkit.svc.cluster.local:1234

But that doesnt help. Am i doing something wrong ?

Edit: got that following error :

Head "http://buildkitd.docker-buildkit:1234/_ping": net/http: HTTP/1.x transport connection broken: malformed HTTP response "\x00\x00\x06\x04\x00\x00\x00\x00\x00\x00\x05\x00\x00@\x00".
* Are you trying to connect to a TLS-enabled daemon without TLS?

TLS is not enabled for sure as described in my chart

With following command, that begin to be worse :

docker context create myctx --docker host=tcp://buildkitd.docker-buildkit.svc.cluster.local:1234,skip-tls-verify=true

with following error =>

$ docker image ls
error during connect: Head "https://buildkitd.docker-buildkit:1234/_ping": tls: first record does not look like a TLS handshak

[knqyf263]

@afdesk Can you please take a look?

[afdesk]

Hi @Momotoculteur!
I'd like to resolve your issue, and it seems I need your assist to reproduce this one.

Now my steps are next:

1. Deploy Kubernetes manifests for BuildKit on my local minikube through Deployment + Service

$ ./create-certs.sh 127.0.0.1
$ kubectl apply -f .certs/buildkit-daemon-certs.yaml
$ kubectl apply -f deployment+service.rootless.yaml
$ kubectl scale --replicas=3 deployment/buildkitd
$ kubectl port-forward service/buildkitd 1234

2. Create a new builder for buildx:

$ docker buildx create \
  --name mybuilder \
  --driver remote \
  --driver-opt cacert=${PWD}/.certs/client/ca.pem,cert=${PWD}/.certs/client/cert.pem,key=${PWD}/.certs/client/key.pem,servername=127.0.0.1 \
 tcp://localhost:1234
mybuilder
$ docker buildx inspect --bootstrap 
Name:          mybuilder
Driver:        remote
Last Activity: 2024-07-25 06:42:37 +0000 UTC

Nodes:
Name:             mybuilder0
Endpoint:         tcp://localhost:1234
...

3. Build the simplest docker image

$ cat Dockerfile 
FROM alpine:3.16.1

$ docker buildx build --builder mybuilder -t myrego/myimage:1 . --load
[+] Building 0.9s (6/6) FINISHED                                                                                                                       remote:mybuilder
 => [internal] load build definition from Dockerfile                                                                                                               0.0s
 => => transferring dockerfile: 55B                                                                                                                                0.0s
 => [internal] load metadata for docker.io/library/alpine:3.16.1                                                                                                   0.6s
 => [internal] load .dockerignore                                                                                                                                  0.0s
 => => transferring context: 2B                                                                                                                                    0.0s
 => CACHED [1/1] FROM docker.io/library/alpine:3.16.1@sha256:7580ece7963bfa863801466c0a488f11c86f85d9988051a9f9c68cb27f6b7872                                      0.0s
 => => resolve docker.io/library/alpine:3.16.1@sha256:7580ece7963bfa863801466c0a488f11c86f85d9988051a9f9c68cb27f6b7872                                             0.0s
 => exporting to docker image format                                                                                                                               0.2s
 => => exporting layers                                                                                                                                            0.0s
 => => exporting manifest sha256:414f5de3ccda6ae498a2bad1a08bc4543394462e67e2568e3b9df75ce2c472a9                                                                  0.0s
 => => exporting config sha256:df2f35046d25a99c91ff8add145d166f54ef42d07dd896654cf0ccf643954dad                                                                    0.0s
 => => sending tarball                                                                                                                                             0.2s
 => importing to docker   

$ docker buildx ls
NAME/NODE        DRIVER/ENDPOINT            STATUS    BUILDKIT   PLATFORMS
mybuilder*       remote                                          
 \_ mybuilder0    \_ tcp://localhost:1234   running   655a124    linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386
default          docker                                          
 \_ default       \_ default                running   v0.15.0    linux/amd64, linux/amd64/v2, linux/amd64/v3, linux/386

4. Run a Trivy scan:

$ trivy image myrego/myimage:1
2024-07-25T12:47:41+06:00	INFO	Need to update DB
2024-07-25T12:47:41+06:00	INFO	Downloading DB...	repository="ghcr.io/aquasecurity/trivy-db:2"
50.21 MiB / 50.21 MiB [----------------------------------------------------------------------------------------------------------------------] 100.00% 5.68 MiB p/s 9.0s
2024-07-25T12:47:52+06:00	INFO	Vulnerability scanning is enabled
2024-07-25T12:47:52+06:00	INFO	Secret scanning is enabled
2024-07-25T12:47:52+06:00	INFO	If your scanning is slow, please try '--scanners vuln' to disable secret scanning
2024-07-25T12:47:52+06:00	INFO	Please see also https://aquasecurity.github.io/trivy/v0.53/docs/scanner/secret#recommendation for faster secret detection
2024-07-25T12:47:52+06:00	INFO	Detected OS	family="alpine" version="3.16.1"
2024-07-25T12:47:52+06:00	INFO	[alpine] Detecting vulnerabilities...	os_version="3.16" repository="3.16" pkg_num=14
2024-07-25T12:47:52+06:00	INFO	Number of language-specific files	num=0
2024-07-25T12:47:52+06:00	WARN	This OS version is no longer supported by the distribution	family="alpine" version="3.16.1"
2024-07-25T12:47:52+06:00	WARN	The vulnerability detection may be insufficient because security updates are not provided

myrego/myimage:1 (alpine 3.16.1)

Total: 23 (UNKNOWN: 0, LOW: 0, MEDIUM: 14, HIGH: 8, CRITICAL: 1)

[afdesk]

can you pull any images?

$ docker pull alpine:3.15.1

[Momotoculteur]

I can't.. :( that why i would like if you could target a docker host which is in another namespace and not in tcp://localhost:1234 ?

[afdesk]

I can't..

maybe i missed something, but if you can't pull any images, it's a docker issue

it looks you should resolve this issue at first.

~/agent $ docker image ls
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

[Momotoculteur]

I will check TLS and namespace subjects then. But Buildkit's team is not as fast as you are for communicate :) Seems to be a docker issue and not Trivy.

Thanks a lot for your help, and your time @afdesk