custom scanner to scan an image without mounting

[navidshaikh]

User story: I want to implement a scanner which finds out layer details of a container image using docker inspect command. This atomic scanner needs to have docker tooling and image present in docker format.

AFAIK, atomic scanner mounts the image under process at /scannin/ directory, which gives the root fs of the image under process. What if I do not want the container image to be mounted but have access to image on which I can operate via docker CLI?

Is it possible ?

[navidshaikh]

The aim of the scanner is to find out number of layers and metadata of each layer for an image under test, by running docker inspect on every parent image. I have a script which runs on host and using docker CLI finds out this data. I am trying to transform the script into atomic scanner.

[rhatdan]

The atomic scanner does not hand the docker.socket into the container. we are explicity attempting to not do this. Since we don't want the scanning tool to have privileges, and be able to take over the machine.

We could talk about a atomic scan option that could mount all layers of an image and insert the json associated with the image into the /scannin directories, if this makes sense. Otherwise you would need to run an SPC and mount the /run/docker.socket into your container, but this would not work with atomic scan.

[navidshaikh]

The atomic scanner does not hand the docker.socket into the container. we are explicity attempting to not do this. Since we don't want the scanning tool to have privileges, and be able to take over the machine.

Okay, make sense.

We could talk about a atomic scan option that could mount all layers of an image and insert the json associated with the image into the /scannin directories, if this makes sense.

will that be same as the the output of docker save -o /tmp/image.tar <image> && tar -xf /tmp/image.tar ? OR mounting the sub-layers as well which are present in the <layer-id>/layer.tar

[rhatdan]

We mount the rootfs fully exploded. Basically like docker save -o /tmp/image.tar && tar -xf /tmp/image.tar without all of the tarring. Much more efficient.

[navidshaikh]

Associating JSON should help, should we also be able to achieve the ordering of layers as well? (the order in which they are stacked to build the resulting layered image?)

[rhatdan]

@baude WDYT

[rhatdan]

@baude Anything?

[baude]

As Dan said, we do not provide the scanner with enough permissions to run a docker instance. At first glance, I am not sure how you would be able to do this.

What upside do you see coming from running this in a scanner versus outside?

[navidshaikh]

What upside do you see coming from running this in a scanner versus outside?

Say if I want to get layering information about given container image; number of layers; parent of each layer, etc. Without atomic scanner, I will run docker inspect on given container image, take parent Id and inspect <parent-ID> and traverse till base image. The data collected will thus give the meta about each layer.

If we can not have docker instance in atomic scanner because of permissions, is there another way we can get such information?

[rhatdan]

We can do that but probably as a separate tool or a separate option to atomic scan. If you could document exactly what you would want I am sure we can implement it. Atomic scan currently is about scanning the rootfs, you seem to want to scan the structure of an OCI/Docker bundle.

[rhatdan]

@navidshaikh Any comment, or should I close?

[navidshaikh]

We can do that but probably as a separate tool or a separate option to atomic scan.

+1 for a separate option to atomic scan.

If you could document exactly what you would want I am sure we can implement it.

Sure, I'll do that.

Atomic scan currently is about scanning the rootfs, you seem to want to scan the structure of an OCI/Docker bundle.

The structure of image detailing the number of layers and information of each layer, yes!

[baude]

@navidshaikh is this still relevant?

[navidshaikh]

@navidshaikh is this still relevant

@baude

I want to write an atomic scanner which will enlist number of layers for a container image. And information about each layer (json file contents per layer after one extracts the tar ball of repository).

For eg:

➜  ~ docker save -o /tmp/image.tar registry.centos.org/bamachrn/python
➜  ~ mkdir -p /tmp/extracted_image && tar -xf /tmp/image.tar -C /tmp/extracted_image            

➜  ~ cd /tmp/extracted_image 

➜  extracted_image ls
089c585eafbdfee38025525dd968c74d33991a7f1c84048f3f17e40b2a1377dd  84feb3bd8da6721cf6af55365f4f44e1441a43d8241b32be97de77609e7d1e3a
16056948aa4787c3969c3ad9cd52afc4ad85f44c871060e5e567e477a176e8c6  a92385d456318880b01c5cfb9c658aca529fd4a07d9ed7b0333ad103c5076be8
1dcf12d60288fc57e565b01c9f58a69dc6b59432932179637b31306364d4a40a  bb402d5dcd007f9027083cfe3e03143e04f0b238c4c3a97d892c94135f1670bc.json
23777e1e0d73a2bbea57bb248af43e36867612e616032f1b14e4dfc410e84440  c5238c45d837703060ec5f29ed734096d20a689e811eaa30574e1483bd20ffab
36f06df6c65c8bcacc488cdf7834bcaf6379af5c390c0984f70383b91d632d20  ce5daf19330884df2b018573f4171bb28ff9097ca2944399439e3a3e9f24b3af
496b5352fbb7b62327d84482f66959d96bf7b386b811e889e366d4cfa0895d48  f68a33dc803ade42323fda24cc5378381d3a7b9502a3c13e5ef20b3484fb7da5
5035871be84c477ede62d3de5cead05c1f2815f0f16b80417d2517598ddd1ab5  manifest.json
703519d7c01ea0ad97c35613f205a2cdf8cde0a559bc6c246ce4eefdb2ba4429  repositories

➜  extracted_image cat repositories 
{"registry.centos.org/bamachrn/python":{"latest":"16056948aa4787c3969c3ad9cd52afc4ad85f44c871060e5e567e477a176e8c6"}}

➜  extracted_image cat 089c585eafbdfee38025525dd968c74d33991a7f1c84048f3f17e40b2a1377dd/json 
{"id":"089c585eafbdfee38025525dd968c74d33991a7f1c84048f3f17e40b2a1377dd","created":"0001-01-01T00:00:00Z","container_config":{"Hostname":"","Domainname":"","User":"","AttachStdin":false,"AttachStdout":false,"AttachStderr":false,"Tty":false,"OpenStdin":false,"StdinOnce":false,"Env":null,"Cmd":null,"Image":"","Volumes":null,"WorkingDir":"","Entrypoint":null,"OnBuild":null,"Labels":null}}%                                                                                                                                           

➜  extracted_image cat 16056948aa4787c3969c3ad9cd52afc4ad85f44c871060e5e567e477a176e8c6/json 
{"id":"16056948aa4787c3969c3ad9cd52afc4ad85f44c871060e5e567e477a176e8c6","parent":"84feb3bd8da6721cf6af55365f4f44e1441a43d8241b32be97de77609e7d1e3a","created":"2016-09-06T05:33:58.588436817Z","container":"76a5c20c886c0d4dc511094525df0f76a66c9117562ba0282cd3ebd9c15aa770","container_config":{"Hostname":"66388f647a9e","Domainname":"","User":"","AttachStdin":false,"AttachStdout":false,"AttachStderr":false,"Tty":false,"OpenStdin":false,"StdinOnce":false,"Env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Cmd":["/bin/sh","-c","yum install --disablerepo=* --enablerepo=base -y PyYAML libyaml \u0026\u0026 python /set_env/cccp_reader.py"],"ArgsEscaped":true,"Image":"sha256:3858fdea2e8abc3990754291b6dd492696aba791569331ba4464319993e72185","Volumes":null,"WorkingDir":"/test/project","Entrypoint":null,"OnBuild":[],"Labels":{"build-date":"20160729","license":"GPLv2","name":"CentOS Base Image","vendor":"CentOS"}},"docker_version":"1.10.3","config":{"Hostname":"66388f647a9e","Domainname":"","User":"","AttachStdin":false,"AttachStdout":false,"AttachStderr":false,"Tty":false,"OpenStdin":false,"StdinOnce":false,"Env":["PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"],"Cmd":["/bin/sh","-c","'./server.py'"],"ArgsEscaped":true,"Image":"sha256:3858fdea2e8abc3990754291b6dd492696aba791569331ba4464319993e72185","Volumes":null,"WorkingDir":"/test/project","Entrypoint":null,"OnBuild":[],"Labels":{"build-date":"20160729","license":"GPLv2","name":"CentOS Base Image","vendor":"CentOS"}},"architecture":"amd64","os":"linux"}%   

[..]

This way, extract image layers and populate information about each layer in given image. Is this possible to do with an atomic scanner ?