atomic
atomic copied to clipboard
Published
20 hours ago •
projectatomic
projectatomic
Atomic pull tries to get signature files that don't exist off http sigstore
My problem:
The issue is that when I do an sudo atomic pull, it tries to download signature-1 and then signature-2 but signature-2 was never generated when I previously did sudo atomic push.
I'm not sure what I'm doing wrong or if this is an actual bug. The sudo atomic pull command just tries to download signature-1, signature-2, signature-3, etc... even though sudo atomic push only generates a single signature-1 file.
Steps I took:
- On image creation workstation: downloaded alpine image (as a test), re-tag and use
sudo atomic pushto push to atomic registry (running on a "stand-alone" openshift 3.9 docker registry configuration)
sudo docker pull alpine:latest
sudo docker tag alpine:latest docker-registry-default.common.xxx.com/test/alpine:latest
sudo atomic push docker-registry-default.common.xxx.com/test/alpine:latest --sign-by [email protected] --gnupghome /home/vagrant/.gnupg
- Copy signatures created to s3 bucket
- On "destination" workstation, I log into docker and
sudo atomic pull:
oc login https://atomic-registry01.common.xxx.com:8443 -u registryuser -p xxxxxxxxx
sudo docker login -u registryuser -p $(oc whoami -t) docker-registry-default.common.xxx.com
sudo atomic pull docker-registry-default.common.xxx.com/test/alpine
FATA[0004] Source image rejected: Error reading signature from https://s3.amazonaws.com/dis.xxx.com/test/alpine@sha256=cf2d5c15199a1f6161650f2cfe35fd502d0b661823030c7df599050ad895580e/signature-2: status 403
On Image Creation workstation:
/etc/containers/registries.d/*
cat /etc/containers/registries.d/docker-registry-default.common.xxx.com-test.yaml
docker:
docker-registry-default.common.xxx.com/test:
sigstore: https://s3.amazonaws.com/dis.xxx.com
sigstore-staging: file:///home/vagrant/atomic-sigstore
cat /etc/containers/registries.d/default.yaml
# This is a default registries.d configuration file. You may
# add to this file or create additional files in registries.d/.
#
# sigstore: indicates a location that is read and write
# sigstore-staging: indicates a location that is only for write
#
# sigstore and sigstore-staging take a value of the following:
# sigstore: {schema}://location
#
# For reading signatures, schema may be http, https, or file.
# For writing signatures, schema may only be file.
# This is the default signature write location for docker registries.
default-docker:
# sigstore: file:///var/lib/atomic/sigstore
sigstore-staging: file:///var/lib/atomic/sigstore
# The 'docker' indicator here is the start of the configuration
# for docker registries.
#
# docker:
#
# privateregistry.com:
# sigstore: http://privateregistry.com/sigstore/
# sigstore-staging: /mnt/nfs/privateregistry/sigstore
policy.json
cat /etc/containers/policy.json
{
"default": [
{
"type": "insecureAcceptAnything"
}
],
"transports": {
"docker": {
"docker-registry-default.common.xxx.com/test": [
{
"keyType": "GPGKeys",
"type": "signedBy",
"keyData": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
}
]
},
"docker-daemon": {
"": [
{
"type": "insecureAcceptAnything"
}
]
}
}
}
atomic.conf
cat /etc/atomic.conf
# Atomic CLI configuration file
default_scanner:
default_docker: docker
registry_confdir: /etc/containers/registries.d/
discover_sigstores: true
sigstore_metadata_image: sigstore
# Default storage backend [ostree, docker]
# default_storage: docker
# ostree_repository: /ostree/repo
# checkout_path: /var/lib/containers/atomic
#
# Default identity for signing images
# default_signer:
# Absolute path to GPG keyring. Value set as environment variable GNUPGHOME
#gnupg_homedir: /home/USER/.gnupg
#
# To always use a proxy with atomic, you can uncomment and fill out
# below.
#
#http_proxy:
#https_proxy:
#no_proxy:
On Destination workstation:
/etc/containers/policy.json
$ cat /etc/containers/policy.json
{
"default": [
{
"type": "reject"
}
],
"transports": {
"docker": {
"docker-registry-default.common.xxx.com/test": [
{
"keyType": "GPGKeys",
"type": "signedBy",
"keyPath": "/home/vagrant/docker_image_key.gpg"
}
]
},
"docker-daemon": {
"": [
{
"type": "insecureAcceptAnything"
}
]
}
}
}
/etc/containers/registries.d/*
$ cat /etc/containers/registries.d/docker-registry-default.common.xxx.com-test.yaml
docker:
docker-registry-default.common.xxx.com/test:
sigstore: https://s3.amazonaws.com/dis.xxx.com
$ cat /etc/containers/registries.d/default.yaml
# This is a default registries.d configuration file. You may
# add to this file or create additional files in registries.d/.
#
# sigstore: indicates a location that is read and write
# sigstore-staging: indicates a location that is only for write
#
# sigstore and sigstore-staging take a value of the following:
# sigstore: {schema}://location
#
# For reading signatures, schema may be http, https, or file.
# For writing signatures, schema may only be file.
# This is the default signature write location for docker registries.
default-docker:
# sigstore: file:///var/lib/atomic/sigstore
sigstore-staging: file:///var/lib/atomic/sigstore
# The 'docker' indicator here is the start of the configuration
# for docker registries.
#
# docker:
#
# privateregistry.com:
# sigstore: http://privateregistry.com/sigstore/
# sigstore-staging: /mnt/nfs/privateregistry/sigstore
atomic.conf:
$ cat /etc/atomic.conf
# Atomic CLI configuration file
default_scanner:
default_docker: docker
registry_confdir: /etc/containers/registries.d/
discover_sigstores: false
sigstore_metadata_image: sigstore
# Default storage backend [ostree, docker]
# default_storage: docker
# ostree_repository: /ostree/repo
# checkout_path: /var/lib/containers/atomic
#
# Default identity for signing images
# default_signer:
# Absolute path to GPG keyring. Value set as environment variable GNUPGHOME
#gnupg_homedir: /home/USER/.gnupg
#
# To always use a proxy with atomic, you can uncomment and fill out
# below.
#
#http_proxy:
#https_proxy:
#no_proxy:
I should also note the sudo atomic trust show on the destination workstation:
* (default) reject
docker-registry-default.common.xxx.com/test signed [email protected]```
I wonder if the issue is that the skopeo command is a newer build than what the atomic command is expecting. In the debug statements, atomic is using out-dated syntax for skopeo...
FATA[0004] Source image rejected: Error reading signature from https://s3.amazonaws.com/dis.xxx.com/test/alpine@sha256=cf2d5c15199a1f6161650f2cfe35fd502d0b661823030c7df599050ad895580e/signature-2: status 403
Traceback (most recent call last):
File "/bin/atomic", line 185, in <module>
sys.exit(_func())
File "/usr/lib/python2.7/site-packages/Atomic/pull.py", line 62, in pull_image
be.pull_image(self.args.image, remote_image_obj, debug=self.args.debug, assumeyes=self.args.assumeyes, src_creds=src_creds)
File "/usr/lib/python2.7/site-packages/Atomic/backends/_docker.py", line 348, in pull_image
policy_filename=trust.policy_filename, src_creds=src_creds)
File "/usr/lib/python2.7/site-packages/Atomic/util.py", line 454, in skopeo_copy
return check_call(cmd, env=os.environ)
File "/usr/lib/python2.7/site-packages/Atomic/util.py", line 214, in check_call
return subprocess.check_call(cmd, env=env, stdin=stdin, stderr=stderr, stdout=stdout, close_fds=True)
File "/usr/lib64/python2.7/subprocess.py", line 542, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['/usr/bin/skopeo', '--policy=/etc/containers/policy.json', '--debug', 'copy', '--src-tls-verify=false', '--dest-tls-verify=false', '--remove-signatures', 'docker://docker-registry-default.common.xxx.com/test/alpine:latest', 'docker-daemon:docker-registry-default.common.xxx.com/test/alpine:latest']' returned non-zero exit status 1
I was able to manually run the following skopio command using the correct syntax and it works... so atomic is wrapping the skopio command wrong.
[vagrant@centos ~]$ skopeo --policy=/etc/containers/policy.json copy --src-tls-verify=false --dest-tls-verify=false --remove-signatures atomic:docker-registry-default.common.xxx.com/test/alpine:latest docker-daemon:docker-registry-default.common.xxx.com/test/alpine:latest
Copying blob sha256:4fe2ade4980c2dda4fc95858ebb981489baec8c1e4bd282ab1c3560be8ff9bde
2.10 MB / 2.10 MB [========================================================] 3s
Copying config sha256:196d12cf6ab19273823e700516e98eb1910b03b17840f9d5509f03858484d321
1.48 KB / 1.48 KB [========================================================] 0s
Writing manifest to image destination
Storing signatures
Also when I use the proper skopio command (that atomic fails to use), it actually pushes the image properly and stores the signatures in the atomic registry correctly...
The issue is that the atomic command refers to the atomic registry by putting atomic://docker-registry/image:latest in the command. The format skopio expects is: atomic:docker-registry/image:latest (instead of ://, just use :).
This need to be fixed.
Also it should be noted that the skopeo version I'm using as well as the atomic command I'm using was installed via YUM on a CentOS 7.5 box.