Unable Xray an target image

[gnought]

Expected Behavior

run slim x nginx:latest --debug --verbose should return xray reports

---

Actual Behavior

It gives an API error(500)

md=xray state=started
cmd=xray info=params target='nginx:latest' add-image-manifest='false' add-image-config='false' rm-file-artifacts='false'
cmd=xray state=image.api.inspection.start
cmd=xray info=image size.human='68 MB' architecture='arm64' id='sha256:0f04e4f646a3f14bf31d8bc8d885b6c951fdcf42589d06845f64d18aec6a3c4d' size.bytes='67668418'
cmd=xray info=image.stack index='0' name='nginx:latest' id='sha256:0f04e4f646a3f14bf31d8bc8d885b6c951fdcf42589d06845f64d18aec6a3c4d' instructions='17' message='see report file for details'
cmd=xray info=image.exposed_ports list='80/tcp'
cmd=xray state=image.api.inspection.done
cmd=xray state=image.data.inspection.start
cmd=xray info=image.data.inspection.save.image.start
time="2024-06-13T21:03:44+08:00" level=error msg="dockerutil.SaveImage: dclient.ExportImage() error = API error (500): invalid repository name (0f04e4f646a3f14bf31d8bc8d885b6c951fdcf42589d06845f64d18aec6a3c4d), cannot specify 64-byte hexadecimal strings"
time="2024-06-13T21:03:44+08:00" level=fatal msg="slim: failure" error="API error (500): invalid repository name (0f04e4f646a3f14bf31d8bc8d885b6c951fdcf42589d06845f64d18aec6a3c4d), cannot specify 64-byte hexadecimal strings" stack="goroutine 1 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x64\ngithub.com/slimtoolkit/slim/pkg/util/errutil.FailOn({0x102b5ff98, 0x140005220a8})\n\tgithub.com/slimtoolkit/slim/pkg/util/errutil/errutil.go:32 +0x38\ngithub.com/slimtoolkit/slim/pkg/app/master/command/xray.OnCommand(0x140004c0d68, 0x140004c8480, 0x140004c0d88, {0x16f252d37, 0xc}, 0x1, {0x0, 0x0}, {0x0, 0x0}, ...)\n\tgithub.com/slimtoolkit/slim/pkg/app/master/command/xray/handler.go:374 +0x21e0\ngithub.com/slimtoolkit/slim/pkg/app/master/command/xray.glob..func1(0x140000bfa80)\n\tgithub.com/slimtoolkit/slim/pkg/app/master/command/xray/cli.go:337 +0x1710\ngithub.com/urfave/cli/v2.(*Command).Run(0x103c731c0, 0x140000beac0)\n\tgithub.com/urfave/cli/[email protected]/command.go:163 +0x4a8\ngithub.com/urfave/cli/v2.(*App).RunContext(0x140002c51e0, {0x102b7e098?, 0x103cf1e60}, {0x14000116050, 0x5, 0x5})\n\tgithub.com/urfave/cli/[email protected]/app.go:313 +0x808\ngithub.com/urfave/cli/v2.(*App).Run(...)\n\tgithub.com/urfave/cli/[email protected]/app.go:224\ngithub.com/slimtoolkit/slim/pkg/app/master.Run()\n\tgithub.com/slimtoolkit/slim/pkg/app/master/app.go:15 +0x4c\nmain.main()\n\tgithub.com/slimtoolkit/slim/cmd/slim/main.go:15 +0x194\n" version="darwin/arm64|Transformer|1.40.11|latest|latest"

---

Specifications

• Version: slim version darwin/arm64|Transformer|1.40.11|latest|latest
• Platform: macOS 14.5
• Docker info

$ docker info
Client:
Version:    26.1.3
Context:    orbstack
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
  Version:  v0.14.1
  Path:     /Users/devlocal/.docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
  Version:  v2.27.1
  Path:     /Users/devlocal/.docker/cli-plugins/docker-compose

Server:
Containers: 4
Running: 1
Paused: 0
Stopped: 3
Images: 51
Server Version: 26.1.4
Storage Driver: overlayfs
driver-type: io.containerd.snapshotter.v1
Logging Driver: json-file
Cgroup Driver: cgroupfs
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 3a4de459a68952ffb703bbe7f2290861a75b6b67
runc version: 51d5e94601ceffbbd85688df1c928ecccbfa4685
init version: de40ad0
Security Options:
seccomp
 Profile: builtin
cgroupns
Kernel Version: 6.9.3-orbstack-00146-g1a8d02c90788
Operating System: OrbStack
OSType: linux
Architecture: aarch64
CPUs: 12
Total Memory: 15.59GiB
Name: orbstack
ID: ed3c0974-9b58-465e-906c-ea982b257a80
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: true
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine
Default Address Pools:
 Base: 192.168.215.0/24, Size: 24
 Base: 192.168.228.0/24, Size: 24
 Base: 192.168.247.0/24, Size: 24
 Base: 192.168.207.0/24, Size: 24
 Base: 192.168.167.0/24, Size: 24
 Base: 192.168.107.0/24, Size: 24
 Base: 192.168.237.0/24, Size: 24
 Base: 192.168.148.0/24, Size: 24
 Base: 192.168.214.0/24, Size: 24
 Base: 192.168.165.0/24, Size: 24
 Base: 192.168.227.0/24, Size: 24
 Base: 192.168.181.0/24, Size: 24
 Base: 192.168.158.0/24, Size: 24
 Base: 192.168.117.0/24, Size: 24
 Base: 192.168.155.0/24, Size: 24
 Base: 192.168.147.0/24, Size: 24
 Base: 192.168.229.0/24, Size: 24
 Base: 192.168.183.0/24, Size: 24
 Base: 192.168.156.0/24, Size: 24
 Base: 192.168.97.0/24, Size: 24
 Base: 192.168.171.0/24, Size: 24
 Base: 192.168.186.0/24, Size: 24
 Base: 192.168.216.0/24, Size: 24
 Base: 192.168.242.0/24, Size: 24
 Base: 192.168.166.0/24, Size: 24
 Base: 192.168.239.0/24, Size: 24
 Base: 192.168.223.0/24, Size: 24
 Base: 192.168.164.0/24, Size: 24
 Base: 192.168.163.0/24, Size: 24
 Base: 192.168.172.0/24, Size: 24
 Base: 172.17.0.0/16, Size: 16
 Base: 172.18.0.0/16, Size: 16
 Base: 172.19.0.0/16, Size: 16
 Base: 172.20.0.0/14, Size: 16
 Base: 172.24.0.0/14, Size: 16
 Base: 172.28.0.0/14, Size: 16

[kcq]

@gnought are you using Orbstack instead of Docker Desktop?

[gnought]

@kcq yes, you are right.

[kcq]

@gnought Still need to verify this... but it appears it should be straightforward to fix. Looks like Orbstack doesn't like missing sha256 prefix in image IDs, at least, in some of the Docker API calls. Will need your help verifying the enhancement.

[gnought]

@kcq sure, tag me and let me know how to test the fix. :)

[kcq]

@gnought try this new release and see if you get different results with it

[gnought]

Hi @kcq, I got a similar error for v1.41.4

gnought:~/work/bin/dist_mac_m1 ?1 % ./docker-slim --version
mint version darwin/arm64|Aurora|1.41.4|b59472b8cf16b26575df9960d8ede8f6aa4fb6a9|2024-06-23_08:28:41AM
gnought:~/work/bin/dist_mac_m1 √ % ./docker-slim xray nginx
cmd=xray state=started
cmd=xray info=cmd.input.params add-image-manifest='false' add-image-config='false' rm-file-artifacts='false' target='nginx'
cmd=xray state=image.api.inspection.start
cmd=xray info=image size.human='68 MB' architecture='arm64' id='sha256:9c367186df9a6b18c6735357b8eb7f407347e84aea09beb184961cb83543d46e' size.bytes='67669989'
cmd=xray info=image.stack message='see report file for details' index='0' name='nginx:latest' id='sha256:9c367186df9a6b18c6735357b8eb7f407347e84aea09beb184961cb83543d46e' instructions='17'
cmd=xray info=image.exposed_ports list='80/tcp'
cmd=xray state=image.api.inspection.done
cmd=xray state=image.data.inspection.start
cmd=xray info=image.data.inspection.save.image.start
time="2024-06-25T11:43:29+08:00" level=error msg="dockerutil.SaveImage: dclient.ExportImage() error = API error (500): invalid repository name (9c367186df9a6b18c6735357b8eb7f407347e84aea09beb184961cb83543d46e), cannot specify 64-byte hexadecimal strings"
time="2024-06-25T11:43:29+08:00" level=fatal msg="slim: failure" error="API error (500): invalid repository name (9c367186df9a6b18c6735357b8eb7f407347e84aea09beb184961cb83543d46e), cannot specify 64-byte hexadecimal strings" stack="goroutine 1 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x64\ngithub.com/mintoolkit/mint/pkg/util/errutil.FailOn({0x1042f2718, 0x140005d5fc8})\n\tgithub.com/mintoolkit/mint/pkg/util/errutil/errutil.go:32 +0x38\ngithub.com/mintoolkit/mint/pkg/app/master/command/xray.OnCommand(0x140005beaa8, 0x1400037a000, 0x140005beac8, {0x16dd5ad1b, 0x5}, 0x1, {0x0, 0x0}, {0x0, 0x0}, ...)\n\tgithub.com/mintoolkit/mint/pkg/app/master/command/xray/handler.go:375 +0x220c\ngithub.com/mintoolkit/mint/pkg/app/master/command/xray.glob..func1(0x140004d8740)\n\tgithub.com/mintoolkit/mint/pkg/app/master/command/xray/cli.go:340 +0x1754\ngithub.com/urfave/cli/v2.(*Command).Run(0x105574480, 0x140004d8740, {0x14000400960, 0x2, 0x2})\n\tgithub.com/urfave/cli/[email protected]/command.go:279 +0x754\ngithub.com/urfave/cli/v2.(*Command).Run(0x140004fc160, 0x140004d8140, {0x140001161b0, 0x3, 0x3})\n\tgithub.com/urfave/cli/[email protected]/command.go:272 +0x964\ngithub.com/urfave/cli/v2.(*App).RunContext(0x140004e4200, {0x104314b48?, 0x1055f6240}, {0x140001161b0, 0x3, 0x3})\n\tgithub.com/urfave/cli/[email protected]/app.go:337 +0x534\ngithub.com/urfave/cli/v2.(*App).Run(...)\n\tgithub.com/urfave/cli/[email protected]/app.go:311\ngithub.com/mintoolkit/mint/pkg/app/master.Run()\n\tgithub.com/mintoolkit/mint/pkg/app/master/app.go:15 +0x4c\nmain.main()\n\tgithub.com/mintoolkit/mint/cmd/mint/main.go:15 +0x194\n" version="darwin/arm64|Aurora|1.41.4|b59472b8cf16b26575df9960d8ede8f6aa4fb6a9|2024-06-23_08:28:41AM"
gnought:~/work/bin/dist_mac_m1 ?1 %

[gnought]

The error comes from https://github.com/mintoolkit/mint/blob/b59472b8cf16b26575df9960d8ede8f6aa4fb6a9/pkg/docker/dockerutil/dockerutil.go#L313 It looks using github.com/fsouza/go-dockerclient client to contact docker api, and the error comes from upstream. Is it a time to switch to official docker client, https://pkg.go.dev/github.com/docker/docker/client#readme-go-client-for-the-docker-engine-api ?

[kcq]

thanks for checking @gnought ! that's odd that the fix got lost... need to restore it

[kcq]

@gnought try this new version

[gnought]

hi @kcq no more 500 API error, but a new error occurs "invalid tar header"

gnought:~/tmp/dist_mac_m1 √ % ./slim --version
mint version darwin/arm64|Aurora|1.41.5|4cc2b185c9cdcd6d0586246e21d8aecc5d847feb|2024-07-01_04:36:47AM
gnought:~/tmp/dist_mac_m1 √ % ./slim x nginx:latest --debug --verbose
cmd=xray state=started
cmd=xray info=cmd.input.params add-image-config='false' rm-file-artifacts='false' target='nginx:latest' add-image-manifest='false'
cmd=xray state=image.api.inspection.start
cmd=xray info=image id='sha256:1aaa8180df68200fd41f9066cf62155e3b71183c04b2895a7388d5fd84ef3c8b' size.bytes='67669989' size.human='68 MB' architecture='arm64'
cmd=xray info=image.stack name='nginx:latest' id='sha256:1aaa8180df68200fd41f9066cf62155e3b71183c04b2895a7388d5fd84ef3c8b' instructions='17' message='see report file for details' index='0'
cmd=xray info=image.exposed_ports list='80/tcp'
cmd=xray state=image.api.inspection.done
cmd=xray state=image.data.inspection.start
cmd=xray info=image.data.inspection.save.image.start
cmd=xray info=image.data.inspection.save.image.end
cmd=xray info=image.data.inspection.process.image.start
time="2024-07-03T01:08:22+08:00" level=error msg="layerFromStream: error reading layer(3239ea84a00d153b5b8f81b47548275701d325537ee211a15719a09589e90ec0) - archive/tar: invalid tar header"
time="2024-07-03T01:08:22+08:00" level=error msg="dockerimage.LoadPackage: error reading oci layer from archive(/Users/gnought/tmp/dist_mac_m1/.mint-state/images/1aaa8180df68200fd41f9066cf62155e3b71183c04b2895a7388d5fd84ef3c8b/image/1aaa8180df68200fd41f9066cf62155e3b71183c04b2895a7388d5fd84ef3c8b.tar/blobs/sha256/3239ea84a00d153b5b8f81b47548275701d325537ee211a15719a09589e90ec0) - archive/tar: invalid tar header"
time="2024-07-03T01:08:22+08:00" level=fatal msg="slim: failure" error="archive/tar: invalid tar header" stack="goroutine 1 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x64\ngithub.com/mintoolkit/mint/pkg/util/errutil.FailOn({0x1023e9df8, 0x103650430})\n\tgithub.com/mintoolkit/mint/pkg/util/errutil/errutil.go:32 +0x38\ngithub.com/mintoolkit/mint/pkg/app/master/command/xray.OnCommand(0x140004d0aa8, 0x140001a3860, 0x140004d0ac8, {0x16fc62c39, 0xc}, 0x1, {0x0, 0x0}, {0x0, 0x0}, ...)\n\tgithub.com/mintoolkit/mint/pkg/app/master/command/xray/handler.go:412 +0x2398\ngithub.com/mintoolkit/mint/pkg/app/master/command/xray.glob..func1(0x14000614780)\n\tgithub.com/mintoolkit/mint/pkg/app/master/command/xray/cli.go:340 +0x1754\ngithub.com/urfave/cli/v2.(*Command).Run(0x1036704a0, 0x14000614780, {0x140006147c0, 0x4, 0x4})\n\tgithub.com/urfave/cli/[email protected]/command.go:279 +0x754\ngithub.com/urfave/cli/v2.(*Command).Run(0x14000626160, 0x14000614140, {0x140001d2000, 0x5, 0x5})\n\tgithub.com/urfave/cli/[email protected]/command.go:272 +0x964\ngithub.com/urfave/cli/v2.(*App).RunContext(0x1400061c200, {0x10240cbe8?, 0x1036f2280}, {0x140001d2000, 0x5, 0x5})\n\tgithub.com/urfave/cli/[email protected]/app.go:337 +0x534\ngithub.com/urfave/cli/v2.(*App).Run(...)\n\tgithub.com/urfave/cli/[email protected]/app.go:311\ngithub.com/mintoolkit/mint/pkg/app/master.Run()\n\tgithub.com/mintoolkit/mint/pkg/app/master/app.go:15 +0x4c\nmain.main()\n\tgithub.com/mintoolkit/mint/cmd/mint/main.go:15 +0x194\n" version="darwin/arm64|Aurora|1.41.5|4cc2b185c9cdcd6d0586246e21d8aecc5d847feb|2024-07-01_04:36:47AM"
gnought:~/tmp/dist_mac_m1 ?1 % ./mint x nginx:latest --debug --verbose
cmd=xray state=started
cmd=xray info=cmd.input.params target='nginx:latest' add-image-manifest='false' add-image-config='false' rm-file-artifacts='false'
cmd=xray state=image.api.inspection.start
cmd=xray info=image id='sha256:1aaa8180df68200fd41f9066cf62155e3b71183c04b2895a7388d5fd84ef3c8b' size.bytes='67669989' size.human='68 MB' architecture='arm64'
cmd=xray info=image.stack index='0' name='nginx:latest' id='sha256:1aaa8180df68200fd41f9066cf62155e3b71183c04b2895a7388d5fd84ef3c8b' instructions='17' message='see report file for details'
cmd=xray info=image.exposed_ports list='80/tcp'
cmd=xray state=image.api.inspection.done
cmd=xray state=image.data.inspection.start
cmd=xray info=image.data.inspection.process.image.start
time="2024-07-03T01:08:34+08:00" level=error msg="layerFromStream: error reading layer(3239ea84a00d153b5b8f81b47548275701d325537ee211a15719a09589e90ec0) - archive/tar: invalid tar header"
time="2024-07-03T01:08:34+08:00" level=error msg="dockerimage.LoadPackage: error reading oci layer from archive(/Users/gnought/tmp/dist_mac_m1/.mint-state/images/1aaa8180df68200fd41f9066cf62155e3b71183c04b2895a7388d5fd84ef3c8b/image/1aaa8180df68200fd41f9066cf62155e3b71183c04b2895a7388d5fd84ef3c8b.tar/blobs/sha256/3239ea84a00d153b5b8f81b47548275701d325537ee211a15719a09589e90ec0) - archive/tar: invalid tar header"
time="2024-07-03T01:08:34+08:00" level=fatal msg="slim: failure" error="archive/tar: invalid tar header" stack="goroutine 1 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x64\ngithub.com/mintoolkit/mint/pkg/util/errutil.FailOn({0x103109df8, 0x104370430})\n\tgithub.com/mintoolkit/mint/pkg/util/errutil/errutil.go:32 +0x38\ngithub.com/mintoolkit/mint/pkg/app/master/command/xray.OnCommand(0x14000698aa8, 0x14000720460, 0x14000698ac8, {0x16ef42c39, 0xc}, 0x1, {0x0, 0x0}, {0x0, 0x0}, ...)\n\tgithub.com/mintoolkit/mint/pkg/app/master/command/xray/handler.go:412 +0x2398\ngithub.com/mintoolkit/mint/pkg/app/master/command/xray.glob..func1(0x140000a25c0)\n\tgithub.com/mintoolkit/mint/pkg/app/master/command/xray/cli.go:340 +0x1754\ngithub.com/urfave/cli/v2.(*Command).Run(0x1043904a0, 0x140000a25c0, {0x140000a2600, 0x4, 0x4})\n\tgithub.com/urfave/cli/[email protected]/command.go:279 +0x754\ngithub.com/urfave/cli/v2.(*Command).Run(0x140002adce0, 0x1400014fa80, {0x14000152000, 0x5, 0x5})\n\tgithub.com/urfave/cli/[email protected]/command.go:272 +0x964\ngithub.com/urfave/cli/v2.(*App).RunContext(0x14000522200, {0x10312cbe8?, 0x104412280}, {0x14000152000, 0x5, 0x5})\n\tgithub.com/urfave/cli/[email protected]/app.go:337 +0x534\ngithub.com/urfave/cli/v2.(*App).Run(...)\n\tgithub.com/urfave/cli/[email protected]/app.go:311\ngithub.com/mintoolkit/mint/pkg/app/master.Run()\n\tgithub.com/mintoolkit/mint/pkg/app/master/app.go:15 +0x4c\nmain.main()\n\tgithub.com/mintoolkit/mint/cmd/mint/main.go:15 +0x194\n" version="darwin/arm64|Aurora|1.41.5|4cc2b185c9cdcd6d0586246e21d8aecc5d847feb|2024-07-01_04:36:47AM"
gnought:~/tmp/dist_mac_m1 ?1 %

[kcq]

it's possible that Orbstack's docker save API call is different... need to repro the steps manually to confirm

[rfilgueiras]

I can reproduce it when trying to x-ray:

❯ slim --version
slim version linux/amd64|Transformer|1.40.11|1b271555882eacdfb4e6598d6d0552e9b9b1449b|2024-02-02_01:36:22PM
❯ slim xray nvcr.io/nvidia/clara/bionemo-framework:1.8 --debug --verbose
cmd=xray state=started
cmd=xray info=params target='nvcr.io/nvidia/clara/bionemo-framework:1.8' add-image-manifest='false' add-image-config='false' rm-file-artifacts='false' 
cmd=xray state=image.api.inspection.start
cmd=xray info=image size.human='13 GB' architecture='amd64' id='sha256:57f09db9857da64e3054622608ff03284eafae68b46031d4c080d1c662c3fe99' size.bytes='13386288121' 
cmd=xray info=image.stack index='0' name='nvcr.io/nvidia/clara/bionemo-framework:1.8' id='sha256:57f09db9857da64e3054622608ff03284eafae68b46031d4c080d1c662c3fe99' instructions='198' message='see report file for details' 
cmd=xray info=image.exposed_ports list='8888/tcp,6006/tcp' 
cmd=xray state=image.api.inspection.done
cmd=xray state=image.data.inspection.start
cmd=xray info=image.data.inspection.process.image.start
time="2024-09-09T10:19:27+02:00" level=error msg="layerFromStream: error reading layer(0dd472f69204c68b47e676a9c50febca0a431a40420fc94e1a01a7af84e5f770) - archive/tar: invalid tar header"
time="2024-09-09T10:19:27+02:00" level=error msg="dockerimage.LoadPackage: error reading oci layer from archive(/tmp/slim-state/.slim-state/images/57f09db9857da64e3054622608ff03284eafae68b46031d4c080d1c662c3fe99/image/57f09db9857da64e3054622608ff03284eafae68b46031d4c080d1c662c3fe99.tar/blobs/sha256/0dd472f69204c68b47e676a9c50febca0a431a40420fc94e1a01a7af84e5f770) - archive/tar: invalid tar header"
time="2024-09-09T10:19:27+02:00" level=fatal msg="slim: failure" error="archive/tar: invalid tar header" stack="goroutine 1 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x5e\ngithub.com/slimtoolkit/slim/pkg/util/errutil.FailOn({0x2329020, 0x338eb30})\n\tgithub.com/slimtoolkit/slim/pkg/util/errutil/errutil.go:32 +0x4b\ngithub.com/slimtoolkit/slim/pkg/app/master/command/xray.OnCommand(0xc000476a98, 0xc00055c090, 0xc000476ab8, {_, _}, _, {_, _}, {0x0, 0x0}, ...)\n\tgithub.com/slimtoolkit/slim/pkg/app/master/command/xray/handler.go:411 +0x2dbf\ngithub.com/slimtoolkit/slim/pkg/app/master/command/xray.glob..func1(0xc000518700)\n\tgithub.com/slimtoolkit/slim/pkg/app/master/command/xray/cli.go:337 +0x1d98\ngithub.com/urfave/cli/v2.(*Command).Run(0x33a8f00, 0xc000518700, {0xc000518740, 0x4, 0x4})\n\tgithub.com/urfave/cli/[email protected]/command.go:279 +0x9dd\ngithub.com/urfave/cli/v2.(*Command).Run(0xc00055a160, 0xc000518140, {0xc000138000, 0x5, 0x5})\n\tgithub.com/urfave/cli/[email protected]/command.go:272 +0xc2e\ngithub.com/urfave/cli/v2.(*App).RunContext(0xc00052c200, {0x23470c8?, 0x341e0a0}, {0xc000138000, 0x5, 0x5})\n\tgithub.com/urfave/cli/[email protected]/app.go:337 +0x5db\ngithub.com/urfave/cli/v2.(*App).Run(...)\n\tgithub.com/urfave/cli/[email protected]/app.go:311\ngithub.com/slimtoolkit/slim/pkg/app/master.Run()\n\tgithub.com/slimtoolkit/slim/pkg/app/master/app.go:15 +0x45\nmain.main()\n\tgithub.com/slimtoolkit/slim/cmd/slim/main.go:15 +0x187\n" version="linux/amd64|Transformer|1.40.11|1b271555882eacdfb4e6598d6d0552e9b9b1449b|2024-02-02_01:36:22PM"

[kcq]

@rfilgueiras thanks for sharing your report! I'll use nvcr.io/nvidia/clara/bionemo-framework:1.8 to repro. Couldn't repro the original condition with OrbStack, so this ticket got stuck and hopefully now with the accessible repro image it'll be unstuck :-)

[igloo15]

$ slim --version
slim version linux/amd64|Transformer|1.40.11|1b271555882eacdfb4e6598d6d0552e9b9b1449b|2024-02-02_01:36:22PM

$ slim --verbose xray nginx
cmd=xray state=started
cmd=xray info=params target='nginx' add-image-manifest='false' add-image-config='false' rm-file-artifacts='false'
cmd=xray state=image.api.inspection.start
time="2024-12-09T12:41:43-05:00" level=info msg="inspecting 'fat' image metadata..." app=slim cmd=xray
cmd=xray info=image architecture='amd64' id='sha256:28402db69fec7c17e179ea87882667f1e054391138f77ffaf0c3eb388efc3ffb' size.bytes='72950530' size.human='73 MB'
time="2024-12-09T12:41:43-05:00" level=info msg="processing 'fat' image info..." app=slim cmd=xray
cmd=xray info=image.stack index='0' name='nginx:1.27' id='sha256:28402db69fec7c17e179ea87882667f1e054391138f77ffaf0c3eb388efc3ffb' instructions='18' message='see report file for details'
cmd=xray info=image.exposed_ports list='80/tcp'
cmd=xray state=image.api.inspection.done
cmd=xray state=image.data.inspection.start
cmd=xray info=image.data.inspection.save.image.start
time="2024-12-09T12:41:43-05:00" level=error msg="dockerutil.SaveImage: dclient.ExportImage() error = API error (500): invalid repository name (28402db69fec7c17e179ea87882667f1e054391138f77ffaf0c3eb388efc3ffb), cannot specify 64-byte hexadecimal strings"
time="2024-12-09T12:41:43-05:00" level=fatal msg="slim: failure" error="API error (500): invalid repository name (28402db69fec7c17e179ea87882667f1e054391138f77ffaf0c3eb388efc3ffb), cannot specify 64-byte hexadecimal strings" stack="goroutine 1 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x5e\ngithub.com/slimtoolkit/slim/pkg/util/errutil.FailOn({0x23298a0, 0xc0005bb980})\n\tgithub.com/slimtoolkit/slim/pkg/util/errutil/errutil.go:32 +0x4b\ngithub.com/slimtoolkit/slim/pkg/app/master/command/xray.OnCommand(0xc000440a98, 0xc00011a750, 0xc000440ab8, {_, _}, _, {_, _}, {0x0, 0x0}, ...)\n\tgithub.com/slimtoolkit/slim/pkg/app/master/command/xray/handler.go:374 +0x2b85\ngithub.com/slimtoolkit/slim/pkg/app/master/command/xray.glob..func1(0xc0004a66c0)\n\tgithub.com/slimtoolkit/slim/pkg/app/master/command/xray/cli.go:337 +0x1d98\ngithub.com/urfave/cli/v2.(*Command).Run(0x33a8f00, 0xc0004a66c0, {0xc00039c680, 0x2, 0x2})\n\tgithub.com/urfave/cli/[email protected]/command.go:279 +0x9dd\ngithub.com/urfave/cli/v2.(*Command).Run(0xc0004e4160, 0xc0004a6140, {0xc000134040, 0x4, 0x4})\n\tgithub.com/urfave/cli/[email protected]/command.go:272 +0xc2e\ngithub.com/urfave/cli/v2.(*App).RunContext(0xc0004ca200, {0x23470c8?, 0x341e0a0}, {0xc000134040, 0x4, 0x4})\n\tgithub.com/urfave/cli/[email protected]/app.go:337 +0x5db\ngithub.com/urfave/cli/v2.(*App).Run(...)\n\tgithub.com/urfave/cli/[email protected]/app.go:311\ngithub.com/slimtoolkit/slim/pkg/app/master.Run()\n\tgithub.com/slimtoolkit/slim/pkg/app/master/app.go:15 +0x45\nmain.main()\n\tgithub.com/slimtoolkit/slim/cmd/slim/main.go:15 +0x187\n" version="linux/amd64|Transformer|1.40.11|1b271555882eacdfb4e6598d6d0552e9b9b1449b|2024-02-02_01:36:22PM"

I just ran into this same issue with latest version is there a fix or option I can set to get it working? I am not using orbstack just standard Docker. I am using it inside of WSL 2.0 on windows though so I am not sure if that is causing the issue or not.

[kcq]

$ slim --version
slim version linux/amd64|Transformer|1.40.11|1b271555882eacdfb4e6598d6d0552e9b9b1449b|2024-02-02_01:36:22PM

$ slim --verbose xray nginx
cmd=xray state=started
cmd=xray info=params target='nginx' add-image-manifest='false' add-image-config='false' rm-file-artifacts='false'
cmd=xray state=image.api.inspection.start
time="2024-12-09T12:41:43-05:00" level=info msg="inspecting 'fat' image metadata..." app=slim cmd=xray
cmd=xray info=image architecture='amd64' id='sha256:28402db69fec7c17e179ea87882667f1e054391138f77ffaf0c3eb388efc3ffb' size.bytes='72950530' size.human='73 MB'
time="2024-12-09T12:41:43-05:00" level=info msg="processing 'fat' image info..." app=slim cmd=xray
cmd=xray info=image.stack index='0' name='nginx:1.27' id='sha256:28402db69fec7c17e179ea87882667f1e054391138f77ffaf0c3eb388efc3ffb' instructions='18' message='see report file for details'
cmd=xray info=image.exposed_ports list='80/tcp'
cmd=xray state=image.api.inspection.done
cmd=xray state=image.data.inspection.start
cmd=xray info=image.data.inspection.save.image.start
time="2024-12-09T12:41:43-05:00" level=error msg="dockerutil.SaveImage: dclient.ExportImage() error = API error (500): invalid repository name (28402db69fec7c17e179ea87882667f1e054391138f77ffaf0c3eb388efc3ffb), cannot specify 64-byte hexadecimal strings"
time="2024-12-09T12:41:43-05:00" level=fatal msg="slim: failure" error="API error (500): invalid repository name (28402db69fec7c17e179ea87882667f1e054391138f77ffaf0c3eb388efc3ffb), cannot specify 64-byte hexadecimal strings" stack="goroutine 1 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:24 +0x5e\ngithub.com/slimtoolkit/slim/pkg/util/errutil.FailOn({0x23298a0, 0xc0005bb980})\n\tgithub.com/slimtoolkit/slim/pkg/util/errutil/errutil.go:32 +0x4b\ngithub.com/slimtoolkit/slim/pkg/app/master/command/xray.OnCommand(0xc000440a98, 0xc00011a750, 0xc000440ab8, {_, _}, _, {_, _}, {0x0, 0x0}, ...)\n\tgithub.com/slimtoolkit/slim/pkg/app/master/command/xray/handler.go:374 +0x2b85\ngithub.com/slimtoolkit/slim/pkg/app/master/command/xray.glob..func1(0xc0004a66c0)\n\tgithub.com/slimtoolkit/slim/pkg/app/master/command/xray/cli.go:337 +0x1d98\ngithub.com/urfave/cli/v2.(*Command).Run(0x33a8f00, 0xc0004a66c0, {0xc00039c680, 0x2, 0x2})\n\tgithub.com/urfave/cli/[email protected]/command.go:279 +0x9dd\ngithub.com/urfave/cli/v2.(*Command).Run(0xc0004e4160, 0xc0004a6140, {0xc000134040, 0x4, 0x4})\n\tgithub.com/urfave/cli/[email protected]/command.go:272 +0xc2e\ngithub.com/urfave/cli/v2.(*App).RunContext(0xc0004ca200, {0x23470c8?, 0x341e0a0}, {0xc000134040, 0x4, 0x4})\n\tgithub.com/urfave/cli/[email protected]/app.go:337 +0x5db\ngithub.com/urfave/cli/v2.(*App).Run(...)\n\tgithub.com/urfave/cli/[email protected]/app.go:311\ngithub.com/slimtoolkit/slim/pkg/app/master.Run()\n\tgithub.com/slimtoolkit/slim/pkg/app/master/app.go:15 +0x45\nmain.main()\n\tgithub.com/slimtoolkit/slim/cmd/slim/main.go:15 +0x187\n" version="linux/amd64|Transformer|1.40.11|1b271555882eacdfb4e6598d6d0552e9b9b1449b|2024-02-02_01:36:22PM"

I just ran into this same issue with latest version is there a fix or option I can set to get it working? I am not using orbstack just standard Docker. I am using it inside of WSL 2.0 on windows though so I am not sure if that is causing the issue or not.

@igloo15 can you try the latest release and see if you still have the same problem?

[igloo15]

@kcq I got a different error this time related to invalid tar header

$ mint --version
mint version linux/amd64|Aurora|1.41.7|a2a53d2348aeedf0d650d47debf477a2cad47111|2024-09-25_03:07:42PM

 $ mint --verbose xray nginx:1.27
cmd=xray state=started
cmd=xray info=cmd.input.params add-image-config='false' rm-file-artifacts='false' runtime='auto/docker' target='nginx:1.27' add-image-manifest='false'
cmd=xray state=image.api.inspection.start
time="2024-12-10T13:34:23-05:00" level=info msg="inspecting 'fat' image metadata..." app=mint cmd=xray
cmd=xray info=image size.human='73 MB' architecture='amd64' id='sha256:28402db69fec7c17e179ea87882667f1e054391138f77ffaf0c3eb388efc3ffb' size.bytes='72950530'
time="2024-12-10T13:34:23-05:00" level=info msg="processing 'fat' image info..." app=mint cmd=xray
cmd=xray info=image.stack message='see report file for details' index='0' name='nginx:1.27' id='sha256:28402db69fec7c17e179ea87882667f1e054391138f77ffaf0c3eb388efc3ffb' instructions='18'
cmd=xray info=image.exposed_ports list='80/tcp'
cmd=xray state=image.api.inspection.done
cmd=xray state=image.data.inspection.start
cmd=xray info=image.data.inspection.process.image.start
time="2024-12-10T13:34:23-05:00" level=error msg="layerFromStream: error reading layer(11d6fdd0e8a78c038b5c013368f76279d21e5ee239f18a1e20a3593414fa1095) - archive/tar: invalid tar header"
time="2024-12-10T13:34:23-05:00" level=error msg="dockerimage.LoadPackage: error reading oci layer from archive(/tmp/mint-state/.mint-state/images/28402db69fec7c17e179ea87882667f1e054391138f77ffaf0c3eb388efc3ffb/image/28402db69fec7c17e179ea87882667f1e054391138f77ffaf0c3eb388efc3ffb.tar/blobs/sha256/11d6fdd0e8a78c038b5c013368f76279d21e5ee239f18a1e20a3593414fa1095) - archive/tar: invalid tar header"
time="2024-12-10T13:34:23-05:00" level=fatal msg="slim: failure" error="archive/tar: invalid tar header" stack="goroutine 1 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:26 +0x5e\ngithub.com/mintoolkit/mint/pkg/util/errutil.FailOn({0x2f5aa40, 0x4648120})\n\tgithub.com/mintoolkit/mint/pkg/util/errutil/errutil.go:32 +0x4b\ngithub.com/mintoolkit/mint/pkg/app/master/command/xray.OnCommand(0xc0007c58f8, 0xc0005103c0, 0xc0007c53c0, {_, _}, _, {_, _}, {0x0, 0x0}, ...)\n\tgithub.com/mintoolkit/mint/pkg/app/master/command/xray/handler.go:495 +0x3a65\ngithub.com/mintoolkit/mint/pkg/app/master/command/xray.init.func1(0xc000788700)\n\tgithub.com/mintoolkit/mint/pkg/app/master/command/xray/cli.go:343 +0x1d92\ngithub.com/urfave/cli/v2.(*Command).Run(0x46882e0, 0xc000788700, {0xc00051c5e0, 0x2, 0x2})\n\tgithub.com/urfave/cli/[email protected]/command.go:279 +0x7e2\ngithub.com/urfave/cli/v2.(*Command).Run(0xc000790160, 0xc000788140, {0xc0001c80c0, 0x4, 0x4})\n\tgithub.com/urfave/cli/[email protected]/command.go:272 +0xa65\ngithub.com/urfave/cli/v2.(*App).RunContext(0xc000434600, {0x2f85598, 0x4705340}, {0xc0001c80c0, 0x4, 0x4})\n\tgithub.com/urfave/cli/[email protected]/app.go:337 +0x58b\ngithub.com/urfave/cli/v2.(*App).Run(...)\n\tgithub.com/urfave/cli/[email protected]/app.go:311\ngithub.com/mintoolkit/mint/pkg/app/master.Run()\n\tgithub.com/mintoolkit/mint/pkg/app/master/app.go:15 +0x45\nmain.main()\n\tgithub.com/mintoolkit/mint/cmd/mint/main.go:15 +0x187\n" version="linux/amd64|Aurora|1.41.7|a2a53d2348aeedf0d650d47debf477a2cad47111|2024-09-25_03:07:42PM"

[kcq]

@igloo15 thank you for sharing the updated results with the extra context it! i'll help with the repro!

[Shard]

Wanted to chime in to note that I seem to be running into the same issue but on a M4 Macbook Pro using Docker Desktop (Same error with Docker VMM or Apple Virtualisation Framework).

$ Projects mint --version
mint version darwin/arm64|Aurora|1.41.7|a2a53d2348aeedf0d650d47debf477a2cad47111|2024-09-25_03:14:07PM

$  mint --verbose xray nginx:1.27
cmd=xray state=started
cmd=xray info=cmd.input.params runtime='auto/docker' target='nginx:1.27' add-image-manifest='false' add-image-config='false' rm-file-artifacts='false'
cmd=xray state=image.api.inspection.start
time="2024-12-25T11:30:05+10:00" level=info msg="inspecting 'fat' image metadata..." app=mint cmd=xray
cmd=xray info=image id='sha256:90babf6ca20a03b57f1ecabb39163d95842e6c8e010cebca9eb4b6ffa277b955' size.bytes='68524807' size.human='68 MB' architecture='arm64'
time="2024-12-25T11:30:05+10:00" level=info msg="processing 'fat' image info..." app=mint cmd=xray
cmd=xray info=image.stack id='sha256:90babf6ca20a03b57f1ecabb39163d95842e6c8e010cebca9eb4b6ffa277b955' instructions='17' message='see report file for details' index='0' name='nginx:1.27'
cmd=xray info=image.exposed_ports list='80/tcp'
cmd=xray state=image.api.inspection.done
cmd=xray state=image.data.inspection.start
cmd=xray info=image.data.inspection.process.image.start
time="2024-12-25T11:30:05+10:00" level=error msg="layerFromStream: error reading layer(261c6a94b398bd46eee583da22742a4b92e207fa4d8512f3988d873898599b74) - archive/tar: invalid tar header"
time="2024-12-25T11:30:05+10:00" level=error msg="dockerimage.LoadPackage: error reading oci layer from archive(/tmp/mint-state/.mint-state/images/90babf6ca20a03b57f1ecabb39163d95842e6c8e010cebca9eb4b6ffa277b955/image/90babf6ca20a03b57f1ecabb39163d95842e6c8e010cebca9eb4b6ffa277b955.tar/blobs/sha256/261c6a94b398bd46eee583da22742a4b92e207fa4d8512f3988d873898599b74) - archive/tar: invalid tar header"
time="2024-12-25T11:30:05+10:00" level=fatal msg="slim: failure" error="archive/tar: invalid tar header" stack="goroutine 1 [running]:\nruntime/debug.Stack()\n\truntime/debug/stack.go:26 +0x64\ngithub.com/mintoolkit/mint/pkg/util/errutil.FailOn({0x104b97bc8, 0x10616d8e0})\n\tgithub.com/mintoolkit/mint/pkg/util/errutil/errutil.go:32 +0x38\ngithub.com/mintoolkit/mint/pkg/app/master/command/xray.OnCommand(0x140006d7920, 0x14000130640, 0x140006d73e8, {0x16dad754c, 0xa}, 0x1, {0x0, 0x0}, {0x0, 0x0}, ...)\n\tgithub.com/mintoolkit/mint/pkg/app/master/command/xray/handler.go:495 +0x30f8\ngithub.com/mintoolkit/mint/pkg/app/master/command/xray.init.func1(0x140006b4480)\n\tgithub.com/mintoolkit/mint/pkg/app/master/command/xray/cli.go:343 +0x17d4\ngithub.com/urfave/cli/v2.(*Command).Run(0x1061ab980, 0x140006b4480, {0x1400046ab00, 0x2, 0x2})\n\tgithub.com/urfave/cli/[email protected]/command.go:279 +0x600\ngithub.com/urfave/cli/v2.(*Command).Run(0x140006a8160, 0x140001581c0, {0x14000158040, 0x4, 0x4})\n\tgithub.com/urfave/cli/[email protected]/command.go:272 +0x83c\ngithub.com/urfave/cli/v2.(*App).RunContext(0x140006a2000, {0x104bc1ae8, 0x10622ddc0}, {0x14000158040, 0x4, 0x4})\n\tgithub.com/urfave/cli/[email protected]/app.go:337 +0x514\ngithub.com/urfave/cli/v2.(*App).Run(...)\n\tgithub.com/urfave/cli/[email protected]/app.go:311\ngithub.com/mintoolkit/mint/pkg/app/master.Run()\n\tgithub.com/mintoolkit/mint/pkg/app/master/app.go:15 +0x4c\nmain.main()\n\tgithub.com/mintoolkit/mint/cmd/mint/main.go:15 +0x194\n" version="darwin/arm64|Aurora|1.41.7|a2a53d2348aeedf0d650d47debf477a2cad47111|2024-09-25_03:14:07PM"

[kcq]

@Shard thank you for the additional context! It would be great to inspect the tar file for the target nginx image the app saved. The tar file itself should still be there after the failure. In your case it's /tmp/mint-state/.mint-state/images/90babf6ca20a03b57f1ecabb39163d95842e6c8e010cebca9eb4b6ffa277b955/image/90babf6ca20a03b57f1ecabb39163d95842e6c8e010cebca9eb4b6ffa277b955.tar. Is it possible for you to share this file? Google drive, dropbox or any other way?

[Shard]

Can do @kcq,

I have uploaded it via fastmail and sent you an email containing the tarfile in question.

Feel free to poke if I can be of any assistance 👍🏻

[kcq]

Can do @kcq,

I have uploaded it via fastmail and sent you an email containing the tarfile in question.

Feel free to poke if I can be of any assistance 👍🏻

Thank you @Shard ! Got it and it confirmed what I was thinking about... The layer decode failure in xray happened because the layer is compressed and it wasn't handled correctly. There must be an extra Docker config option... Maybe it's the Containerd image store. Wasn't getting the same behavior in my M1 test environment.

[Shard]

$ docker info
Client:
 Version:    27.4.0
 Context:    desktop-linux
 Debug Mode: false
 Plugins:
  ai: Ask Gordon - Docker Agent (Docker Inc.)
    Version:  v0.5.1
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.19.2-desktop.1
  compose: Docker Compose (Docker Inc.)
    Version:  v2.31.0-desktop.2
  debug: Get a shell into any image or container (Docker Inc.)
    Version:  0.0.37
  desktop: Docker Desktop commands (Beta) (Docker Inc.)
    Version:  v0.1.0
  dev: Docker Dev Environments (Docker Inc.)
    Version:  v0.1.2
  extension: Manages Docker extensions (Docker Inc.)
    Version:  v0.2.27
  feedback: Provide feedback, right in your terminal! (Docker Inc.)
    Version:  v1.0.5
  init: Creates Docker-related starter files for your project (Docker Inc.)
    Version:  v1.4.0
  sbom: View the packaged-based Software Bill Of Materials (SBOM) for an image (Anchore Inc.)
    Version:  0.6.0
  scout: Docker Scout (Docker Inc.)
    Version:  v1.15.1

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 14
 Server Version: 27.4.0
 Storage Driver: overlayfs
  driver-type: io.containerd.snapshotter.v1
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 CDI spec directories:
  /etc/cdi
  /var/run/cdi
 Swarm: inactive
 Runtimes: runc io.containerd.runc.v2
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 472731909fa34bd7bc9c087e4c27943f9835f111
 runc version: v1.1.13-0-g58aa920
 init version: de40ad0
 Security Options:
  seccomp
   Profile: unconfined
  cgroupns
 Kernel Version: 6.10.14-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: aarch64
 CPUs: 12
 Total Memory: 9.704GiB
 Name: docker-desktop
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 No Proxy: hubproxy.docker.internal
 Labels:
  com.docker.desktop.address=unix:///Users/user/Library/Containers/com.docker.docker/Data/docker-cli.sock
 Experimental: true
 Insecure Registries:
  hubproxy.docker.internal:5555
  127.0.0.0/8
 Live Restore Enabled: false

This is my docker info minus a few paths. My docker daemon config which im pretty sure is what docker desktop provided:

{
 "builder": {
   "gc": {
     "defaultKeepStorage": "20GB",
     "enabled": true
   }
 },
 "experimental": true
}

I couldn't find any option that seems to directly relate to storage compression and I haven't installed any extra extensions myself that im aware of.

[kcq]

@Shard do you have the Use containerd for pulling and storing images option enabled in General settings?

[Shard]

Ah yes I do! Apologies for not noticing that one.

Turned it off and mint seems happy to xray nginx now 🎉 :

$ mint xray nginx:latest
cmd=xray state=started
cmd=xray info=cmd.input.params runtime='auto/docker' target='nginx:latest' add-image-manifest='false' add-image-config='false' rm-file-artifacts='false'
cmd=xray state=image.api.inspection.start
cmd=xray info=image size.bytes='197054395' size.human='197 MB' architecture='arm64' id='sha256:5e0fa356e6f4a996ca452b017c81aca3d087ae38f873ed8314af16126423b21f'
cmd=xray info=image.stack index='0' name='nginx:latest' id='sha256:5e0fa356e6f4a996ca452b017c81aca3d087ae38f873ed8314af16126423b21f' instructions='17' message='see report file for details'
cmd=xray info=image.exposed_ports list='80/tcp'
cmd=xray state=image.api.inspection.done
cmd=xray state=image.data.inspection.start
cmd=xray info=image.data.inspection.save.image.start
cmd=xray info=image.data.inspection.save.image.end
cmd=xray info=image.data.inspection.process.image.start
cmd=xray info=image.data.inspection.process.image.end
cmd=xray state=image.data.inspection.done
...
cmd=xray state=completed
cmd=xray state=done

[kcq]

@Shard good to have a work around :-) Testing the new version where it's not necessary... I'll share an update when it's ready for use

[kcq]

@Shard by the way, it's a good idea to use additional flags for xray... --changes=all is one example, which will get you all data.

[olly-writes-code]

Also ran into this on Mac and can confirm it's resolved by going into the Desktop and switching off "Use containerd for pulling and storing images"