some nydus images are larger than OCI image by about 30%

[changweige]

Displayed by nerdtcl images These images are converted by acceld

# nerdctl images

REPOSITORY                                   TAG                IMAGE ID        CREATED           PLATFORM       SIZE         BLOB SIZE
xxxxxxxxxxxxxxxxxx.xxxxxxxx/rrrrrr/java      latest             f841a2abd042    7 seconds ago     linux/amd64    662.7 MiB    235.9 MiB
xxxxxxxxxxxxxxxxxx.xxxxxxxx/rrrrrr/java      latest-nydus-v5    00da0fb797d0    30 seconds ago    linux/amd64    0.0 B        314.9 MiB
xxxxxxxxxxxxxxxxxx.xxxxxxxx/rrrrrr/java      latest-nydus-v6    af716a2aa975    27 seconds ago    linux/amd64    0.0 B        314.7 MiB
xxxxxxxxxxxxxxxxxx.xxxxxxxx/rrrrrr/python    3.7                44fd0dc34f00    21 minutes ago    linux/amd64    933.2 MiB    331.0 MiB
xxxxxxxxxxxxxxxxxx.xxxxxxxx/rrrrrr/python    3.7-nydus-v5       c718004ac488    15 hours ago      linux/amd64    0.0 B        438.7 MiB
xxxxxxxxxxxxxxxxxx.xxxxxxxx/rrrrrr/python    3.7-nydus-v6       045764b7b03e    21 minutes ago    linux/amd64    0.0 B        438.3 MiB

[hsiangkao]

who cares? If anyone cares, they should use zstd or whatever algorithm instead of lz4.

[changweige]

Customers have to pay for storage space like Aliyun OSS bucket, larger images cost more fees. 30% more fee can hardly be ignored. No matter what compression algorithm is preferred or defaulted, we'd better give customers advice. If we already figure out a better and more suitable compression algorithm, turning to that one might be better.

[hsiangkao]

nope, I don't think there is a silver bullet of this. Zstd actually makes much worse if running with internal network. This topic is somewhat noisy. I think users should use their own strategy rather than we make a decision for them.

[jiangliu]

We should enable support of zstd in v2.2

[imeoer]

We'd better investigate why it's so much bigger. eStargz uses gzip compression and the chunk size defaults to 4MB, the eStargz java image and OCI image are close in size.

[imeoer]

Looks like zstd is the right way, I tested (using acceld) a java image with lz4 and zstd compression:

ociv1 with gzip: ~234MB nydus with zstd: ~232MB nydus with lz4 (default level): ~317MB

[changweige]

Looks like zstd is the right way, I tested (using acceld) a java image with lz4 and zstd compression:

ociv1 with gzip: ~234MB nydus with zstd: ~232MB nydus with lz4 (default level): ~317MB

Good to hear about it. Thanks

[jiangliu]

When it's network bandwidth limited, zstd seems like the right solution.

[jiangliu]

The default compression algorithm has been switched to zstd, could you please try again?