ollama icon indicating copy to clipboard operation
ollama copied to clipboard

0.2.6-rocm and above cannot be pulled with containerd on fedora

Open volatilemolotov opened this issue 1 year ago • 6 comments

What is the issue?

Pulling the image results in

Error: failed to extract layer sha256:00d2c36d84f963d50ac6a568b0be71eea96f3579770ef47c2ac3f94d4d3c346a: exit status 1: unpigz: skipping: <stdin>: corrupted -- crc32 mismatch

This happens for 0.2.6-rocm and further vesions Not sure why it fails and what kind of issue pigz has here. Im using k0s which comes packaged with its own containerd so its not using fedora installed.

Really not sure where to look for a potential cause

OS

Linux

GPU

No response

CPU

Intel, AMD

Ollama version

0.3.0

volatilemolotov avatar Jul 26 '24 09:07 volatilemolotov

fwiw, I'm able to pull and run with Docker Desktop. (confirmed with ollama/ollama:0.2.6-rocm as well as ollama/ollama:rocm) so I think the images we pushed up to Docker Hub are valid.

dhiltgen avatar Jul 26 '24 18:07 dhiltgen

Im also able to pull with docker on my PC.

Need it on k8s and cannot pull it nor understand what is the issue, im here looking for ideas

volatilemolotov avatar Jul 27 '24 19:07 volatilemolotov

Having the same problem here. When I start pulling an image, either via the command line or via Portainer, it works normall at first. Only after the download and extraction finished, the pulling exited with the error: „layers from manifest don't match image configuration“.

The „normal“ :latest image is working fine, just my homeserver cant take ithe load without the GPU.

Are there any alternative downloads available?

Krios0815 avatar Jul 29 '24 00:07 Krios0815

I'm also having this issue on Fedora 40 with docker engine (non-desktop)

rmcmilli avatar Jul 29 '24 20:07 rmcmilli

I'm having the same issue.

anaxios avatar Jul 31 '24 00:07 anaxios

I have the same problem. Normal image works fine, rocm build (layers from manifest don't match image configuration)

rkoopmans avatar Jul 31 '24 07:07 rkoopmans

Seems that the issue is with the package pigz which in my case when detected on system means that containerd will use unpigz to exctract the image. Removing the package and reboot works. In my case pigz was not a dependency of anything to it was easy to remove

volatilemolotov avatar Aug 01 '24 13:08 volatilemolotov

Containerd on start detects if there is unpigz and should it use it

https://github.com/containerd/containerd/issues/3824#issuecomment-553802643

volatilemolotov avatar Aug 01 '24 13:08 volatilemolotov