ARM64 images not supported on the server

[kyo-takano]

Problem

I was having a problem getting my model to work on replicate.com. Once a prediction was submitted, the status remained "Starting" for hours or even days. After consulting @nickstenning, we found that the server doesn't support docker images build for ARM64, causing exec format error. Building docker images for AMD64 resolved this issue.

Suggestion

Raise an error in cog/pkg/cli/push.go or wherever appropriate if the following two conditions are true: a. the command is cog push r8.im/username/model (pushing to Replicate) b. the builder instances are only for ARM64 or AMD64 is not selectable (e.g., through docker buildx ls)

Removing and/or not compiling linux/arm64 distributions in the first place would also be an option, but cog is not just for pushing a model to Replicate, so probably unreasonable.

[nickstenning]

This has come up for other customers, even those building on M1 Macs: https://discord.com/channels/775512803439280149/1074047343529300131

[nickstenning]

I think what Cog currently does is incorrect. We only cross-build by default if someone is using an M1 Mac, and they have the arm64 build of Cog.

If it's reasonable to assume that most users of Cog are doing cog build and cog push in order to run in an AMD64 production environment -- whether that be Replicate or something else -- we should probably default to attempt a cross-build, and emit a warning if we fall back to a native architecture other than AMD64.

In addition, I think we should probably refuse to Cog push non-AMD64 images to r8.im specifically.

So, in summary:

• cog build (and implied builds, e.g. triggered by cog push) automatically attempt to cross-build to linux/amd64, and fall back to the native architecture if that is not possible, emitting a warning if so
• we provide a --platform flag to force a build to a specific architecture and fail if we are unable to build for that architecture
• we refuse to push any non-amd64 image to r8.im

[whenmoon]

Hi @nickstenning I wonder if you have seen this? Is there a solution? https://discord.com/channels/775512803439280149/1157963178043518996 I am on an M1 mac, cannot get any models I push to start and have been told the error is: exec /sbin/tini: exec format error