`whalesay` does not support arm64 and should not be used as a workflow example

[erikschul]

Pre-requisites

• [X] I have double-checked my configuration
• [X] I can confirm the issues exists when I tested with :latest
• [ ] I'd like to contribute the fix myself (see contributing guide)

What happened/what you expected to happen?

Tag for arm64 doesn't exist: https://hub.docker.com/r/docker/whalesay/tags

Error (exit code 64): failed to start command: fork/exec /usr/local/bin/cowsay: exec format error

Version

irrelevant

Paste a small workflow that reproduces the issue. We must be able to run the workflow; don't enter a workflows that uses private images.

When following this guide:

https://argoproj.github.io/argo-workflows/quick-start/

which uses

https://raw.githubusercontent.com/argoproj/argo-workflows/master/examples/hello-world.yaml

which uses whalesay

Logs from the workflow controller

irrelevant

Logs from in your workflow's wait container

irrelevant

[terrytangyuan]

I've been running whalesay on arm64 without issues.

[terrytangyuan]

You are using cowsay based on your logs /usr/local/bin/cowsay: exec format error

[erikschul]

@terrytangyuan It works with Docker on MacOS, but not with containerd on Kubernetes. I assume that Docker runs the container with x86 emulation.

The template uses image: docker/whalesay:latest.

A simple alternative is:

spec:
  entrypoint: echotest
  templates:
  - name: echotest
    container:
      image: alpine
      command: ["sh","-c"]
      args: ["echo","hello"]

[terrytangyuan]

Would you like to submit a PR to update the examples?

[erikschul]

It seems whalesay is used in 61 files in examples. A different solution could be to provide an arm64 image somehow (asking Docker?). Or to provide a notice in the docs "We use whalesay everywhere, which doesn't work on arm64."

[erikschul]

But also, I experience this frequently. It typically takes me 5 minutes to realize that the problem is lack of arm64 support, but I'm getting used to it. I assume others will have the same experience, so I guess it's low priority. I'm happy to close the issue. But it is a bit annoying when it happens, especially in a quick-start guide, where you have to figure out how to run the example with a different image.

[agilgur5]

@erikschul as mentioned above, we'll happily take a PR to update the examples to a similar image that is cross-architecture compatible

[terrytangyuan]

We can also use argosay image which is something we can manage https://github.com/argoproj/argo-workflows/tree/master/test/e2e/images/argosay/v2

[erikschul]

Cool. I'll take a look when I'm more familiar with Argo Workflows.

[h0tbird]

We can also use argosay image which is something we can manage https://github.com/argoproj/argo-workflows/tree/master/test/e2e/images/argosay/v2

Despite https://github.com/argoproj/argo-workflows/issues/10435, an argoproj/argosay arm64 image has not been published on Docker Hub.

[agilgur5]

For reference, I threw up a tweet at Docker to try and get their attention about the issues with whalesay, since it affects a lot more than Argo and not even just arm64 either: https://github.com/docker/whalesay/issues/6, https://github.com/docker/whalesay/issues/7.

Despite #10435, an argoproj/argosay arm64 image has not been published on Docker Hub.

Yea there's apparently no CI process for that and it was last manually pushed: https://github.com/argoproj/argo-workflows/pull/11613#issuecomment-1920685874. I'd prefer to remove argosay entirely (as it's no longer needed and would reduce complexity) and rely on whalesay per https://github.com/argoproj/argo-workflows/pull/11613#issuecomment-1957786530, but if Docker doesn't update whalesay, we may have to maintain argosay in the long-term

[agilgur5]

PRs welcome for a replacement with another minimal trusted image that can do a simple echo, such as busybox, alpine, etc.

[erikschul]

What about https://hub.docker.com/_/hello-world

[agilgur5]

Docker Support mentioned that in https://github.com/docker/whalesay/issues/6#issuecomment-2246259247, but as I wrote there, it prints out Docker CLI usage instructions after the "hello", so that would be very confusing to see in an example.

[Joibel]

Another option would be to resurrect argosay.

[agilgur5]

argosay was mentioned above as well, and still has some unresolved problems. But as I wrote above, I'd prefer to remove argosay entirely anyway as I don't believe it should be necessary anymore with Emissary.

Reducing the code surface is definitely preferable, and busybox is more than good enough as a replacement. We already have other third-party images in examples and tests as well (like alpine, influxdb, etc etc)

[Joibel]

Yeah, it's unnecessary, but we could get a cute octopus argonaut to say stuff :smile:

Reducing things is better. Marking as a good first issue. Two PRs could be done:

• Replacement of whalesay and argosay in examples & docs
• Removal of legacy argosay from the codebase

[agilgur5]

Could also split the argosay replacement as a third PR (i.e. one for whalesay one for argosay).

I didn't mark it as a good first issue as whalesay alone is used ~191 times in the codebase (as mentioned in https://github.com/argoproj/argo-workflows/issues/13388#issuecomment-2246275879). It's also not a pure find and replace as you'd also have to change the command, template names + entrypoints, and any results output in docs. The sheer size and some required attention to further consequences felt like a bit much, especially compared to some of the lower quality first contributions we've had (that also don't respond to review comments with some frequency). I only marked it as solution/suggested as such

[agilgur5]

I'm going to tackle this since it is a very confusing issue to new users and has gotten a couple related issues too (on manifest v1, for instance). It also does have a lot of usage as I mentioned above so is not the simplest issue either, and no one's picked it up since it was marked as a good first issue