NetApp
Add non-amd64 and install details to BUILD.md
Describe the solution you'd like
The current build instructions seem to work for amd64 only.
It'd be nice to have BUILD.md document steps to build and deploy for another architecture (no changes in code would be necessary), so that those interested in experimenting on other platforms can more easily give it a try.
Of course even better would be to modify the Makefiles and Dockerfiles to make that possible, but that involves comparatively more work and official support for those architectures so I'm instead asking for a basic Doco improvement and not submitting a build-related Bug.
I think the below build procedure works (I haven't tried to install yet), but I don't know if it's strictly necessary or maybe there's a simpler way to get this done. Also, if after tridentctl, trident_operator and container are built, any steps are different from manual deployment x86_64, that could be added as well.
Additional context
Following the process in BUILD.md, I get a30130f9efd9 created. But no matter what I try, I couldn't get it deployed. I tried native build, cross-platform build, Docker and go-based builds, different versions, nothing worked. The problem was that no matter how I build, the container image is always tagged in the amd64 format.
Finally I started looking at Dockerfiles.
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
trident 21.01.0-arm64 a30130f9efd9 19 minutes ago 85.1MB
golang 1.14 c7909254f92e 9 days ago 715MB
alpine/helm latest 584a8902bb56 13 days ago 46MB
quay.io/coreos/flannel v0.13.1-rc2 25dbdd806019 2 weeks ago 63.6MB
k8s.gcr.io/kube-proxy v1.20.2 788e63d07298 5 weeks ago 116MB
k8s.gcr.io/kube-controller-manager v1.20.2 3a1a2b528610 5 weeks ago 107MB
k8s.gcr.io/kube-apiserver v1.20.2 95d99817fc33 5 weeks ago 113MB
k8s.gcr.io/kube-scheduler v1.20.2 60d957e44ec8 5 weeks ago 43.3MB
registry 2 357aab9e21a8 2 months ago 24.5MB
k8s.gcr.io/etcd 3.4.13-0 05b738aa1bc6 5 months ago 312MB
k8s.gcr.io/coredns 1.7.0 db91994f4ee8 8 months ago 42.8MB
k8s.gcr.io/pause 3.2 2a060e2e7101 12 months ago 484kB
gcr.io/distroless/static 480a6179aef97ba11817a86d1c7a0fed1e07e4b3 7b715d870c4e 51 years ago 1.79MB
# My trident container image
$ docker inspect a30130f9efd9 | grep Arch
"Architecture": "amd64",
# Revisit one of few remaining suspects - gcr.io/distroless/static, base image in Trident Dockerfiles.
$ docker inspect 7b715d870c4e | grep Arch
"Architecture": "amd64",
# Hmmm, let's check its issues and bingo:
# https://github.com/GoogleContainerTools/distroless/issues/583
# switch to an ARM64 image
$ docker pull gcr.io/distroless/static@sha256:8ea529b8f58b0a8f8c9197ceec0188a34c00861d3a88d5b0e51bcfc4a7bd2137
# docker inspect f0a9eb72f317 | grep Arch
"Architecture": "arm64",
# modify all 3 Dockerfiles in repo to use it
# try again
$ GOOS=linux GOARCH=arm64 make trident_build
# docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
trident 21.01.0-arm64 08310958adfd 8 seconds ago 85.1MB
golang 1.14 c7909254f92e 9 days ago 715MB
alpine/helm latest 584a8902bb56 13 days ago 46MB
quay.io/coreos/flannel v0.13.1-rc2 25dbdd806019 2 weeks ago 63.6MB
k8s.gcr.io/kube-proxy v1.20.2 788e63d07298 5 weeks ago 116MB
k8s.gcr.io/kube-apiserver v1.20.2 95d99817fc33 5 weeks ago 113MB
k8s.gcr.io/kube-controller-manager v1.20.2 3a1a2b528610 5 weeks ago 107MB
k8s.gcr.io/kube-scheduler v1.20.2 60d957e44ec8 5 weeks ago 43.3MB
registry 2 357aab9e21a8 2 months ago 24.5MB
k8s.gcr.io/etcd 3.4.13-0 05b738aa1bc6 5 months ago 312MB
k8s.gcr.io/coredns 1.7.0 db91994f4ee8 8 months ago 42.8MB
k8s.gcr.io/pause 3.2 2a060e2e7101 12 months ago 484kB
gcr.io/distroless/static <none> f0a9eb72f317 51 years ago 1.79MB
gcr.io/distroless/static 480a6179aef97ba11817a86d1c7a0fed1e07e4b3 7b715d870c4e 51 years ago 1.79MB
# check Trident build from latest build atempt
$ docker inspect 08310958adfd | grep Arch
"Architecture": "arm64",
Anyone wanting to build for a non-amd64 arch could just get the right gcr.io/distroless/static image for their Dockerfile.
I completed the rest, basically there's just other 2 tricky places:
- find amd64 and replace it with arm64 in setup/*.yaml
- remove the entire NeApp autosupport container section in setup/trident-deployment.yaml because it’s a closed source image only released for amd64
More here.
@scaleoutsean, ARM64 support is on the Trident roadmap and will be supported in a future release.
@skuethe you're likely looking for an officially supported version, but if you want to experiment with current version, give it a try... I've been running various releases (from v21.01 to current) on different versions of vanilla K8s for over a year now and didn't notice anything out of ordinary. I use iSCSI in my environment.
thank you for your feedback! We are currently in the process of testing on s390x arch :)
@christianjedroCDT, I somehow got your comment asking about additional details via email (Github email notifications), but I can't see it here. Yes, I documented the details (above) but even even more detail on my blog, but I didn't want to "advertise" it. Now that you ask, for ARM64 the above is likely enough to get everything working. For (Linux on) PowerPC and S390 I had to make a small patch.
@scaleoutsean actually i´ve asked if you have detailed info on how to do it but then found that you´ve already provided the link to your article, which i´ve anyway found before through google.. then deleted the comment once realizing it.
One note to your article - i´ve actually deployed an image that was built with your guide to s390x and it seems to be working fine. Actually waiting to be able to create a backend but it looks promising.
