kubeapps icon indicating copy to clipboard operation
kubeapps copied to clipboard

Support ARM/multi-arch images for Kubeapps containers

Open obeyler opened this issue 6 years ago • 34 comments

Does a version of Kubeapps already exist for arm processor ? I've done a Kubernetes cluster with raspberry pi, and as I think kubeapps is a wonderfull product I want to put it on my personal cluster :-) Thanks Olivier

obeyler avatar Jan 17 '19 22:01 obeyler

Hey @obeyler, we aren't yet building images for ARM, but given Kubeapps is mostly Go I don't think this would be too difficult. It's a nice idea, I can't say it's high on our priority list, though. If you would like to help port Kubeapps to ARM, that would be really amazing!

prydonius avatar Jan 18 '19 18:01 prydonius

@obeyler I will give it a try!

arnaubennassar avatar Feb 02 '19 23:02 arnaubennassar

@arnaubennassar were you able to give this a try?

arapulido avatar Feb 25 '19 16:02 arapulido

@arapulido I started but then I got some new components for my cluster and I've been focused on changing the network and storage. Once I've the cluster working again I will keep trying.

So far I was trying to build a multi arch, multi stage docker image to be able to build the containers for any architecture and being able to do so entirely with docker. Something similar to this. I tried to build the containers directly on the Raspberry as described in the documentation, but the process didn't finish apparently due to not having enough memory.

My goal is to be able to do some CI/CD magic to keep updated versions for ARM. But pleas keep in mind that I'm learning kubernetes and CI/CD tech in my free time, so I don't expect to have this any time soon.

If you think there is a better approach for this, or better suited for the flow of the project let me know and I will try to do my best to make this land in the future to this repo :D

arnaubennassar avatar Feb 26 '19 12:02 arnaubennassar

@arnaubennassar thanks for the update, that's completely fine and we appreciate the efforts! I don't really have any experience with building multi-arch or ARM containers, so will happily defer to you on that ;). If there is any way we can help with this, though, let us know.

Happy to keep this issue open in the meantime.

prydonius avatar Feb 26 '19 19:02 prydonius

@prydonius @arnaubennassar @obeyler is there any progress here ?

fr34k8 avatar Feb 11 '20 08:02 fr34k8

Hi @fr34k8, not really. As @prydonius pointed out we don't have experience building ARM containers so we are accepting community contributions here :)

andresmgot avatar Feb 11 '20 08:02 andresmgot

Nope sorry, I no longer host an ARM cluster, so I won't work on this

arnaubennassar avatar Feb 11 '20 09:02 arnaubennassar

@andresmgot @arnaubennassar did anybody of you start any code to achieve the goal ?

fr34k8 avatar Feb 11 '20 14:02 fr34k8

not in my side

andresmgot avatar Feb 11 '20 14:02 andresmgot

Not really. If you want to start, this may be helpful: https://docs.docker.com/buildx/working-with-buildx/#build-multi-platform-images There's good chances that the images used in this project will cross compile using buildx.

arnaubennassar avatar Feb 12 '20 22:02 arnaubennassar

hey ! Any news on this ? I host a k3s cluster on some armv8 boards and would love to deploy kubeapps to ease it's management! I can try to work on this using buildx as @arnaubennassar suggested !

gabriel-duque avatar Jul 16 '20 18:07 gabriel-duque

With the new multicluster feature you can deploy kubeapps on top of a regular x86 cluster and manage your k3s arm cluster from there.

batiati avatar Jul 16 '20 21:07 batiati

Thanks for the info, I didn't know I could do that. Still, I would like to deploy kubeapps on the k3s cluster itself :)

gabriel-duque avatar Jul 16 '20 23:07 gabriel-duque

We haven't yet prioritized building arm/multi-arch images, sorry. As @batiati mentioned, the only current way would be to install Kubeapps on a regular cluster supporting multiple clusters (but that work is not yet complete - you can see how it will work in https://www.youtube.com/watch?v=KIoW4zZDtdY ).

absoludity avatar Jul 20 '20 05:07 absoludity

@absoludity thanks for the video. I looked into this and it seems like this would not be a lot of work but sort of a hassle because the deploying kubeapps using helm on arm boards would imply building a bunch of bitnami's Docker images for arm too.

gabriel-duque avatar Jul 20 '20 14:07 gabriel-duque

@zuh0 if I can help in any way, let me know.

fr34k8 avatar Jul 20 '20 21:07 fr34k8

@absoludity thanks for the video. I looked into this and it seems like this would not be a lot of work but sort of a hassle because the deploying kubeapps using helm on arm boards would imply building a bunch of bitnami's Docker images for arm too.

The idea mentioned above was that you wouldn't need to deploy kubeapps on arm boards at all - it'd be running on a different x86 cluster, but able to deploy to other clusters, including your arm one. In the video demo, there are no Kubeapps components installed on the additional cluster. The only requirement is that the additional cluster's k8s api server be setup with oidc pointing at dex running on the initial cluster. But yes, I can't say whether it's worth trying out (and can say multicluster support won't be available for a couple of weeks yet).

absoludity avatar Jul 20 '20 23:07 absoludity

@absoludity I understood what you were saying.

But I would like to avoid setting up a x86 cluster. I am self-hosting my cluster using k3s on some raspberry pi and the whole point was to run everything on armv8.

gabriel-duque avatar Jul 22 '20 21:07 gabriel-duque

@fr34k8 In order for this to be possible, here are the Docker images which would need to be built for armv8:

  • kubeapps/apprepository-controller
  • kubeapps/assetsvc
  • kubeapps/asset-syncer
  • kubeapps/dashboard
  • kubeapps/kubeops
  • kubeapps/tiller-proxy (for people using Helm 2)
  • bitnami/kubectl
  • bitnami/nginx
  • bitnami/oauth2-proxy
  • bitnami/node (used by some of kubeapps Dockerfiles)

I will try to open issues on the bitnami repos to build multi-architecture images. If the maintainers seem positive about this I will try to do this but from what I saw on the kubeapps images, it should be rather straightforward as the Dockerfiles use multi-architecture base images like golang or ubuntu.

The complicated work is going to be having a multi-architecture bitnami/minideb image which is a dependency of bitnami/node and others.

gabriel-duque avatar Jul 22 '20 21:07 gabriel-duque

@absoludity I understood what you were saying.

But I would like to avoid setting up a x86 cluster. I am self-hosting my cluster using k3s on some raspberry pi and the whole point was to run everything on armv8.

Sorry, I understood "this" in

@absoludity thanks for the video. I looked into this and it seems like this would not be a lot of work but sort of a hassle because the deploying kubeapps using helm on arm boards would imply building a bunch of bitnami's Docker images for arm too.

to be referring to the multicluster setup, but I see you meant actually doing the arm setup :)

absoludity avatar Jul 22 '20 23:07 absoludity

I think the ARM(v8) support would be really helpful since the architecture support is growing globally. Our company develops small Kubernetes clusters based on the ARM64 platform (see http://www.edgeclusters.com/ if curious) - and we would really appreciate the possibility to run kubeapps by simply using helm install... I've tried to build it from sources recently, but find it too difficult to use since it depends on too many not-arm-enabled images. I hope it will change soon. Is there any progress?

amoravek avatar Nov 20 '20 20:11 amoravek

hi @amoravek, not that I am aware of. I am not sure if @zuh0 got more information from the bitnami team regarding ARM since they are the ones that at the end of the day, build all the containers, including our dependencies.

andresmgot avatar Nov 23 '20 08:11 andresmgot

If offer my help with building and testing.

amoravek avatar Nov 24 '20 10:11 amoravek

Thanks @amoravek, we are evaluating this and working in a POC. We'll let you know here once there is some progress so you can test it.

andresmgot avatar Nov 24 '20 14:11 andresmgot

Hello again, are there any good news regarding arm64 Kubeapps? ;-)

amoravek avatar Dec 19 '20 00:12 amoravek

The Bitnami team plans to officially support it (for the Bitnami open source catalog, including Kubeapps) but there is no ETA yet.

andresmgot avatar Jan 07 '21 09:01 andresmgot

@andresmgot could you link a reference to that announcement? I'm currently browsing for a quick-fix to run bitnami-docker-owncloud on arm64, and I'm considering resorting to the slightly outdated https://hub.docker.com/r/thenoots/arm-bitnami-minideb (closest I got to finding the corresponding repo for that image was https://github.com/radicand/minideb-extras-base-arm/commit/9ef34b94d57c4b1375a541419d2ff76a4a36ef37).

ddelange avatar Jan 07 '21 20:01 ddelange

It's still early stages so there is no link I can send you yet (that image in the link you posted is not related to Bitnami).

A related question, in which platform are you interested? Raspberries, cloud ARM instances, the new Mac?

andresmgot avatar Jan 08 '21 08:01 andresmgot

I see, all good. Yes the image is of unknown source, so not very keen on using it.

Raspberries, cloud ARM instances, the new Mac?

You picked three very good reasons to prioritise arm bitnami images! For me Raspberry is the one at the moment, but the cheaper AWS ARM instances (and in a few years the ARM Mac...) are definitely on my radar as well!

ddelange avatar Jan 08 '21 09:01 ddelange