Setup script doesn't work on IPv6-only machine

[gbirke]

trafficstars

Environmental Info: K3s Version: k3s version v1.24.3+k3s1 (990ba0e8) go version go1.18.1

Node(s) CPU architecture, OS, and Version: Linux control-1 5.15.0-43-generic #46-Ubuntu SMP Tue Jul 12 10:30:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Cluster Configuration: 3 servers, 2 agents

Describe the bug: The download of files from github.com fails, starting with the hash file.

Steps To Reproduce:

• Get a virtual machine without access to ipv4 (e.g. Hetzner Cloud server)
• run curl -sfL https://get.k3s.io | sh -s -

Expected behavior: The install script works or at least gives me a hint that it doesn't work on ipv6-only machines

Actual behavior: After figuring out the version to download, the script stops, without downloading.

[INFO]  Finding release for channel stable
curl: (7) Couldn't connect to server
[INFO]  Using v1.24.3+k3s1 as release
[INFO]  Downloading hash https://github.com/k3s-io/k3s/releases/download/v1.24.3+k3s1/sha256sum-amd64.txt

The order of messages is also confusing, the message curl: (7) Couldn't connect to server comes before the Downloading hash XXX message, making it look as if a previous download failed and the hash download succeeded.

Additional context / logs: The problem here is that GitHub does not serve ipv6 with their github.com address. See this discussion: https://github.com/orgs/community/discussions/10539

raw.githubusercontent.com seems to have at least AAAA records and they are pingable

[brandond]

The problem here is that GitHub does not serve ipv6 with their github.com address.

I'm not sure this is something we can fix, but I guess we could document it. Since you're the first to complain about it, I suspect not many folks have pure ipv6 nodes.

[gbirke]

I agree that you can't do much about what GitHub does and does not (except rage in discussions and forums).

I updated the description with the actual log output, so you can see what you could improve. As you can see, the curl output shows before the download and the script does not output "Download failed". I don't know if that has something to do with the shell. Ubuntu 22.04 LTS uses the dash for the /bin/sh, this is the version info I get from apt: 0.5.11+git20210903+057cd650a4ed-3build1

Some documentation would also be nice.

As a bad workaround, you could go to the releases page on github, copy the commit id of the newest release and do export INSTALL_K3S_COMMIT=<NEWEST_COMMIT_ID> before running the install command. I just did that with success. Maybe that's also something for the documentation.

[brandond]

Commit installs come from S3, which does support ipv6.

[soletan]

I'm not sure this is something we can fix, but I guess we could document it. Since you're the first to complain about it, I suspect not many folks have pure ipv6 nodes.

Ever heard of gitlab.com? ;) This is just another issue proving to me that moving our projects there was a really good idea.

Everyone's been talking for years about IPv4 namespace being exhausted, about how it's time to move to IPv6 eventually for all of its benefits. And github.com owned by one of the major tech companies (which is running a major cloud service consuming lots of IPv4 addresses for sure) is failing to assign an IPv6 address to the sun of a universe of projects relying on it. Seriously?

Actually, using public IPv4 addresses has become a matter of costs nowadays. Hetzner for example is charging extra per server which is definitely sth. I'd like to prevent when setting up a cluster of such servers.

[brandond]

We're not planning on changing to a different project host. Do you have anything reasonable to suggest for users that want to install on ipv6 only nodes, or did you just want to soapbox for a moment?

[soletan]

I was just opposing to the fact that there is nothing you could do about it. Using multiple remote repositories is a feature of git that's available, too. IMHO, doing so would be solution-oriented. IoT is one of the major forces behind the growing demand for a viable public network based on IPv6. If a contemporary IPv6-only IoT device isn't capable of reaching out to a product like k3s for installation, I'd prefer having solutions at hand for that. Since I'm no k8s expert at all (thus being attracted by k3s' offer for a simple start with k8s) I don't have a solution myself other than setting up reverse proxies to overcome such shortcomings on behalf of github/Microsoft. But most simple solution seems to be a second remote repository hosted by a different git provider.

Sorry for ranting about github as I don't get it how a company like Microsoft is failing to offer IPv6 connectivity for it. I didn't expect this to offend you personally. Sorry for that.

UPDATE: Not to forget, I was also trying to provide a perspective in opposition to your suspicion, that this issue won't be quite that popular. I bet to differ.

[dereknola]

Closing this as GH is now rolling out IPv6 endpoints, see latest updates on discussion linked.

[gchazot]

Just sharing my experience with IPv6-only node install. TLDR: It's not for tomorrow.

• As mentioned above, release hashes is retrieved by the install script from GitHub, which fails because github.com doesn't have IPv6. This can be worked-around by using INSTALL_K3S_COMMIT=abce... env var to bypass the hash finding.
• But then, as mentionned above, the script tries to download the packages from AWS S3 (k3s-ci-builds.s3.amazonaws.com) which unlike suggested above, does NOT support IPv6. But this can be worked-around by modifying script to use the S3 dual-stack endpoint (k3s-ci-builds.s3.dualstack.us-east-1.amazonaws.com).

It seems that with these 2 work-arounds, the script finally passes. It installs and starts k3s successfully.

BUT then, k3s never fully configures, because any pod it tries to start fails downloading the Docker image, simply because docker.io doesn't support IPv6 either.

To sum up, IPv6-only is not for tomorrow out of the box. Too many cloud services out there still don't support it. Here we've identified GitHub, AWS and DockerHub. It's probably only the tip of the iceberg but that's enough for me to give up now and keep paying for IPv4 addresses a while longer.