rancher-desktop
rancher-desktop copied to clipboard
rancher-sandbox
Build and proxies
Are there any docs on how to use proxies with rancher desktop and the underlying components (lima and nerdctl)? I have seen a couple of issues (#384, #699) around proxies/env variables but I can't find any docs or a clear path forward for setting them and being able to build. I am happy to contribute a README or something after this is sorted.
For bugs, describe what you're seeing I was able to get rancher desktop to launch by setting the http/https env variables, but get issues building images that need to come through a proxy.
During a build, my internal image pulls fine but the external gcr.io one times out. I did try adding build-args with the envs but didn't work (described here https://github.com/moby/buildkit/issues/971)
❯ nerdctl build .
[+] Building 30.1s (4/4) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 32B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 35B 0.0s
=> ERROR [internal] load metadata for gcr.io/distroless/static:nonroot 30.0s
=> [internal] load metadata for internal-reg/golang:1.17.1 2.8s
------
> [internal] load metadata for gcr.io/distroless/static:nonroot:
------
error: failed to solve: failed to solve with frontend dockerfile.v0: failed to create LLB definition: failed to do request: Head https://gcr.io/v2/distroless/static/manifests/nonroot: dial tcp 74.125.142.82:443: i/o timeout
FATA[0030] unrecognized image format
But nerdctl pull works "outside" the VM
❯ nerdctl pull gcr.io/distroless/static:nonroot
gcr.io/distroless/static:nonroot: resolved |++++++++++++++++++++++++++++++++++++++|
index-sha256:bca3c203cdb36f5914ab8568e4c25165643ea9b711b41a8a58b42c80a51ed609: done |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:213a6d5205aa1421bd128b0396232a22fbb4eec4cbe510118f665398248f6d9a: done |++++++++++++++++++++++++++++++++++++++|
config-sha256:bff4de2cb7e1dd0ed9797c6e33688f32f2ff0293ecee6fa069051081710bb61b: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:e8614d09b7bebabd9d8a450f44e88a8807c98a438a2ddd63146865286b132d1b: done |++++++++++++++++++++++++++++++++++++++|
elapsed: 5.0 s total: 787.5 (157.4 KiB/s)
nerdctl pull also works "inside" the VM
❯ LIMA_HOME="$HOME/Library/Application Support/rancher-desktop/lima" ./limactl shell 0
lima-rancher-desktop:~$ nerdctl pull gcr.io/distroless/static:nonroot
gcr.io/distroless/static:nonroot: resolved |++++++++++++++++++++++++++++++++++++++|
index-sha256:bca3c203cdb36f5914ab8568e4c25165643ea9b711b41a8a58b42c80a51ed609: done |++++++++++++++++++++++++++++++++++++++|
manifest-sha256:213a6d5205aa1421bd128b0396232a22fbb4eec4cbe510118f665398248f6d9a: done |++++++++++++++++++++++++++++++++++++++|
config-sha256:bff4de2cb7e1dd0ed9797c6e33688f32f2ff0293ecee6fa069051081710bb61b: done |++++++++++++++++++++++++++++++++++++++|
layer-sha256:e8614d09b7bebabd9d8a450f44e88a8807c98a438a2ddd63146865286b132d1b: done |++++++++++++++++++++++++++++++++++++++|
elapsed: 5.1 s total: 787.5 (154.4 KiB/s)
lima vm has my proxies
❯ LIMA_HOME="$HOME/Library/Application Support/rancher-desktop/lima" ./limactl shell 0
lima-rancher-desktop:~$ env | grep -i proxy | wc -l
8
lima-rancher-desktop:~$ env | grep -i proxy
HTTPS_PROXY=http://my-proxy
no_proxy=my-noproxy
NO_PROXY=my-noproxy
https_proxy=http://my-proxy
http_proxy=http://my-proxy
ftp_proxy=ftp://my-proxy
FTP_PROXY=ftp://my-proxy
HTTP_PROXY=http://my-proxy
To Reproduce Steps to reproduce the behaviour:
- Go to '...'
- Click on '....'
- Scroll down to '....'
Result Try to build an image referencing a public image behind a proxy.
Expected behaviour Public image can be pulled and used as a base image during an image build.
Screenshots If applicable, add screenshots to help explain your problem.
Setup (please complete the following information):
- OS
- macOS Big Sur
- Rancher Desktop version
- 0.6.1
- Kubernetes version
- v1.21.5
Additional context
Happy to contribute time, docs, or code just point me at where to look :).
I saw the VPN tag, maybe that should be added here as well?
Isn't this issue about proxies, which is separate from VPN?
I would agree though that we probably should have a bunch more labels, like area/dns, area/proxy, area/vpn, etc.
Yea it's about proxies but I only use proxies because I am on my company's VPN haha. If it doesn't fit that label, no worries. Just thought I would highlight it.
@rumstead I think the following env is unnecessary.
no_proxy=my-noproxy
NO_PROXY=my-noproxy
Perhaps the following command will work.
nerdctl build \
--build-arg HTTP_PROXY=http://my-proxy \
--build-arg HTTPS_PROXY=http://my-proxy \
--build-arg NO_PROXY="localhost,127.0.0.1" \
.
Sorry, Not related to this, but "my-private-registry.example.com" was not needed for "--build-arg NO_PROXY=~". (So I am deleting it from my comment above.)
The reason I added --build-arg NO_PROXY=~ is because your NO_PROXY / no_proxy" setting is NO_PROXY=my-noproxy`, which is not correct, and I wanted to avoid that.
Can I tell nerdctl only at buildtime if it should use a proxy or not? I didn't find a flag to configure this at every call - so I can use a alias to do the job ...
HTTP-PROXY configuration? I sometimes need the proxy and the next time I don't need the proxy. But I think the Proxy should be configured at the VirtualMachine - or does nerdctl forward the env variables at every call?
CONTAINER-PROXY configuration? If I need the HTTP-Proxy configuration - in my case I can also use our Container proxy. But even I can't find a solution to tell nerdctl/ rancher desktop to replace "registry-1.docker.io" with our Container-Proxy-Server like "container.mycompany.com".
If someone have an idea it would be great! Thanks.
I was able to get around my issue by setting proxies on the docker init.d and restarting the docker service.
lima-rancher-desktop:/$ cat /etc/init.d/docker
#!/sbin/openrc-run
supervisor=supervise-daemon
name="Docker Daemon"
description="Persistent process that manages docker containers"
description_reload="Reload configuration without exiting"
command="${DOCKERD_BINARY:-/usr/bin/dockerd}"
command_args="${DOCKER_OPTS}"
DOCKER_LOGFILE="${DOCKER_LOGFILE:-/var/log/${RC_SVCNAME}.log}"
DOCKER_ERRFILE="${DOCKER_ERRFILE:-${DOCKER_LOGFILE}}"
DOCKER_OUTFILE="${DOCKER_OUTFILE:-${DOCKER_LOGFILE}}"
supervise_daemon_args="--stderr \"${DOCKER_ERRFILE}\" --stdout \"${DOCKER_OUTFILE}\""
export HTTPS_PROXY=http://my-proxy
export no_proxy=my-noproxy
export NO_PROXY=my-noproxy
export https_proxy=http://my-proxy
export http_proxy=http://my-proxy
export ftp_proxy=ftp://my-proxy
export FTP_PROXY=ftp://my-proxy
export HTTP_PROXY=http://my-proxy
extra_started_commands="reload"
rc_ulimit="${DOCKER_ULIMIT:--c unlimited -n 1048576 -u unlimited}"
retry="${DOCKER_RETRY:-TERM/60/KILL/10}"
depend() {
need sysfs cgroups
after iptables ip6tables
}
start_pre() {
checkpath -f -m 0644 -o root:docker "$DOCKER_ERRFILE" "$DOCKER_OUTFILE"
}
reload() {
ebegin "Reloading configuration"
$supervisor $RC_SVCNAME --signal HUP
eend $?
}
@jandubois - I see this persisted even when I close rancher desktop or do a lima shutdown 0. Are these files persisted on the underlying host? Any docs I can read?
A more elegant but less-than-ideal solution...
Using overrides to toggle between using a proxy and not.
Proxies:
provision:
- mode: system
script: |
cat <<EOF > /tmp/proxy.sh
#!/bin/sh
export http_proxy="http://foo.com"
export https_proxy="http://foo.com"
export no_proxy=kubernetes.docker.internal,127.0.0.1,127.0.0.0/8
export ftp_proxy="ftp://foo.com/"
export all_proxy="http://foo.com"
export HTTP_PROXY="http://foo.com"
export HTTPS_PROXY="http://foo.com"
export NO_PROXY=kubernetes.docker.internal,127.0.0.1,127.0.0.0/8
export FTP_PROXY="ftp://foo.com/"
export ALL_PROXY="http://foo.com"
EOF
echo ". /tmp/proxy.sh" | sed -i -e '7r /dev/stdin' /etc/init.d/docker
echo ". /tmp/proxy.sh" | sed -i -e '9r /dev/stdin' /etc/init.d/containerd
No Proxies
provision:
- mode: system
script: |
sed -i 's/. \/tmp\/proxy.sh//' /etc/init.d/docker
sed -i 's/. \/tmp\/proxy.sh//' /etc/init.d/containerd
You have to restart rancher desktop or restart the Lima VM.
A similar approach works with windows as well except the files need to end in .start and be placed under %APPDATA%\rancher-desktop\provisioning
EDIT: Added containerd
How to pass windows env to WSL2 to be used by nerdctl.
PS > $env:WSLENV="HTTP_PROXY:HTTPS_PROXY:NO_PROXY"
PS > rdctl shell env
HTTP_PROXY="http://foo.com"
HTTPS_PROXY="http://foo.com"
NO_PROXY=kubernetes.docker.internal,127.0.0.1,127.0.0.0/8