RS_ETCD_CLIENT_CERT environment variable is not set

[rushins]

i am getting following error using using Rshifter App on kubernetes. my kubernets cluster is core Tectonic method .

Having problems calculating stats, no vanialla keys found: Can't find client cert file: RS_ETCD_CLIENT_CERT environment variable is not set.

What version of Kubernetes (kubectl version) are you using?

1.7.3

What version of etcd (etcdctl --version) are you using?

3.2.15

What version of ReShifter are you using (for rcli: rcli version otherwise at the bottom of the app)?

0.3.21

Development only: what version of Go (go version) and dep (git describe --tags) are you using?

no GO using.

[rushins]

based on the document i have also set the following 3 parameters in my master node where ETCD running.

export RS_ETCD_CLIENT_CERT=/etc/kubernetes/secrets/etcd-client.crt export RS_ETCD_CLIENT_KEY=/etc/kubernetes/secrets/etcd-client.key export RS_ETCD_CA_CERT=/etc/kubernetes/secrets/etcd-client-ca.crt

[mhausenblas]

@rushins thanks for raising this issue. The environment variables RS_ETCD_XXX need to be set on the machine where you run ReShifter, that is, for rcli in the in the terminal session you're launching it and for the app from within the pod it's launching the app.

[rushins]

Hi

Sorry to come back late . I have question. I set the parameters on my kubernetes master host where the ETCD running as "Rkt" container . as mentioned i have tectonic based kubernetes setup which ETCD uses as container but the export parameters all works except RCLI was not there on the master node ? should i install RCLI on master node (my master node is "coreos") also should i install reshifter on my master ? or the RSHIFTER contianer ?

can you clarify my doubt.

thank you again.

[mhausenblas]

Yes, installing rcli on the master node is an option.

[rushins]

ok i copied RCLI as well reshifter binaries in my kubernetes master node (its coreos so i had to do symbolic links to /usr/bin/ ) anyways but after doing i get different errors.

when i try to do backup i get this error from RESHIFTER container web app as well master node executing reshifter backup

Get http://lvpal3040.pal.sap.corp:2379/version: malformed HTTP response "\x15\x03\x01\x00\x02\x02"

---

any idea. ?

[rushins]

just curious to find, is this error due to any persistency stoage of NFS. I have NFS setup with persistence storage but for reshifter container i didnt' configure any NFS

i have this PV in my cluster.

NFS Server: 10.48.226.90 Path: /vol_kubesdn_nfs_0001/vora2/vflow-stdlib

please advise what this error.

[mhausenblas]

Very strange error message there but I can't imagine that a PV has anything to do with it. The only thing that ReShifter does is trying to connect to etcd. You could try if you can manually communicate with etcd using etcdctl, https://github.com/mhausenblas/reshifter/tree/master/testbed has a number of hints there.

[drdivano]

I am getting a similar error on Kubernetes 1.10

export RS_ETCD_CLIENT_CERT=/etc/kubernetes/pki/etcd/peer.crt
export RS_ETCD_CLIENT_KEY=/etc/kubernetes/pki/etcd/peer.key
export RS_ETCD_CA_CERT=/etc/kubernetes/pki/etcd/ca.crt
export RS_ETCD_API_VERSION=v3

./rcli-linux.dms stats
Collecting stats from etcd endpoint http://127.0.0.1:2379
ERRO[0000] Get http://127.0.0.1:2379/version: malformed HTTP response "\x15\x03\x01\x00\x02\x02"

At the same time etcdctl works just fine:

cid=$(docker ps -q -f label=io.kubernetes.container.name=etcd)

docker exec -e ETCDCTL_API=3 $cid etcdctl --key /etc/kubernetes/pki/etcd/peer.key --cert /etc/kubernetes/pki/etcd/peer.crt --cacert /etc/kubernetes/pki/etcd/ca.crt endpoint status

2018-06-03 14:01:55.541895 I | warning: ignoring ServerName for user-provided CA for backwards compatibility is deprecated
127.0.0.1:2379, 8e9e05c52164694d, 3.1.12, 78 MB, true, 78, 15138549

[mhausenblas]

Thanks @drdivano and it seems ReShifter tries to use HTTP on an HTTPS endpoint. Can you run rcli explore please and paste the output here? Also, you could try to overwrite the endpoint as in: rcli stats --endpoint https://127.0.0.1:2379.

[drdivano]

Thanks @mhausenblas ! It works with --endpoint https:

./rcli-linux.dms stats --endpoint https://127.0.0.1:2379
Collecting stats from etcd endpoint https://127.0.0.1:2379
Vanilla Kubernetes [keys:788, size:1430983]

./rcli-linux.dms explore --endpoint https://127.0.0.1:2379
Exploring etcd endpoint https://127.0.0.1:2379
etcd version: 3.1.12
API version: v3
Secure: secure etcd, SSL/TLS configure
Distro: Vanilla Kubernetes

[mhausenblas]

Cool, thanks @drdivano and if you could share the output of ./rcli-linux.dms explore (that is, without the endpoint parameter as well that would be awesome, so I get a chance to understand what goes wrong in the auto-discovery ;)

[drdivano]

./rcli-linux.dms explore
Exploring etcd endpoint http://127.0.0.1:2379
ERRO[0000] Get http://127.0.0.1:2379/version: malformed HTTP response "\x15\x03\x01\x00\x02\x02"

[mhausenblas]

Thanks, yeah, that's what I thought :(