Credentials are not passed along with REST backend

[schemen]

Describe the bug

Using the rest backend, with the --private-repos option enabled, it seems that k8up is not correctly passing the credentials.

Additional context

I'm using the --private-repos, meaning every user has it's own URL. It would be great if k8up appends the username to the URL itself but that might be a feature request. It could be handled within this bug report though as it seems to affect a similar thing.

https://github.com/restic/rest-server

Logs & config

Without repo url:

E0315 09:35:26.125655       1 logging.go:127] wrestic/RepoInit/restic "msg"="Fatal: create repository at rest:https://restic.example.com:8888/ failed: Fatal: server response unexpected: 401 Unauthorized (401)" "error"="error during command"  
E0315 09:35:26.127959       1 logging.go:127] wrestic/RepoInit/restic "msg"="" "error"="error during command"  
E0315 09:35:26.128732       1 main.go:82] wrestic "msg"="failed to inialise the repository" "error"="cmd.Wait() err: 1"  

With repo url:

E0315 09:40:29.530192       1 logging.go:127] wrestic/RepoInit/restic "msg"="Fatal: create repository at rest:https://restic.example.com:8888/k8up_example_project/ failed: Fatal: server response unexpected: 401 Unauthorized (401)" "error"="error during command"  
E0315 09:40:29.532682       1 logging.go:127] wrestic/RepoInit/restic "msg"="" "error"="error during command"  
E0315 09:40:29.536866       1 main.go:82] wrestic "msg"="failed to inialise the repository" "error"="cmd.Wait() err: 1"  

Passing the entire url with basic auth data works:

I0315 09:41:50.887671       1 logging.go:118] wrestic/RepoInit/restic "level"=0 "msg"="created restic repository aa6023aff6 at rest:https://k8up_example_project:[email protected]:8888/k8up_example_project"  
I0315 09:41:50.887748       1 logging.go:118] wrestic/RepoInit/restic "level"=0 "msg"=""  
I0315 09:41:50.887774       1 logging.go:118] wrestic/RepoInit/restic "level"=0 "msg"="Please note that knowledge of your password is required to access"  
I0315 09:41:50.887798       1 logging.go:118] wrestic/RepoInit/restic "level"=0 "msg"="the repository. Losing your password means that your data is"  
I0315 09:41:50.887817       1 logging.go:118] wrestic/RepoInit/restic "level"=0 "msg"="irrecoverably lost."  
I0315 09:41:50.895233       1 snapshots.go:39] wrestic/snapshots "level"=0 "msg"="getting list of snapshots"  
I0315 09:41:51.723765       1 pod_list.go:50] wrestic/k8sClient "level"=0 "msg"="listing all pods"  "annotation"="k8up.syn.tools/backupcommand" "namespace"="example_project"
I0315 09:41:51.782518       1 main.go:177] wrestic "level"=0 "msg"="all pod commands have finished successfully"  

Backup config:

apiVersion: backup.appuio.ch/v1alpha1
kind: Backup
metadata:
  name: backup
spec:
  keepJobs: 4
  backend:
    repoPasswordSecretRef:
      name: backup-credentials
      key: encryption-key
    rest:
      url: https://k8up_example_project:[email protected]:8888/k8up_example_project
      userSecretRef:
        name: backup-credentials
        key: username
      passwordSecretReg:
        name: backup-credentials
        key: password

Expected behavior

The password and user from the secret are actually used to connect to the restic URL. I only want to define the actual URL of the restic instance.

To Reproduce

Steps to reproduce the behavior:

1. Specs

apiVersion: backup.appuio.ch/v1alpha1
kind: Backup
metadata:
  name: backup
spec:
  keepJobs: 4
  backend:
    repoPasswordSecretRef:
      name: backup-credentials
      key: encryption-key
    rest:
      url: https://restic.example.com:8888/k8up_example_project
      userSecretRef:
        name: backup-credentials
        key: username
      passwordSecretReg:
        name: backup-credentials
        key: password

2. Commands

kubectl apply -f backup.yaml

Environment (please complete the following information):

• Image Version: v1.0.3
• K8s Version: v1.20.0+k3s2
• K8s Distribution: K3s

[ccremer]

When the REST backend is configured, K8up should create the wrestic pod with USER and PASSWORD environment variables. Can you confirm that those were passed along to the pod with the expected values?

[ccremer]

Closing due to no activity. Please respond if there's still an issue

[pczerkas]

Still an issue:

Schedule:

`backend:

repoPasswordSecretRef:
  name: backup-repo-secret
  key: password
rest:
  # URL of the Rest server instance (include scheme like https:// on your own)
  url: http://restic.int.pczerkas.info:8000/restic
  # Kubernetes secret reference containing the basic auth user
  userSecretRef:
    name: backup-credentials-secret
    key: username
  # Kubernetes secret reference containing the basic auth password
  passwordSecretReg:
    name: backup-credentials-secret
    key: password`

Pod logs:

1.6617079835027344e+09 INFO k8up Starting k8up… {"version": "2.4.0", "date": "2022-07-15T12:46:47Z", "commit": "37fc6af9487af5485d15c8b7608f31627419c1db", "go_os": "linux", "go_arch": "amd64", "go_version": "go1.18.3", "uid": 65532, "gid": 0} 1.6617079835033963e+09 INFO k8up.restic initializing 1.6617079835034695e+09 INFO k8up.restic setting up a signal handler 1.6617079835036757e+09 INFO k8up.restic.restic using the following restic options {"options": [""]} 1.661707983503756e+09 INFO k8up.restic.restic.RepoInit.command restic command {"path": "/usr/local/bin/restic", "args": ["init", "--option", ""]} 1.661707983503798e+09 INFO k8up.restic.restic.RepoInit.command Defining RESTIC_PROGRESS_FPS {"frequency": 0.016666666666666666} 1.6617079835273046e+09 INFO k8up.restic.restic.RepoInit.restic.stderr Fatal: create repository at rest:http://restic.int.pczerkas.info:8000/restic/ failed: Fatal: server response unexpected: 401 Unauthorized (401) 1.6617079835274544e+09 INFO k8up.restic.restic.RepoInit.restic.stderr 2022/08/28 17:33:03 unable to start k8up: failed to initialise the restic repository: cmd.Wait() err: 1 `

USER and PASSWORD environment variables are set in pod:

[Kidswiss]

It looks like the credentials for the rest connection don't need to be passed as env vars at all: https://restic.readthedocs.io/en/stable/030_preparing_a_new_repo.html#rest-server

@pczerkas could you please test if it works with url: http://username:[email protected]:8000/restic?

If so we'd have to adjust the logic a bit and add it from the secrets to the URL.

[JcGKitten]

Hey @Kidswiss, I tested it with the hardcoded user and password in the url and then I didn't get the unauthorized error anymore. So yes this needs fixing. Thanks.

[JcGKitten]

In k8up/api/v1/backend.go would it need a change like

func (in *RestServerSpec) String() string {
        protocol, url, _ := strings.Cut(in.URL, "://")
	return fmt.Sprintf("rest:%s://%s:%s@%s", protocol, in.UserSecretRef, in.PasswordSecretReg, url)
}

? But I'm not really familiar with go and programming for k8s. I guess it needs more to hide the password in the resource descriptions etc.