Error refreshing state

[aellwein]

Hi, i am trying out terraform-backend-git v0.1.4 on a M1 Mac.

For the backend, i've created a script on PATHcalled tfbg with following content:

#!/bin/sh
ssh-agent  # this is probably not needed
terraform-backend-git -l git terraform "$@"

In a terraform folder in my repo, i've created a terraform-backend-git.hcl file with following content:

git.repository = "ssh://CEPBEPOK/~alex/git/infra_state.git"
git.ref = "main"
git.state = "/tfstate.json"

Upon using tfbg init, i get a git_http_backend_auto.tf file created:

terraform {
	backend "http" {
		address = "http://localhost:6061/?type=git&repository=ssh://CEPBEPOK/~alex/git/infra_state.git&ref=main&state=/tfstate.json"
		lock_address = "http://localhost:6061/?type=git&repository=ssh://CEPBEPOK/~alex/git/infra_state.git&ref=main&state=/tfstate.json"
		unlock_address = "http://localhost:6061/?type=git&repository=ssh://CEPBEPOK/~alex/git/infra_state.git&ref=main&state=/tfstate.json"
		skip_cert_verification = false
		username = ""
		password = ""
	}
}

But the command execution leaves me with following:

[terraform-backend-git]: Using config file: /Users/alex/git/infra/terraform/terraform-backend-git.hcl
[terraform-backend-git]: WARNING: HTTP basic auth is disabled, please specify TF_BACKEND_GIT_HTTP_USERNAME and TF_BACKEND_GIT_HTTP_PASSWORD
[terraform-backend-git]: listen on 127.0.0.1:6061

Initializing the backend...
[terraform-backend-git]: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
[terraform-backend-git]: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
[terraform-backend-git]: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error refreshing state: Failed to get state: GET http://localhost:6061/?type=git&repository=ssh://CEPBEPOK/~alex/git/infra_state.git&ref=main&state=/tfstate.json giving up after 3 attempt(s)
Error: exit status 1

Looks like something is going wrong on the SSH level. Cloning the infra_state.git repo from the command line works just fine (BTW it's a bare repository).

P.S.: Maybe worth mentioning: the CEPBEPOK hostname is not directly resolvable; it's defined in some ~/.ssh/config file of mine (it's a server located on my local network). As being said, the git clone works without any issues.

[aellwein]

After some tweaks on mentionedtfbg script and founding out that the default configuration of Mac's ssh-agent is pretty weird, i've ended up with this script:

#!/bin/sh
set -eu

ssh-add --apple-use-keychain ~/.ssh/id_ecdsa
export SSH_PRIVATE_KEY=/Users/alex/.ssh/id_ecdsa

echo "Using private key: $SSH_PRIVATE_KEY"
echo "Auth socket: $SSH_AUTH_SOCK"
terraform-backend-git -l git terraform "$@"

but this gives another error output:

tfbg init -reconfigure
Identity added: /Users/alex/.ssh/id_ecdsa (alex@xyz)
Using private key: /Users/alex/.ssh/id_ecdsa
Auth socket: /var/folders/40/mjxwzzd163b2h6jthg5hh9lw0000gn/T//ssh-a5J6lJHKktCT/agent.41225
[terraform-backend-git]: Using config file: /Users/alex/git/infra/terraform/terraform-backend-git.hcl
[terraform-backend-git]: WARNING: HTTP basic auth is disabled, please specify TF_BACKEND_GIT_HTTP_USERNAME and TF_BACKEND_GIT_HTTP_PASSWORD
[terraform-backend-git]: WARNING: Access Logs enabled
[terraform-backend-git]: listen on 127.0.0.1:6061

Initializing the backend...

Successfully configured the backend "http"! Terraform will automatically
use this backend unless the backend configuration changes.
[terraform-backend-git]: repository not found
127.0.0.1 - - [10/Apr/2023:18:10:08 +0200] "GET /?type=git&repository=ssh://CEPBEPOK/~alex/git/infra_state.git&ref=main&state=./tfstate.json HTTP/1.1" 500 27
[terraform-backend-git]: repository not found
127.0.0.1 - - [10/Apr/2023:18:10:10 +0200] "GET /?type=git&repository=ssh://CEPBEPOK/~alex/git/infra_state.git&ref=main&state=./tfstate.json HTTP/1.1" 500 27
[terraform-backend-git]: repository not found
127.0.0.1 - - [10/Apr/2023:18:10:13 +0200] "GET /?type=git&repository=ssh://CEPBEPOK/~alex/git/infra_state.git&ref=main&state=./tfstate.json HTTP/1.1" 500 27
Error refreshing state: Failed to get state: GET http://localhost:6061/?type=git&repository=ssh://CEPBEPOK/~alex/git/infra_state.git&ref=main&state=./tfstate.json giving up after 3 attempt(s)
Error: exit status 1

[aellwein]

@dee-kryvenko any suggestions here?

[dee-kryvenko]

I am not immediately sure, I have not tried it with self-hosted Git repos yet. Can you try export StrictHostKeyChecking=no on the off chance it has something to do with the host key validation and the error message is off? If that doesn't help, I'm gonna need a bit more details (like - how is your Git server is set up) to try to reproduce it locally, although I am a little swamped at work at the moment so it will have to wait for a bit longer.

[aellwein]

Unfortunately, setting "StrictHostKeyChecking" doesn't change anything. I will try to find some time to debug into it.