[Bug]: GitOps for Azure DevOps repos doesn't work (over HTTP)

[baterja]

Bug Description

Error: performing initial clone: unexpected client error: unexpected requesting "https://dev.azure.com/org/project/_git/repo/git-upload-pack" status code: 400

Essentially we hit https://github.com/go-git/go-git/issues/64. Technically it's not flipt issue per se, but I wanted to report it there for visibility (so it will also show up in go-git issue as mentioned in another repo).

Version Info

1.48.1

Search

• [X] I searched for other open and closed issues before opening this

Steps to Reproduce

docker run -it --rm -p 8080:8080 -p 9000:9000 -e FLIPT_STORAGE_TYPE=git -e FLIPT_STORAGE_GIT_DIRECTORY="src/" -e FLIPT_STORAGE_GIT_REPOSITORY="https://dev.azure.com/org/project/_git/repo" -e FLIPT_STORAGE_GIT_REF=branch_name -e FLIPT_STORAGE_GIT_AUTHENTICATION_BASIC_USERNAME=<username> -e FLIPT_STORAGE_GIT_AUTHENTICATION_BASIC_PASSWORD=<PAT> flipt/flipt:v1.48.1

Expected Behavior

Being able to clone Azure DevOps repositories just like every other git repo.

Additional Context

No response

[GeorgeMac]

Thanks for raising this @baterja 🙏 I see some folks updating from 5.10 to 5.11 and claiming that solves it, but we're on 5.12 (and we were on 5.12 for Flipt version 1.48.1 too). So that is a mystery 🤔 I wonder if they regressed.

[GeorgeMac]

I think I see, go-git doesn't fully support multi_ack and only support v2 cloning, not subsequent fetches. We clone and subsequently (continously) fetch in Flipt. Hmmm pesky.

[GeorgeMac]

I think we also have to manually remove mutli-ack and multi-ack-detailed from the unsupported capabilities list: https://github.com/fluxcd/go-git/blob/9c7d2df0f3b85745bea4d764e51ab9e811bf644d/plumbing/protocol/packp/srvresp.go#L49-L51

In order for the initial clone to work, which is why you see the error immediately. We could definitely remove that first and see what happens (I expect subsequent fetches might be problematic).

A lazy and expensive fix to get Azure unblocked might be to switch to periodic full clones with these capabilities removed from the unsupported list. It would get things unblocked at-least. Maybe if we're brave enough, we could lend some cycles to investigating the work required to add multi-ack support to go-git (I assume this is a very non-trivial exercise).

[baterja]

Today I tried to set up cloning via SSH instead of HTTP, and now I get:

Version: v1.50.0
Commit: dfc6c800c79e53b076f64d3968b289c41f07f778
Build Date: 2024-09-17T20:14:25Z
Go Version: go1.22.7
OS/Arch: linux/amd64

2024-09-20T11:33:46Z    DEBUG   checking for updates
2024-09-20T11:33:47Z    DEBUG   version info    {"current_version": "v1.50.0", "latest_version": "1.50.0"}
You are currently running the latest version of Flipt [v1.50.0]!
2024-09-20T11:33:47Z    DEBUG   local state directory exists    {"path": "/home/flipt/.config/flipt"}
2024-09-20T11:33:47Z    DEBUG   starting telemetry reporter     {"component": "telemetry"}
2024-09-20T11:33:47Z    DEBUG   initialized new state   {"component": "telemetry"}
Error: fetching commit ("<my commit hash>"): object not found

That commit hash exists in the repository (btw. it somehow gets to know that hash - so it was probably able to fetch?) and I was able to clone the repo via SSH locally, so I bet SSH "workaround" also doesn't work with Azure DevOps (I didn't check other sources).

.env file used (anonymized a bit):

FLIPT_LOG_LEVEL=debug
FLIPT_STORAGE_TYPE=git
FLIPT_STORAGE_GIT_DIRECTORY=src/
[email protected]:v3/organization/project/repository
FLIPT_STORAGE_GIT_REF=master
FLIPT_STORAGE_GIT_AUTHENTICATION_SSH_INSECURE_IGNORE_HOST_KEY=true
FLIPT_STORAGE_GIT_AUTHENTICATION_SSH_PASSWORD=mypassword
FLIPT_STORAGE_GIT_AUTHENTICATION_SSH_PRIVATE_KEY_PATH=/var/opt/flipt/id_rsa

docker command:

docker run -it --rm -p 8080:8080 -p 9000:9000 --mount type=bind,source="./id_rsa",target=/var/opt/flipt/id_rsa,readonly --env-file ./flipt_env.env flipt/flipt:latest

[baterja]

It seems that go-git will finally support multi_ack in v6 but its release date is not known: https://github.com/go-git/go-git/discussions/1253#discussioncomment-11695774

[GeorgeMac]

Hey @baterja Thanks for sharing this update with us! Hopefully that gets settled in the not so distant future and we can get Flipt working with Azure then.

[erka]

@baterja As fyi Flipt v2 uses go-git@v6 and it works with Azure and https.