Terraform modules in azure repos

[AurimasNav]

Can't figure out a way to make it work for my terraform when I have terraform modules in Azure Repos

  :host          => "dev.azure.com"
  :local_address => "redacted"
  :local_port    => 45288
  :path          => "/<org>/<project>/_git/az-application-insights.git/info/refs"
  :port          => 443
  :reason_phrase => "Not Found"
  :remote_ip     => "13.107.42.20"
  :status        => 404
  :status_line   => "HTTP/1.1 404 Not Found\r\n"
Checking if app_insights::git_provider::repo_name/git_repo(758284703d54a9cb9c82fc911cbaea3cba3c7f2d)::1.0.1 1.0.1 needs updating
/home/dependabot/dependabot-script/vendor/ruby/2.7.0/gems/dependabot-common-0.196.0/lib/dependabot/git_metadata_fetcher.rb:58:in `fetch_upload_pack_for': The following git URLs could not be retrieved: https://dev.azure.com/<org>/<project>/_git/az-application-insights (Dependabot::GitDependenciesNotReachable)

I've tried to add

variables:
- group: dependabot
- name: DEPENDABOT_EXTRA_CREDENTIALS
  value: '[{"type":"terraform_registry", "url":"https://dev.azure.com", "token":"$(PAT)"}, {"type":"git", "url":"https://dev.azure.com", "username":"x-access-token", "password":"$(PAT)"}]'

to no avail.

In the browser I get not found error for the path /<org>/<project>/_git/az-application-insights.git/info/refs however it works if I remove the .git part from path.

I've been able to reproduce this in public Azure Devops project: https://dev.azure.com/aurimasnavardauskas/dependabot/_build

[mburumaxwell]

Hi @AurimasNav,

I can see the pipeline and the simple repository. Could you clarify if the section whose update is failing makes use of git submodules? I ask this because I suspect that dependabot does not yet support git submodules inside Azure and I haven't gotten around to add support for it.

If you are convinced that that could not be the issue, could you offer a more complete repository (or set of repositories) that I can test with locally. This should a sample that fully public without any sensitive information that I can use to recreate on my end. The one you provided cannot be used for anything else but logs.

[AurimasNav]

Hi, there is no use of git submodules. These are very simple repos for minimal reproduction scenario, you should be able to see all of them.

There are two ADO projects:

1. terraform project contains:
   • Teraform "module" repository: https://dev.azure.com/aurimasnavardauskas/terraform/_git/tf-modules
2. dependabot project contains:
   • Repo scanned by dependabot, terraform "deployment" code which utilizes the module from repo mentioned above: https://dev.azure.com/aurimasnavardauskas/dependabot/_git/iac-tf
   • The repo where dependabot pipeline is defined: https://dev.azure.com/aurimasnavardauskas/dependabot/_git/dependabot

All of these should be public for read access, if you need I can add contributor permissions for your account.

[mburumaxwell]

@AurimasNav This has taken me longer than I thought to figure out what is happening. I had to clone this over to our own organization to reproduce. At some point dependabot is adding a .git to the URL which Azure does not serve. In the logs you will see the path aurimasnavardauskas/terraform/_git/tf-modules.git/info/refs which if you used on Postman/Insomnia would result in a 404 too. However, if you change the path to aurimasnavardauskas/terraform/_git/tf-modules/info/refs (remove the .git), it returns 200 which is what we expect.

There is no solution I could find for this. Only options that exist; either create a PR fix in the core repo if you understand the terraform logic there, or use a non-git registry. The latter is likely easiest.

[mburumaxwell]

Thanks @AurimasNav for creating a related issue. I will keep this issue open much longer in hopes that we find a solution.

[mburumaxwell]

This is fixed in https://github.com/dependabot/dependabot-core/pull/6320 but it is yet to be merged. You can voice your support and opinion on it, hoping it gets merged.

[mburumaxwell]

@AurimasNav there's still no new tagged release for dependabot since 0.215.0 but the new bits are now available here. https://github.com/dependabot/dependabot-core/pull/6320 was merged earlier today. Could you test using the specific input dockerImageTag: 0.14.2-ci.20?

[AurimasNav]

Hi, seems to be working fine. Thanks!

[mburumaxwell]

Almost 5 months later, I'm glad it is fixed. Will now close the issue