Mergeable requirement in Azure DevOps only ignores genre "Atlantis Bot/atlantis"

[mhennecke]

Atlantis plan, apply and policy checks are tracked in Azure DevOps via a PR status. An Azure DevOps PR status is built from a genre and a status name.

The atlantis Azure DevOps VCS client publishes the plan, apply and policy check status with the genre "Atlantis Bot/$ATLANTIS_VCS_STATUS_NAME". However, the Azure DevOps VCS client ignores only the hard-coded genre "Atlantis Bot/atlantis" in the check whether a PR is mergeable.

Steps to reproduce:

1. Repo configuration with "apply_requirements": ["mergeable"]
2. non-default vcs-status-name, e.g. "atlantis-global"
3. Azure DevOps repository with a mandatory status check branch policy for "Atlantis Bot/atlantis-global/plan" and/or "Atlantis Bot/atlantis-global/apply", e.g. on main branch
4. Open a PR with main branch as target with a succesful atlantis plan run
5. Trigger atlantis apply

Actual Behaviour: Apply Failed: Pull request must be mergeable before running apply.

Expected Behaviour: Successul atlantis apply

Found in Atlantis version: 0.17.0

I'll try to prepare a fix for this issue.

[boxrick]

Weirdly I get an almost identical problem using Azure DevOps, not sure of the cause. But everything is mergeable and it just doesn't want to merge in...

[pszypowicz]

@boxrick I know it's more than a year, but maybe other users will have that problems too. I've spent more time trying to debug that, than I'm willing to admit ;)

Atlantis evaluates policies using Atlantis ADO user with a PTA. You can simulate this from CLI with:

az devops login #and provide atlantis user PTA 
az repos pr policy list --organization https://dev.azure.com/<organization> --id <pr number>

Investigate it carefully, first look for policies where "isBlocking" equal to "true", then look for "status".

I got different result from policyEvaluation API endpoint depending on the PTA.

In my case the user was lacking organization permission "Reader" in ADO.