trigger-autoscaling-ci-job not returning error to ADO / Gitlab

[mvfc]

trafficstars

On azure devops, if I call the Slim CI job with dbtc trigger-autoscaling-ci-job with the --should-poll, when the job finishes it always gives a success on ADO, when it should give an error/failure when the job fails in dbt Cloud.

[dpguthrie]

Thanks for opening this @mvfc. The autoscaling action should always show a successful state regardless of the should_poll argument. However, there should be a dbt Cloud check that's triggered because of the way the payload is set up. Do you see anything like this?

Would you mind showing how you setup your pipeline?

[mvfc]

Yeah, what I'm saying is, its not polling the Error status from the job. If the job fails/is cancelled in dbt Cloud, it still gives success.

It's setup like this:

`

- script: |
    SO=cicd
    pattern=refs/heads/
    pr=$(System.PullRequest.SourceBranch)
    fixed_branch=${pr#$pattern}
    dbtc trigger-autoscaling-ci-job --account-id ${{ parameters.dbt_account_id }} --job-id ${{ parameters.dbt_job_id }} --payload '{"cause": "Kicked off by build: $(Build.BuildNumber),$(Build.SourceVersionMessage)","git_branch":"'"$fixed_branch"'","schema_override":"'"$SO"'","azure_pull_request_id":'"$(System.PullRequest.PullRequestId)"'}' --should-poll
  displayName: Autoscaled Slim CI

`

[ljones9119]

I am having a similar problem. Using Azure DevOps also, my setup is below:

name: Autoscaling CI

pr: [main]

trigger: none

variables:

    DBT_CLOUD_HOST: "emea.dbt.com"
    DBT_CLOUD_ACCOUNT_ID: 79
    JOB_ID: 1333
    # GIT_SHA: $(Build.SourceVersion)
    GIT_SHA: $(system.pullRequest.sourceCommitId)
    PULL_REQUEST_ID: $(System.PullRequest.PullRequestId)

pool:
  vmImage: ubuntu-latest

steps:
- task: UsePythonVersion@0
  inputs:
   versionSpec: '3.9'
- script: |
    pip install dbtc
  displayName: 'Install dependencies'
- script: |
    SO="dbt_cloud_pr_"$(JOB_ID)"_"$(PULL_REQUEST_ID)
    dbtc trigger-autoscaling-ci-job --job-id $(JOB_ID) --payload '{"cause": "Autoscaling Slim CI!","git_sha":"'"$(GIT_SHA)"'","schema_override":"'"$SO"'","azure_pull_request_id":'"$(PULL_REQUEST_ID)"'}' --should-poll
  displayName: Trigger Job
  env:
    DBT_CLOUD_SERVICE_TOKEN: $(DBT_CLOUD_SERVICE_TOKEN)

I've amended your script to use '$(system.pullRequest.sourceCommitId)' as I was getting an error when using '$(Build.SourceVersion)':

The build pipeline succeeds and the CI job gets triggered but the dbt Cloud check isn't appearing on the pull request (only the build pipeline status) so the pull request doesn't show when a job fails.

[dpguthrie]

Thanks for adding this @ljones9119. Let me do a bit more investigation on my side - I'm guessing that there's some configuration with the pipeline or modification to the payload that should also trigger the dbt Cloud check.