[2.2.1] `until_front_of_line` reports null timestamps and continues without waiting

[ghost]

trafficstars

Orb version

2.2.1

What happened

I have this configuration:

      - queue/until_front_of_line:
          only-on-branch: master
          only-on-workflow: orange_build
          block-workflow: true
          time: '10'
          circleci-api-key: CIRCLECI_TOKEN

But when two concurrent orange_build workflows runs the second one reports null timestamp and continues:

This Workflow Timestamp: null
Oldest Workflow Timestamp: null
Front of the line, WooHoo!, Build continuing

Downgrading to 1.8.4 solves the issues. But of course missing the new only-on-workflow parameter.

Expected behavior

Should wait for the previous workflow to finish.

[eddiewebb]

Is this duplicate of #94 ?

[ghost]

The issue sounds familiar. We have block-workflow: true but not sure if that impacts anything. In the comments to #94 it also says

This Workflow Timestamp: "2023-04-27T12:11:43Z"
Oldest Workflow Timestamp: null

Whereas we get:

This Workflow Timestamp: null
Oldest Workflow Timestamp: null

But might still be same underlying issue. So feel free to close if you think it is the same issue.

[ghost]

BTW. I also tried to remove only-on-workflow to make it wait on all workflows on master. But still didn't wait.

[eddiewebb]

Which VCS?

On Wed, May 17, 2023, 7:00 AM Mads Tandrup @.***> wrote:

BTW. I also tried to remove only-on-workflow to make it wait on all workflows on master. But still didn't wait.

— Reply to this email directly, view it on GitHub https://github.com/eddiewebb/circleci-queue/issues/104#issuecomment-1551183268, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABIEWSQQXITLEINWWT35NR3XGSVVRANCNFSM6AAAAAAYDE6OBM . You are receiving this because you commented.Message ID: @.***>

[joaoReigota1]

Hi, Changing to version [2.2.0] seems to work I believe the issue was introduced here https://github.com/eddiewebb/circleci-queue/commit/aba14f1bc0229fa768a5dbac06dd5e4c00bc61b9#diff-51176543868ecfedca657c95003c21b9aa413af677aa2a9f06a5eaa88f3f2ed7L180-L186

[eddiewebb]

Which VCS?

On Wed, May 31, 2023, 10:49 AM João Reigota @.***> wrote:

Hi, Changing to version [2.2.0] seems to work I believe the issue was introduced here aba14f1 #diff-51176543868ecfedca657c95003c21b9aa413af677aa2a9f06a5eaa88f3f2ed7L180-L186 https://github.com/eddiewebb/circleci-queue/commit/aba14f1bc0229fa768a5dbac06dd5e4c00bc61b9#diff-51176543868ecfedca657c95003c21b9aa413af677aa2a9f06a5eaa88f3f2ed7L180-L186

— Reply to this email directly, view it on GitHub https://github.com/eddiewebb/circleci-queue/issues/104#issuecomment-1570383882, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABIEWSR7LF245RTWWXSZKXDXI5K6LANCNFSM6AAAAAAYDE6OBM . You are receiving this because you commented.Message ID: @.***>

[joaoReigota1]

Circle CI,

I think I don't understand the question sorry

[eddiewebb]

Are you using GitHub, bit bucket, or gitlab?

Also this is not an official CircleCI orb.

On Thu, Jun 1, 2023, 4:08 AM João Reigota @.***> wrote:

Circle CI,

I think I don't understand the question sorry

— Reply to this email directly, view it on GitHub https://github.com/eddiewebb/circleci-queue/issues/104#issuecomment-1571566153, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABIEWSRFSRRXED4G5MBAEGDXJBEXTANCNFSM6AAAAAAYDE6OBM . You are receiving this because you commented.Message ID: @.***>

[joaoReigota1]

Using Github

[j2em6h]

I am also running into this issue, so far it seems the null values appear only on pipelines that were trigged via webhook (not a commit or merge).

VCS: GitHub

[eddiewebb]

Thank you J Meredith! That might be the key.

@joaoReigota1 - are you also using webhooks?

On Tue, Jun 6, 2023, 11:36 AM J Meredith @.***> wrote:

I am also running into this issue, so far it seems the null values appear only on pipelines that were trigged via webhook.

VCS: GitHub

— Reply to this email directly, view it on GitHub https://github.com/eddiewebb/circleci-queue/issues/104#issuecomment-1579000811, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABIEWSWQ4VFG2LQVDEUH4ZTXJ5FBHANCNFSM6AAAAAAYDE6OBM . You are receiving this because you commented.Message ID: @.***>

[j2em6h]

Also confirming 2.2.0 does work with my webhook triggered builds.

[ghost]

VCS: GitHub

[ghost]

I can confirm that 2.2.0 works as well for us

[eddiewebb]

Ok, need to update to use new /pipelines/ endpoint to get the created_at time of the pipeline, which will be consistent across all workflows in that pipeline, and across different sources of change (Github, Bitbucket, webtriggers, etc)

[StefanWerW]

Confirming that 2.2.0 is working, 2.2.1 is not

[ConstantinoSchillebeeckx]

I'm seeing similar behavior, curiously enough I only started seeing this behavior sometime after 2023-09-04.

Checking time of workflow: asdf
DEBUG: Making API Call to https://circleci.com/api/v2/workflow/asdf
DEBUG: API Success
Workflow was created at: 2023-09-11T13:34:03Z
Checking time of workflow: fasd
DEBUG: Making API Call to https://circleci.com/api/v2/workflow/fasd
DEBUG: API Success
Workflow was created at: 2023-09-11T13:33:21Z
Checking time of workflow: 123
DEBUG: Making API Call to https://circleci.com/api/v2/workflow/123
DEBUG: API Success
Workflow was created at: 2023-09-11T13:20:48Z
Orb parameter block-workflow is true.
This job will block until no previous workflows have *any* jobs running.
Oldest job: 69447
This Workflow Timestamp: null
Oldest Workflow Timestamp: null
Front of the line, WooHoo!, Build continuing

[alanjaouen]

Confirming that 2.2.0 is working, 2.2.1 is not

same here

[nampas]

Confirming that 2.2.0 is working, 2.2.1 is not

Same here as well. Our builds are triggered from GitHub webhooks.

[renan]

Can confirm the issue. Running v2.2.1 on GitHub webhooks as well.

[KrisTresData]

@eddiewebb Hmm... idk if this is it but here: https://github.com/eddiewebb/circleci-queue/blob/main/src/commands/until_front_of_line.yml#L159

    fetch_active_workflows(){
          cp /tmp/jobstatus.json /tmp/augmented_jobstatus.json
          for workflow in `jq -r ".[] | .workflows.workflow_id //empty" /tmp/augmented_jobstatus.json | uniq`
          do
            echo "Checking time of workflow: ${workflow}"
            workflow_file=/tmp/workflow-${workflow}.json
            if [ ! -z $TESTING_MOCK_WORKFLOW_RESPONSES ] && [ -f $TESTING_MOCK_WORKFLOW_RESPONSES/${workflow}.json ]; then
              echo "Using test mock workflow response"
              cat $TESTING_MOCK_WORKFLOW_RESPONSES/${workflow}.json > ${workflow_file}
            else
              fetch "${CIRCLECI_BASE_URL}/api/v2/workflow/${workflow}" "${workflow_file}"
            fi
            created_at=`jq -r '.created_at' ${workflow_file}`
            echo "Workflow was created at: ${created_at}"
            cat /tmp/augmented_jobstatus.json | jq --arg created_at "${created_at}" --arg workflow "${workflow}" '(.[] | select(.workflows.workflow_id == $workflow) | .workflows) |= . + {created_at:$created_at}' > /tmp/augmented_jobstatus-${workflow}.json
            #DEBUG echo "new augmented_jobstatus:"
            #DEBUG cat /tmp/augmented_jobstatus-${workflow}.json
            mv /tmp/augmented_jobstatus-${workflow}.json /tmp/augmented_jobstatus.json
          done
        }

When you fetch the status of each workflow, at the end you override the file to /tmp/augmented_jobstatus.json, meaning only last iteration of that for loop will have it's data stored. So when you later call: https://github.com/eddiewebb/circleci-queue/blame/main/src/commands/until_front_of_line.yml#L211-L213

load_current_workflow_values(){
     my_commit_time=`jq '.[] | select( .build_num == '"${CIRCLE_BUILD_NUM}"').committer_date' /tmp/augmented_jobstatus.json`
 }

it will not find the data unless your workflow id happens to be last on the list. So you end up with null value.

[peter-evans]

It looks to me like the problem is using committer_date instead of workflows.created_at in the latest merge commit to main. See https://github.com/eddiewebb/circleci-queue/commit/e74ca30d0bc5a82b4d8fea49092f2551e62f698e. There are no tags, so it's not clear, but I think this is the latest version 2.2.1.

Version 2.2.0, which other users have reported to be working, I think is the previous merge commit to main: https://github.com/eddiewebb/circleci-queue/commit/944d963eff09815513c17723c4f073307077ebab

@eddiewebb Perhaps it would be good to roll the changes to until_front_of_line.yml back in commit https://github.com/eddiewebb/circleci-queue/commit/e74ca30d0bc5a82b4d8fea49092f2551e62f698e as a fix.

[peter-evans]

Made a PR for the above mentioned change: https://github.com/eddiewebb/circleci-queue/pull/108

[eddiewebb]

Milestone 3.0 uses the best across sources of change which is to delegate to build-processor's assignment of pipeline.id See #111

[eddiewebb]

ATTENTION If you are having ordering issues on 1.x 2.x releases and are eager to push a fix, please try eddiewebb/queue@dev:499

Comments ---> #112

[eddiewebb]

3.1.0 released and fix