Task "Output Variables" are not available to Dependent Stages

[pflugs30]

Summary

When reading the documentation regarding output variables (such as the Elastic Beanstalk Create App Version task), I got the impression the variable would be available as other output variables, such as in this example from the Azure documentation. That documentation shows this example:

stages:
- stage: One
  jobs:
  - job: A
    steps:
    - task: MyTask@1  # this step generates the output variable
      name: ProduceVar  # because we're going to depend on it, we need to name the step
- stage: Two
  - job: B
    variables:
      # map the output variable from A into this job
      varFromA: $[ stageDependencies.One.A.outputs['ProduceVar.MyVar'] ]
    steps:
    - script: echo $(varFromA) # this step uses the mapped-in variable

In this example, the output variable from stage One, task ProduceVar is available to the second stage through the stageDependencies object. I assumed the AWS Toolkit output variables would behave this way, too, given their names as "output variables".

When I ran some tests, however, I found the values passed to the task's output variable was not available to other stages. Here's a simple pipeline I ran to exercise this task:

stages:
- stage: build
  jobs:
  - job: CI_Build
    steps:
    # a few tasks to build the zipped artifact
    - task: BeanstalkCreateApplicationVersion@1
      name: createNewEbAppVer
      inputs:
        awsCredentials: 'AWS CI/CD Admin'
        regionName: 'us-east-1'
        applicationName: 'my-api'
        applicationType: 's3'
        deploymentBundleBucket: 'my-bucket'
        deploymentBundleKey: 'build/$(Build.BuildId).zip'
        outputVariable: 'versionLabel'

    - bash: echo "##vso[task.setvariable variable=testStageOutputVar;isOutput=true]this is a stage output var"
      name: setVarTest

    - task: Bash@3
      name: OutputTest
      inputs:
        targetType: 'inline'
        script: |        
          echo versionLabel: $(versionLabel)    # outputs something like 'v1637798886271'
          echo buildStageVar: $(buildStageVar)  # outputs 'this is a stage output var'

- stage: deploy
  dependsOn: build
  variables:
    buildStageVar: $[ stageDependencies.build.CI_Build.outputs['setVarTest.testStageOutputVar'] ]
    versionLabel: $[ stageDependencies.build.CI_Build.outputs['createNewEbAppVer.versionLabel'] ]
  jobs:
  - job: deploy
    steps:
    - task: Bash@3
      name: OutputTest
      inputs:
        targetType: 'inline'
        script: |        
          echo versionLabel: $(versionLabel)    # outputs ''
          echo buildStageVar: $(buildStageVar)  # outputs 'this is a stage output var'

Notice in the OutputTest task, the versionlabel variable outputs an empty string. I wanted to understand why. Upon a closer inspection of the BeanstalkCreateApplicationVersion/TaskOperations code (see this line), I found a call to the azure-pipelines-task-lib/task.js: setVariable function. In all usages of this function in the AWS Toolkit project, the isOutput parameter is not passed, and so it is set to false by default. Thus, the task "output variable" is only exposed to the job and is not available to dependent stages.

Suggestions

I see two possible paths forward:

1. Clarify in the AWS Toolkit for Azure DevOps user guide the true scope of the task output variables. Ensure that future developers know that the output not available to other stages.
2. Modify the various tasks' call to setVariable to pass isOutput = true so the variable will be available as an output to dependent

Workaround

Here's a possible workaround to expose the variable to other stages:

stages:
- stage: build
  jobs:
  - job: CI_Build
    steps:
    # a few tasks to build the zipped artifact
    - task: BeanstalkCreateApplicationVersion@1
      name: createNewEbAppVer
      inputs:
        awsCredentials: 'AWS CI/CD Admin'
        regionName: 'us-east-1'
        applicationName: 'my-api'
        applicationType: 's3'
        deploymentBundleBucket: 'my-bucket'
        deploymentBundleKey: 'build/$(Build.BuildId).zip'
        outputVariable: 'versionLabel'

    - bash: echo "##vso[task.setvariable variable=versionLabel;isOutput=true]$(versionLabel)"
      displayName: 'Set stage output variables'
      name: setOutputs

- stage: deploy
  dependsOn: build
  variables:
    versionLabel: $[ stageDependencies.build.CI_Build.outputs['setOutputs.versionLabel'] ]
  jobs:
  - job: deploy
    steps:
    - bash: echo 'versionLabel: $(versionLabel)'    # outputs something like 'v1637798886271'
      name: VariableTest

In the above example, the setOutputs step loads the local task variable into a stage output variable so it can be referenced in the next stage.