LambdaNETCoreDeploy task fails when run simultaneously multiple time on the same host for different pipelines

[thibautbrard]

Hi!

Context

We start building Azure DevOps pipelines in charge of packaging Lambda function using the LambdaNETCoreDeploy@1 task. Our build servers (static, not ephemeral) host multiple build agents that allows to run multiple pipeline at the same time on the same host.

Issue

Inside the LambdaNETCoreDeploy@1 task, one of the steps is to create a DotNetCliWrapper object, which is among other things in charge of installing (lib/dotNetCliWrapper.ts#L63) or updating (lib/dotNetCliWrapper.ts#L46) the Amazon.Lambda.Tools component.

This is a nice approach when working with an ephemeral server, but starts to become a nightmare in our case when multiple pipeline are triggering the same command almost at the same time on the same host. The dotnet tool install -g Amazon.Lambda.Tools command is not an issue as it is executed almost instantly, but as the tool is already installed, LambdaNETCoreDeploy@1 now tried to update it with dotnet tool update -g Amazon.Lambda.Tools. (lib/dotNetCliWrapper.ts#L76). The update step takes more than few seconds to execute and cannot be run in multiple process at the same time.

lib/dotNetCliWrapper.ts#L79

    public static async buildDotNetCliWrapper(cwd: string, env: any, dotnetCliPath: string): Promise<DotNetCliWrapper> {
        const wrapper = new DotNetCliWrapper(cwd, env, dotnetCliPath)

        tl.debug(tl.loc('InstallingOrUpdatingLambdaTools'))
        if (!(await wrapper.installGlobalTools())) {
            // if checking for lambda tools fails and installing them fails, we are probably on the
            // wrong instance type because we were unable to install. This is fine, we might be able to
            // use the old tools
            tl.error(
                'Unable to install global Amazon.Lambda.Tools! The old package based version of Amazon.Lambda.Tools ' +
                    'is now deprecated. Newer .NET core versions will need to use a newer hosted agent and the ' +
                    "global tools (which this task auto installs). Refer to Microsoft's guide for the correct hosted " +
                    'agent for which hosted agent you need to use newer .NET Core versions:' +
                    'https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted'
            )
        }

        return wrapper
    }

Task logs

Pipeline are run at the same time

Host-A/Agent-CI-1 LambdaNETCoreDeploy@1 logs (Every thing is right there):

Processing Lambda project at D:\APPS\Agent-CI-1\wip\218\s\Brand.FirstProject
Reading existing aws-lambda-tools-defaults.json
Configuring region for task
...received instance identity document from metadata
...configured to use region eu-west-1, from EC2 instance metadata.
[command]"C:\Program Files\dotnet\dotnet.exe" tool install -g Amazon.Lambda.Tools
Tool 'amazon.lambda.tools' is already installed.

[command]"C:\Program Files\dotnet\dotnet.exe" tool update -g Amazon.Lambda.Tools
Tool 'amazon.lambda.tools' was reinstalled with the latest stable version (version '5.1.2').

Host-A/Agent-CI-2 LambdaNETCoreDeploy@1 logs (Error at update):

Processing Lambda project at D:\APPS\Agent-CI-2\wip\207\s\Brand.SecondProject
Reading existing aws-lambda-tools-defaults.json
Configuring region for task
...received instance identity document from metadata
...configured to use region eu-west-1, from EC2 instance metadata.
[command]"C:\Program Files\dotnet\dotnet.exe" tool install -g Amazon.Lambda.Tools
Failed to create shell shim for tool 'amazon.lambda.tools': Command 'dotnet-lambda' conflicts with an existing command from another tool.
Tool 'amazon.lambda.tools' failed to install.

[command]"C:\Program Files\dotnet\dotnet.exe" tool update -g Amazon.Lambda.Tools
Tool 'amazon.lambda.tools' failed to update due to the following:
Failed to create shell shim for tool 'amazon.lambda.tools': Command 'dotnet-lambda' conflicts with an existing command from another tool.
Tool 'amazon.lambda.tools' failed to install.

##[error]Unable to install global Amazon.Lambda.Tools! The old package based version of Amazon.Lambda.Tools is now deprecated. Newer .NET core versions will need to use a newer hosted agent and the global tools (which this task auto installs). Refer to Microsoft's guide for the correct hosted agent for which hosted agent you need to use newer .NET Core versions:https://docs.microsoft.com/en-us/azure/devops/pipelines/agents/hosted

Task template

- task: amazonwebservices.aws-vsts-tools.LambdaNETCoreDeploy.LambdaNETCoreDeploy@1
  displayName: 'Package .NET Core to Lambda: ${{parameters.projectName}}'
  inputs:
    packageOnly: true
    packageOutputFile: '$(build.artifactstagingdirectory)\$(build.BuildNumber)\${{parameters.packageName}}'
    lambdaProjectPath: ${{parameters.lambdaProjectPath}}

How could we prevent that?

I wish we could have a new optional input parameter in LambdaNETCoreDeploy@1 task that would prevent trying to reinstall or update the Amazon.Lambda.Tools component.

preventInstallOrUpdate: true

In that case, I understand that keeping this tool up to date on our build hosts would be our own responsibility. But a new task AWS tool installer could later be created to handle the tool installation/update in the same way as many third-party components are doing. For backward compatibility, without this option the task should behave as now.

Thanks for you help

[bryceitoc9]

Hi @thibautbrard and apologies for the delay. We'll take a look into this on our end.