Azure DevOps Task 'AWS Tools for Windows PowerShell Script' - Ignore PS Module Check

[Chetancarma]

Current situation: Using Azure DevOps on a private Azure DevOps server to connect to AWS resources.

In Azure DevOps pipelines, the AWS Task 'AWS Tools for Windows PowerShell Script' is used to run PowerShell scripts against AWS resources. The scripts are executed on a Self-Hosted Agent (Private VM).

As per documentation 'The module will be automatically installed if needed', however every time this task is executed, it checks if the module is present on the Self-Hosted Agent.

Problematic Scenario: If we have have more than 10 similar tasks on the same pipeline, and the check is made every time, which consumes nearly 1 minute per task only for the checks. In total, this increases the pipeline execution time by 10 minutes.

Question: Is it possible to ignore this check?

Expected Solution: Add an option to ignore the Powershell Module checks in the AWS task.

Thank you

[justinmk3]

The check is here: https://github.com/aws/aws-toolkit-azure-devops/blob/2e99eec9c7f200f7477ca370380bfdb585a1a763/src/tasks/AWSPowerShellModuleScript/RunAWSPowerShellModuleScript.ps1#L39

If that takes 1 minute, that implies that the Get-Module -Name AWSPowerShell -ListAvailable call is slow.

Possible improvements:

• change the code to always try Import-Module -Name AWSPowerShell in a try-catch and only then doing the "check" step above.
• find a faster way to check for a module instead of Get-Module -ListAvailable
  • related: https://github.com/PowerShell/PowerShell/issues/11387

[Chetancarma]

Hello @justinmk3,

Thanks for your reply.

However in this case, we are using the AWS Task 'AWS Tools for Windows PowerShell Script' in an Azure DevOps pipeline.

This Azure DevOps task from AWS "uses cmdlets from the AWS Tools for Windows PowerShell module (AWSPowerShell) module. The module will be automatically installed if needed."

Task reference: https://docs.aws.amazon.com/vsts/latest/userguide/awspowershell-module-script.html

So, the import of AWSPowerShell modules is handled by this Task which simply references our PowerShell scripts. As such, we cannot control which AWSPowerShell modules are being installed or skipped, as it checks for everything by default.

Thank you

[justinmk3]

Are you able to run Get-Module -Name AWSPowerShell -ListAvailable manually and see how long it takes? Checking how long Import-Module -Name AWSPowerShell would also be useful.

[Chetancarma]

Hello @justinmk3,

Kindly note we tried above commands on our VM, and we found the below results:

• "Get-Module -Name AWSPowerShell -ListAvailable" took 0.5 seconds, and

• "Import-Module -Name AWSPowerShell" took 15.3 seconds.

Thank you

[justinmk3]

Great info, thanks!

"Get-Module -Name AWSPowerShell -ListAvailable" took 0.5 seconds, and

"Import-Module -Name AWSPowerShell" took 15.3 seconds.

So Import-Module is the slow part. Reading the AWS Tools for PowerShell,

• AWS.Tools - The modularized version of AWS Tools for PowerShell. ...

• AWSPowerShell.NetCore - The single, large-module version of AWS Tools for PowerShell. All AWS services are supported by this single, large module.

• AWSPowerShell - The legacy Windows-specific, single, large-module version of AWS Tools for PowerShell. All AWS services are supported by this single, large module.

Proposal

Based on the above, perhaps we could improve performance of Import-Module by migrating to AWS.Tools instead of the legacy AWSPowerShell

[Chetancarma]

Hello @justinmk3,

You're right, migrating from the legacy AWSPowerShell to the new AWS.Tools could help gain around 15 seconds.

Thank you :) Feel free to proceed.

However, I think if we could add a checkbox to ignore/skip the PowerShell Module verification (i.e. make the Import-Module part optional), that would instantly help us gain 1 minute per task. This is because we do not need to verify/import the module every time a script is executed when we use this extension.

Appreciate your thoughts on this.

Thank you

[justinmk3]

we do not need to verify/import the module every time a script is executed when we use this extension.

If there's a faster way to check that the module is already imported, we could use that and skip Import-Module.

Adding a "skip" option is (1) a very special-case solution and (2) doesn't save much in terms of implementation cost, so it's unlikely we would take that approach. Instead the code should skip unnecessary things by default.

[Chetancarma]

@justinmk3 Well noted, thank you👍

Please feel free to proceed with your initial proposal, i.e. to migrate towards the use of AWS.Tools.

Thank you

[Breno709]

Same issue here. In my case, it takes 5 MINUTES to import modules. Idk why, but at this moment its been the slowest step in my pipeline.

[magic2]

Same here, 1 min mininum waiting time, AWSPowershell: 4.1.562 is pre-installed on all agents.