ModuleFast fails in an Azure DevOps Release Pipeline

[raandree]

Problem description

Building a project with ~80 dependencies fail with weird error messages when using ModuleFast. The errors are like:

[pre-build] Starting bootstrap process.
Both ModuleFast and PSResourceGet is configured or/and passed as parameter.
PowerShell 7.2 or higher being used, prefer ModuleFast over PSResourceGet.
ModuleFast is configured to use latest released version.
Install-ModuleFast : Cannot convert value to type System.String.
At C:\Agent\_work\1\s\Resolve-Dependency.ps1:871 char:39
+ …     $moduleFastPlan | Install-ModuleFast @installModuleFastParameters
+                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidResult: (:) [Install-ModuleFast], RuntimeException
+ FullyQualifiedErrorId : JobStateFailed,Install-ModuleFast
##[error]PowerShell exited with code '1'.

or

Install-ModuleFast : Exception calling ".ctor" with "2" argument(s): "The HTTP/3 server reset the stream. HTTP/3 error code 'H3_NO_ERROR' (0x100). (HttpProtocolError)"
At C:\Agent\_work\1\s\Resolve-Dependency.ps1:871 char:39
+ …     $moduleFastPlan | Install-ModuleFast @installModuleFastParameters
+                         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidResult: (:) [Install-ModuleFast], MethodInvocationException
+ FullyQualifiedErrorId : JobStateFailed,Install-ModuleFast
##[error]PowerShell exited with code '1'.

Verbose logs

NA

How to reproduce

Build Microsoft365DscWorkshop in an Azure pipeline.

Expected behavior

Some error handling should be added. I have noticed that the issue disappears almost completely after retrying.

Current behavior

Ignores errors in ModuleFast and continues without having all the dependencies downloaded.

Suggested solution

Add error handling / retry behaviour.

Operating system the target node is running

Azure Pipelines

PowerShell version and build the target node is running

PowerShell 7 on Azure Pipelines

Module version used

0.118.2

[johlju]

When this happens we need to find out if there is an issue with ModuleFast's backend.

Either: a) Try using -Debug and see if it outputs what module it is having an issue with. b) Or, debug the script and get the content of $moduleFastPlan from the line $moduleFastPlan | Install-ModuleFast @installModuleFastParameters and then run Install-ModuleFast separately using the same parameters plus -Debug.

[raandree]

It seems to be an issue with the backend. It is not a particular module causing trouble, it rather seems to be related to the number of dependencies. I am observing this issue only with projects having 50+ dependencies.

I have solved it like that:

if ($moduleFastPlan)
{
    # Clear all modules in plan from the current session so they can be fetched again.
    try
    {
        $moduleFastPlan.Name | Get-Module | Remove-Module -Force
        $moduleFastPlan | Install-ModuleFast @installModuleFastParameters
    }
    catch
    {
        Write-Warning -Message 'ModuleFast could not save one or more dependencies. Retrying...'
        try
        {
            $moduleFastPlan.Name | Get-Module | Remove-Module -Force
            $moduleFastPlan | Install-ModuleFast @installModuleFastParameters
        }
        catch
        {
            Write-Error 'ModuleFast could not save one or more dependencies even after a retry.'
        }
    }
}

Is it worth a PR?

[johlju]

I rather see an issue is created in the repo https://github.com/JustinGrote/ModuleFast. An issue that reproduces the error using Install-ModuleFast. Because this does not look like a Sampler issue. We should first see if this can be resolved by ModuleFast. 🤔 Maybe the retry can happen in ModuleFast if that is the only solution.

[raandree]

We just had a discussion about this with @justingrote. The job of ModuleFast is to be super fast. Any retries could slow down the flow. For Sampler, the retries wouldn't make much difference. @johlju, what do you think? Is a PR worth trying?

[JustinGrote]

We just had a discussion about this with @justingrote. The job of ModuleFast is to be super fast. Any retries could slow down the flow. For Sampler, the retries wouldn't make much difference. @johlju, what do you think? Is a PR worth trying?

To clarify, its design is to fail fast with no retries, because it is idempotent and will just pick up where it left off, so you can just re-run it in a try catch loop for your own recovery if needed if for some reason it's flaky due to network issues or otherwise.

https://github.com/JustinGrote/ModuleFast/issues/102#issuecomment-2581527726

[raandree]

Thanks, @JustinGrote, will create a PR for Sampler then.