Publishing many modules with dependencies is difficult due to eventual consistency

[matteo-prosperi]

Steps to reproduce

We publish almost 180 modules (AWS.Tools.*) to PowerShell Gallery in large batches. These modules have dependencies on each other. I wrote a script that looks at RequiredModules, resolves the dependency graph and does Publish-Module in the right order. Even if I only try publishing a module when all its dependencies are already published, I still receives failures because Publish-Module checks for presence on the PSGallery of the dependencies and PSGallery doesn't reliably report that the modules I have just published are actually there. Because of the large number of modules we are publishing, waiting minutes between each Publish-Module invocation is not an option.

Expected behavior

Either PSGallery could guarantee consistency in a specific amount of time for this particular use case (for example wait 5 seconds after Publish-Module succeeds before calling Publish-Module for a dependent module); or there could be a Publish-Module switch to disable dependency checking.

Environment data

> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      6.2.0
PSEdition                      Core
GitCommitId                    6.2.0
OS                             Microsoft Windows 10.0.17763
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

> Get-Module -ListAvailable PowerShellGet,PackageManagement

ModuleType Version    Name                                PSEdition ExportedCommands
---------- -------    ----                                --------- ----------------
Script     1.4.3      PackageManagement                   Desk      {Find-Package, Get-Package, Get-PackageProvider, Get-PackageSource…}
Script     2.2        PowerShellGet                       Desk      {Find-Command, Find-DSCResource, Find-Module, Find-RoleCapability…}

    Directory: C:\Program Files\PowerShell\Modules

ModuleType Version    Name                                PSEdition ExportedCommands
---------- -------    ----                                --------- ----------------
Script     1.4        PackageManagement                   Desk      {Find-Package, Get-Package, Get-PackageProvider, Get-PackageSource…}
Script     2.2        PowerShellGet                       Desk      {Find-Command, Find-DSCResource, Find-Module, Find-RoleCapability…}
Script     2.1.4      PowerShellGet                       Desk      {Find-Command, Find-DSCResource, Find-Module, Find-RoleCapability…}

    Directory: C:\program files\powershell\6\Modules

ModuleType Version    Name                                PSEdition ExportedCommands
---------- -------    ----                                --------- ----------------
Script     1.3.1      PackageManagement                   Desk      {Find-Package, Get-Package, Get-PackageProvider, Get-PackageSource…}
Script     2.1.2      PowerShellGet                       Desk      {Find-Command, Find-DSCResource, Find-Module, Find-RoleCapability…}

> Get-PackageProvider -ListAvailable

Name                     Version          DynamicOptions
----                     -------          --------------
NuGet                    3.0.0.1          Destination, ExcludeVersion, Scope, SkipDependencies, Headers, FilterOnTag, Contains, AllowPrereleaseVersions, ConfigFile, SkipValidate
PowerShellGet            2.2.0.0          PackageManagementProvider, Type, Scope, AllowClobber, SkipPublisherCheck, InstallUpdate, NoPathUpdate, AllowPrereleaseVersions, Filter, Tag, Includes, DscResource, RoleCapability, Command, AcceptLicense, PublishLocation, ScriptSourceLocation, ScriptPublishLocation
PowerShellGet            2.1.4.0
PowerShellGet            2.1.2.0

[SteveL-MSFT]

There will always be a lag between publishing and being available due to validation. Perhaps a -SkipDependencyCheck type of switch for publishing might be ok, but need to think through the ramifications.