PowerShell
[UnixCompleters] Improve start-up speed when enabled
Summary of the new feature/enhancement
The UnixCompleters module should create a cache of completions to be lazy-loaded when PowerShell starts up.
Currently, it would appear the module loads all completions when PowerShell starts - from my understanding. This causes startup to be slow, and this can become irritating. It is reproducible with a minimal $profile that only loads the UnixCompleters module.
The output of PowerShell starting up with the described profile above:
> /opt/microsoft/powershell/pwsh
PowerShell 7.0.3
Copyright (c) Microsoft Corporation. All rights reserved.
https://aka.ms/powershell
Type 'help' to get help.
Loading personal and system profiles took 699ms.
PS /home/dzr/.config/powershell>
As you can see, the startup time of having this module alone can contribute to a long startup time with a proper $profile. Now, this may only affect me, so apologies if this is a faux issue.. I can provide my current $profile if necessary, though.
Proposed technical implementation details (optional)
I would propose that PowerShell stores the completions in a fast-ish cache, that is potentially lazy-loaded when a completion is required, or loaded in parallel during start-up, so that it doesn't slow down PowerShell when starting.
In terms of updating the cache, this could be done with a command, or run occasionally with a scheduled task.
In fact we already spent some time trying to optimise startup time. The remaining issues are probably API limitations with PowerShell itself. See https://github.com/PowerShell/PowerShell/issues/10722.
Basically the only supported way to register a native argument completer in PowerShell today is to use Register-ArgumentCompleter to register a scriptblock.
The UnixCompleters module bypasses the need to create a pipeline and run a cmdlet by instead using reflection to directly access the completer table:
https://github.com/PowerShell/Modules/blob/1db9ddf3241b9b02e4e0b7d8d58a24606f39bdba/Modules/Microsoft.PowerShell.UnixCompleters/Microsoft.PowerShell.UnixCompleters/UtilCompleterInitializer.cs#L38-L53
But this doesn't stop the need for us to create ScriptBlocks for each command. The ScriptBlocks we create are very minimal callbacks back into our .NET code:
https://github.com/PowerShell/Modules/blob/1db9ddf3241b9b02e4e0b7d8d58a24606f39bdba/Modules/Microsoft.PowerShell.UnixCompleters/Microsoft.PowerShell.UnixCompleters/UnixUtilCompletion.cs#L31-L43
But we are forced to create them eagerly, since (1) PowerShell only allows ScriptBlocks to be native argument completers, and (2) there's no way to hook into command invocation, so instead we must register the completer ahead of time (on load).
I suspect we would be able to almost entirely eliminate any startup time issues by getting a direct .NET hook instead, prevent the need to do any reflection or ScriptBlock creation.
Ah, I see now. Interesting, and makes sense.
Direct .NET hook sounds the ideal solution. Would that be both .NET Framework and .NET Core?
Thanks for the quick reply, too!
Would that be both .NET Framework and .NET Core?
Well we could only expect such an API from PS 7.1+, since new APIs wouldn't be backported. Plus our need is only *nix only. So that means .NET Core only (or more accurately, .NET 5+).
When I say .NET hook though, I just mean a hook in PowerShell that allows for a .NET class or delegate to directly implement a completer, rather than needing to execute PowerShell script like with a ScriptBlock.
@rjmholt Maybe worth looking at the internal ScriptBlock.CreateDelayParsedScriptBlock method. I don't recall the exact differences or whether it would help/work in this circumstance but worth investigating.
Also since you know the exact length of the string and the actual building of it isn't too complicated, consider using string.Create (assuming you already have a Core only build).
@SeeminglyScience, this is an interesting idea. After speaking with @rjmholt, there may be some additional ideas, such as invoking bash in server mode that may offer improvements. We will take another look at this in the future after the 1.0 GA release of the module.
After further discussing with @rjmholt we would like to investigate the internal ScriptBlock.CreateDelayParsedScriptBlock as suggested after the initial GA.