platyPS
platyPS copied to clipboard
PowerShell
Update-MarkdownHelp should support multiple threads to speed up large projects
My project has several hundred cmdlets. I'd like to integrate platyPS into our build workflow to automatically generate stub documentation when there's changes, but with so many cmdlets, it adds 10-15 minutes to the build for Update-MarkdownHelpModule to complete.
Supporting a thread pool versus reading every single file sequentially could greatly improve performance with very large projects.
The main source of slowness in the project is our home-written markdown parser. #112 will be awesome to implement and finally switch to markdig.
I like the idea of using threads, we can apply it to the markdown parsing for short term perf gains.
Reading files in parallel (IO itself) would not really help because:
- reading files is a pretty small part of the program
- if you have HDD, reading files in parallel can actually slow down the overall time, because of additional spinning for seeks.
Great info. I hadn't dug into the implementation so I just assumed it was due to sequential file access. :) Any perf improvements would be much welcome. I'd love to get this integrated into our build workflow and this is the only blocker right now. With 700+ cmdlets it takes about 10 minutes on average to run.
I can't help but wonder in reading this if your module needs a little refactoring. 700 commands is a very large number of commands. The most I can find on my desktop is v2 of the Hyper-V module with 249 exported commands. Is there really no distinguishing element where you could group commands into modules? Or even using submodules would help you.
The biggest I have seen so far was Skype MS modules family, about ~1k. So big modules are a reality for enterprises :)
It's a very large module for a very large product. There's really no practical way to break it up into submodules - it's already some 20 assemblies and a single psd1.
-- Sent from my phone. Expect brevity and typos.
On Dec 14, 2017, at 07:57, Jeff Hicks <[email protected]mailto:[email protected]> wrote:
I can't help but wonder in reading this if your module needs a little refactoring. 700 commands is a very large number of commands. The most I can find on my desktop is v2 of the Hyper-V module with 249 exported commands. Is there really no distinguishing element where you could group commands into modules? Or even using submodules would help you.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/PowerShell/platyPS/issues/315#issuecomment-351752380, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ALGV5GegjdaDtRyVwRRoZGAN4X49tb42ks5tAUVvgaJpZM4P2K3s.
