nvim-neorocks
[Perf] Parallel `sync` and `update`
I think I have an idea for this:
Option 1
- First, install (shared?) dependencies sequentially
- Then update rocks in parallel
Option 2
Implement a per-package locking mechanism in luarocks
I'm inclined to start with option 1 and refactor in case option 2 ever becomes a reality. We could pester hisham a bit to try and make that work :p (hell, maybe I can try to contribute, seems like a simplish thing to implement).
The processing time of checking shared dependencies is outweighed by the speed gains of parallel updates.
Played around with this a bit more on the parallel-installs branch.
The luarocks --force-lock option is completely broken when doing parallel installs.
luarocks needs either a way to lock installs per directory or an --ignore-lock option.
Added an --ignore-lock flag to my fork of luarocks. Some packages seem to update fine, but others fail with
Warning: Failed searching manifest: Failed loading manifest for https://nvim-neorocks.github.io/rocks-binaries-dev/: Error loading file: [string "/tmp/luarocks_local_cache-7694486/https___nvi..."]:2887: ']' expected near '}'
error: no results matching query were found for lua 5.1.
to check if it is available for other lua versions, use --check-lua-versions.
It looks like we may have to improve the luarocks locking mechanism.
Something maybe helpful: Before rocks sync support parallel, we can use parallel luarocks install ::: package1 package2 ... to do same work.
That would add a dependency on GNU parallel, and would not solve the luarocks locking mechanism issue.
Oh, I means before rocks.nvim support it. We can install the packages in command line.
The issue is in luarocks, not rocks.nvim, so I don't think parallel will help.
Why you not wrap
luaraocks.cmd.installwithuv.new_thread?
Wouldn't help. Multiple instances of luarocks running in parallel leads to install tree corruption. Anyways, This issue will be closed with #539.