[Performance] boosting performance with libuv threads

[kyazdani42]

I'd like to see if we could boost the performance of the tree using libuv threads. This would also allow nvim-tree be completely non blocking for neovim, and would avoid freezing the whole UI.

I think we could:

• 1. compute the lines / highlight groups before drawing
• 2. expore and build nodes in parallel
• 3. compute git status / assign git statuses in parallel
• 4. reloading the whole tree (might not be necessary since we added the watchers)

One thing i'd like to explore would be: writing a Pmap/Pfilter utility that would allow passing a table, a pure function and a callback in order to perform things in parallel.

A bit of refactoring might be needed in order to do that. The parallelized code should be completely pure.

[alex-courtis]

One thing i'd like to explore would be: writing a Pmap/Pfilter utility that would allow passing a table, a pure function and a callback in order to perform things in parallel.

This is necessary. We don't have any "first class" semaphores/locks from libuv or elsewhere, and locks would quickly become complex and prone to deadlocks.

I think I understand: the pure function mutates no shared state until completion. The callback would "join" a synchronous context via, say, vim.schedule:

1. merge results e.g. update nodes
2. act e.g. draw

There would be no need for explicit awaits or joins as vim.schedule effectively handles that.

2 should be simple, no argument actions, allowing them to be performed after multiple different threaded operations in any order. Debouncing would not be absolutely necessary however very desirable to prevent multiple sequential blocking actions with the same result e.g. draw.

3. compute git status / assign git statuses in parallel

This is pretty much done.

[alex-courtis]

Let's finish off FS watchers before we start this, to avoid any unnecessary work or refactioring.

I've finally sorted out debounced thread concurrency, PR soon. This unlocks fs_poll -> fs_event which I intend to tackle next.

[alex-courtis]

I have a branch I have been working on that extracts Node classes, including the root node.

That change may make this threading approach easier.

[alex-courtis]

Following git and fs performance improvements it doesn't appear this is necessary.

Profiling operations such as opening, rendering, diagnostics update etc. indicates that there is no performance issue and async processing would just add fragile complexity.

Profiling refresh indicates some performance issues due to its async nature, however refreshes are very rare and the async fs and git operations do the bulk of the work.

Opening could be faster, however that is a one off cost - caches exist.