Parallelize readdir() reading and stat().

[james-antill]

This is a port of the parallel node.Visit() code from my fork. It spawns upto 32 goroutines to do a node.Visit() and when it reaches the limit each goroutine will fall back to processing them directly. The code should work identically if the global semWait is set to 0.

It does work, however there are currently four test failures that only happen when semWait is set all due to of the same root problem ... the nodes are added to the directory in the semi random order they are received from the channel (see line 513). The way each test "sorts" is a noop, so the original order is kept.

In theory you can "fix" some of this with:

• Part of this patch which sorts by name/mtime/etc. as well as directory, when we do dir. sorts: https://github.com/james-antill/tree/commit/d27032aae2d9670771daf4af00fea4705e5ceacf

• This patch which sorts by name as well as size/ctime/mtime when we sort by those: https://github.com/james-antill/tree/commit/ff168c233d0550b05a4244ea4bb515094058ec73

• and in a similar situation, sort directory sizes by printed size: https://github.com/james-antill/tree/commit/76d027f7e68240dd1fcb6271157dec2efe49cb58