scc
scc copied to clipboard
boyter
Performance: Use os.File.ReadDir / DirEntry to avoid stat (go 1.16+)
Consider adding support for ReadDir and DirEntry (added in go1.16). This API often enables walking code to avoid stat(2) [release notes]. A small DirEntry shim could be used to continue support for older Go toolchains (without the performance improvement).
Afaict, DirectoryWalker could avoid stat for directories, and ignored/excluded paths.
This is probably only a minor speedup in the common case since os.FileInfo.Size is used for processed files. However, it could be much larger when the ratio of non-processed to processed paths in high.
Yep pretty happy to have seen this come in. Just need to evaluate its impact on performance. Something to do for sure though.
Seems to be much faster than most other implementations based on my tests https://engineering.kablamo.com.au/posts/2021/quick-comparison-between-go-file-walk-implementations
Ill hack in a version soon and see how it goes.