io
io copied to clipboard
The JNA implementation seems unbarably slow on Windows (7)
When running things like sbt eclipse
SBT munches through the local Ivy repository very aggressively. On Windows it used the JNA code for this
at sbt.internal.io.WinMilli$.getModifiedTimeNative(Milli.scala:271)
and it seems to be unbearably slow. Many of the file accesses take multiple seconds twice, once to figure out a symlink that needs to be resolved (repo is relocated via symlink) and once to access the actual file. (observed via procmon) Because these slow cases run in the hundreds (for certain projects; iterates multiple times too), it can take up to 15 minutes to munch through.
When enabling -Dsbt.io.jdktimestamps=true
the same file access becomes sub millisecond. (I will re-iterate on Monday with a cold(er) system to verify).
This is a:
- SBT 1.1.4
- Intel i7 4th gen
- Win 7
- Spectre/Meltdown fixes enabled (given the milisecond story that might be of relevance)
- Trend Micro AV and more anti-nasty protection
Thanks for the report @spangaer
Confirming improvement by system property flag.
- post weekend run
- post reboot run (force cold FS cache)
- post eclipse start run
All have consistent improvement. It's not blazingly fast, because it generates 300k file related procmon entries for a single run. But only between 0 and 5 (across runs) are over 1 second instead of hundreds, the remainder is in the millisecond or sub-millisecond range and at least the CPU is doing something else then waiting for timestamp info.