dlang
Make import of std.internal.unicode_tables lazy.
There's no need to open and open and parse the gigantic
std.internal.unicode_tables if it's never needed.
My journey:
std.net.curl -> etc.c.curl -> std.socket -> std.stdio -> std.uni -> std.internal.unicode_tables.
> cd std && echo "import std.uni;" > foo.d
> time dmd -c -o- foo.d
dmd -c -o- foo.d 0.10s user 0.00s system 99% cpu 0.101 total
> time dmd -c -o- foo.d -I..
dmd -c -o- foo.d -I.. 0.04s user 0.01s system 99% cpu 0.050 total
Follow-up to https://github.com/dlang/phobos/pull/5916
Alternatively a benchmark with avgtime:
> avgtime -r100 dmd -c -o- foo.d
Total time (ms): 9942.26
Repetitions : 100
Sample mode : 96 (15 occurrences)
Median time : 96.908
Avg time : 99.4226
Std dev. : 6.38707
Minimum : 92.079
Maximum : 120.269
95% conf.int. : [86.9042, 111.941] e = 12.5184
99% conf.int. : [82.9706, 115.875] e = 16.452
EstimatedAvg95%: [98.1707, 100.674] e = 1.25184
EstimatedAvg99%: [97.7774, 101.068] e = 1.6452
> avgtime -r100 dmd -c -o- foo.d -I..
Total time (ms): 4678.42
Repetitions : 100
Sample mode : 46 (40 occurrences)
Median time : 46.537
Avg time : 46.7842
Std dev. : 1.39462
Minimum : 44.338
Maximum : 53.491
95% conf.int. : [44.0508, 49.5176] e = 2.73341
99% conf.int. : [43.1919, 50.3765] e = 3.59231
EstimatedAvg95%: [46.5108, 47.0575] e = 0.273341
EstimatedAvg99%: [46.425, 47.1434] e = 0.359231
Thanks for your pull request, @wilzbach!
Bugzilla references
Your PR doesn't reference any Bugzilla issue.
If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.
Another module:
> cd std && echo "import std.net.curl;" > foo.d
> time dmd -c -o- foo.d
time dmd -c -o- foo.d
dmd -c -o- foo.d 0.24s user 0.04s system 99% cpu 0.288 total
> time dmd -c -o- foo.d -I..
dmd -c -o- foo.d -I.. 0.21s user 0.03s system 99% cpu 0.241 total
With avgtime:
> avgtime -r100 dmd -c -o- foo.d
------------------------
Total time (ms): 31928.2
Repetitions : 100
Sample mode : 340 (29 occurrences)
Median time : 315.292
Avg time : 319.282
Std dev. : 28.5593
Minimum : 276.011
Maximum : 375.411
95% conf.int. : [263.307, 375.257] e = 55.9752
99% conf.int. : [245.718, 392.846] e = 73.5638
EstimatedAvg95%: [313.685, 324.88] e = 5.59752
EstimatedAvg99%: [311.926, 326.639] e = 7.35638
> avgtime -r100 dmd -c -o- foo.d -I..
------------------------
Total time (ms): 24724
Repetitions : 100
Sample mode : 230 (46 occurrences)
Median time : 237.948
Avg time : 247.24
Std dev. : 19.8293
Minimum : 228.172
Maximum : 301.335
95% conf.int. : [208.375, 286.105] e = 38.8647
99% conf.int. : [196.163, 298.317] e = 51.0769
EstimatedAvg95%: [243.353, 251.126] e = 3.88647
EstimatedAvg99%: [242.132, 252.347] e = 5.10769
I managed to make a couple of other imports lazy and now std.uni is quite slim:
Before:
parse foo
importall foo
import object (/usr/include/dlang/dmd/object.d)
import std.uni (/usr/include/dlang/dmd/std/uni.d)
import std.meta (/usr/include/dlang/dmd/std/meta.d)
import std.traits (/usr/include/dlang/dmd/std/traits.d)
import std.functional (/usr/include/dlang/dmd/std/functional.d)
import std.range.primitives (/usr/include/dlang/dmd/std/range/primitives.d)
import std.internal.unicode_tables (/usr/include/dlang/dmd/std/internal/unicode_tables.d)
semantic foo
import std.range (/usr/include/dlang/dmd/std/range/package.d)
import std.typecons (/usr/include/dlang/dmd/std/typecons.d)
import core.stdc.stdint (/usr/include/dlang/dmd/core/stdc/stdint.d)
import core.stdc.stddef (/usr/include/dlang/dmd/core/stdc/stddef.d)
import core.stdc.signal (/usr/include/dlang/dmd/core/stdc/signal.d)
import core.stdc.wchar_ (/usr/include/dlang/dmd/core/stdc/wchar_.d)
import core.stdc.config (/usr/include/dlang/dmd/core/stdc/config.d)
import core.stdc.stdarg (/usr/include/dlang/dmd/core/stdc/stdarg.d)
import core.stdc.stdlib (/usr/include/dlang/dmd/core/stdc/stdlib.d)
import core.stdc.stdio (/usr/include/dlang/dmd/core/stdc/stdio.d)
import core.stdc.time (/usr/include/dlang/dmd/core/stdc/time.d)
import core.sys.posix.sys.types (/usr/include/dlang/dmd/core/sys/posix/sys/types.d)
import core.sys.posix.config (/usr/include/dlang/dmd/core/sys/posix/config.d)
import std.array (/usr/include/dlang/dmd/std/array.d)
import std.algorithm.iteration (/usr/include/dlang/dmd/std/algorithm/iteration.d)
import std.range.interfaces (/usr/include/dlang/dmd/std/range/interfaces.d)
import std.format (/usr/include/dlang/dmd/std/format.d)
import core.vararg (/usr/include/dlang/dmd/core/vararg.d)
import std.exception (/usr/include/dlang/dmd/std/exception.d)
semantic2 foo
semantic3 foo
code foo
After
parse foo
importall foo
import object (/home/seb/dlang/druntime/import/object.d)
import std.uni (/home/seb/dlang/phobos/std/uni.d)
import std.meta (/home/seb/dlang/phobos/std/meta.d)
import std.traits (/home/seb/dlang/phobos/std/traits.d)
import std.functional (/home/seb/dlang/phobos/std/functional.d)
import std.range.primitives (/home/seb/dlang/phobos/std/range/primitives.d)
semantic foo
semantic2 foo
semantic3 foo
code foo
Maybe another approach that will speed up imports without massive mutilation of std.uni might be to generate .di file from unicode_tables.d and import that instead. That way, no actual code change will be required, and we avoid template bloat.
Can we use .di files here to just provide an interface to the tables, so that the actual tables are parsed only when building Phobos and never when building user code?
Can we use .di files here to just provide an interface to the tables, so that the actual tables are parsed only when building Phobos and never when building user code?
Yeah that should work in theory, but then we would probably have to change the layout of Phobos a bit (e.g. like Druntime with it's import folder). However, I have an idea and will give it a try.
Can we use .di files here to just provide an interface to the tables, so that the actual tables are parsed only when building Phobos and never when building user code?
Tried it here: https://github.com/dlang/phobos/pull/6123.
tl;dr: DMD's header generation has many bugs, I bumped into two. They can be fixed though (submitted PRs). The big problem though is that the unicode_tables file uses templates too, so they won't be removed by the header file generation.
Yeah that should work in theory, but then we would probably have to change the layout of Phobos a bit (e.g. like Druntime with it's import folder). However, I have an idea and will give it a try.
Dmd prefers importing .di over .d files, so you could have them side by side. But our experience with .di files in druntime was fairly bad, try to avoid them.
