compiler: Lazy IET visitors + Search
This PR implements lazy visitors in the IET layer which operate via generators instead of flattening lists of children's results at every node. Replaces FindSymbols, FindNodes, FindWithin and FindApplications with such versions, as well as introducing type hints for the affected visitors.
From my testing this results in a speedup in IET lowering of up to 50%.
Also reimplements Search similarly, benchmarks forthcoming.
@enwask can you give us one or more command lines to reproduce such a compilation speed improvement? perhaps something using the available examples/seismic/... ? or a script of your own? thanks!
also note @enwask that CI is currently failing
Yeah will cook up a demo. Did run the test suite yesterday but some of my typing changes for the review must have broken things, will take a look
Profiled with this script on dewback through numactl, getting a less substantial but measurable 5% improvement overall: you can view the cProfile output from both the upstream head and my fork with lazy visitors.
I also separately ran some of the examples through cProfile; this gist has my test script as well as the profiling output across three examples (namely acoustic, elastic and TTI), where I'm seeing a similar 5% improvement in operator construction. Also in Python 3.10 on dewback.
Codecov Report
Attention: Patch coverage is 92.85714% with 9 lines in your changes missing coverage. Please review.
Project coverage is 87.60%. Comparing base (
77ffae7) to head (c67bb40). Report is 6 commits behind head on main.
| Files with missing lines | Patch % | Lines |
|---|---|---|
| devito/symbolics/search.py | 82.50% | 5 Missing and 2 partials :warning: |
| devito/ir/iet/visitors.py | 97.67% | 2 Missing :warning: |
Additional details and impacted files
@@ Coverage Diff @@
## main #2621 +/- ##
==========================================
- Coverage 92.00% 87.60% -4.40%
==========================================
Files 245 245
Lines 48813 48790 -23
Branches 4310 4305 -5
==========================================
- Hits 44908 42744 -2164
- Misses 3211 5313 +2102
- Partials 694 733 +39
| Flag | Coverage Δ | |
|---|---|---|
| pytest-gpu-aomp-amdgpuX | ? |
|
| pytest-gpu-nvc-nvidiaX | ? |
Flags with carried forward coverage won't be shown. Click here to find out more.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
Noticed that the Search rewrite is actually a regression; trying to work out why...
Found out why! Turns out bfs_first_hit is neither a BFS nor does it return only the first hit