GraphScope
GraphScope copied to clipboard
[GIE-GAIA] Profiling for GAIA
What do these changes do?
- Provide profile tools in GAIA-IR, to show information of time cost, number of intermediate results in each operator, etc.
- Provide graphviz to visualize the profiling results
Related issue number
Fixes #1805
Codecov Report
Merging #1824 (55f04f0) into main (67a45aa) will decrease coverage by
31.15%. The diff coverage isn/a.
@@ Coverage Diff @@
## main #1824 +/- ##
===========================================
- Coverage 71.48% 40.32% -31.16%
===========================================
Files 89 89
Lines 9847 9847
===========================================
- Hits 7039 3971 -3068
- Misses 2808 5876 +3068
| Impacted Files | Coverage Δ | |
|---|---|---|
| ...ython/graphscope/tests/unittest/test_cython_ast.py | 0.00% <0.00%> (-100.00%) |
:arrow_down: |
| ...ython/graphscope/tests/unittest/test_serailaize.py | 0.00% <0.00%> (-100.00%) |
:arrow_down: |
| python/graphscope/analytical/udf/patch.py | 3.47% <0.00%> (-96.53%) |
:arrow_down: |
| python/graphscope/tests/unittest/test_lazy.py | 0.00% <0.00%> (-96.30%) |
:arrow_down: |
| ...thon/graphscope/tests/unittest/test_scalability.py | 0.00% <0.00%> (-96.16%) |
:arrow_down: |
| python/graphscope/tests/unittest/test_app.py | 0.00% <0.00%> (-95.66%) |
:arrow_down: |
| ...hon/graphscope/tests/unittest/test_create_graph.py | 0.00% <0.00%> (-92.00%) |
:arrow_down: |
| python/graphscope/tests/unittest/test_graph.py | 0.00% <0.00%> (-85.57%) |
:arrow_down: |
| python/graphscope/tests/unittest/test_context.py | 0.00% <0.00%> (-81.04%) |
:arrow_down: |
| python/graphscope/analytical/udf/wrapper.py | 27.27% <0.00%> (-72.73%) |
:arrow_down: |
| ... and 37 more |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact),ø = not affected,? = missing dataPowered by Codecov. Last update 67a45aa...55f04f0. Read the comment docs.
Consider use graphviz to visualize the results. Additionally, we can a bi case to demonstrate the result of how this profiling can help optimization.
Have added graphviz for visualizing the results.
@bmmcq Hi, bingqing wrote some codes for profiling Pegasus, by logging the run time, size of output/communication data of each operator. See if you have any good advice.