root
root copied to clipboard
root-project
DistRDF profiling
The pr includes the implementation of a profiling mechanism for DistRDF. When the feature is activated, profiling data is collected in each distributed node and merged to produce a visualization (i.e. flamegraph) on the client side.
Activation of the feature is done using the ClingProfile ctx manager, accessed from ROOT.RDF.Experimental.Distributed module. Usage example (dimuon analysis):
RDataFrame = ROOT.RDF.Experimental.Distributed.Dask.RDataFrame
ClingProfile = ROOT.RDF.Experimental.Distributed.ClingProfile
...
df = RDataFrame("Events", files, npartitions=npartitions, daskclient=client)
with ClingProfile(df, filename="flamegraph.svg"):
df = df.Filter("nMuon == 2", "Events with exactly two muons")
...
h.GetValue()
This pull request introduces 6 alerts when merging 2da558f20940b9cee2b422fae73499a5756b8ed5 into 6a95198424bb5f0bd1a84268e1063fd4ed6d9114 - view on LGTM.com
new alerts:
- 4 for Property in old-style class
- 1 for Wrong name for an argument in a call
- 1 for Unused import
![lgtm-com[bot] avatar](https://avatars.githubusercontent.com/in/17324?v=4 "Published by a pub.dev verified publisher"){kind=small}
This pull request introduces 6 alerts when merging 040a4f7b7814df328990056ca4433b95039c2758 into 3798716e44a9c24b01bac9a4dcc6fefb97dd9bda - view on LGTM.com
new alerts:
- 4 for Property in old-style class
- 1 for Wrong name for an argument in a call
- 1 for Unused import
This pull request introduces 5 alerts when merging 5db72253ea51171deb4ee3684a1398399a908ef0 into 452f82119a07fb873a7b050004477d9d9f93120d - view on LGTM.com
new alerts:
- 4 for Property in old-style class
- 1 for Wrong name for an argument in a call
This pull request introduces 4 alerts when merging 2e5a088d253e65a6889d9c1e3f0e264d7f804435 into d11acb0d72e61089abfd15fa2f946afb0bb4b149 - view on LGTM.com
new alerts:
- 2 for Signature mismatch in overriding method
- 1 for Wrong name for an argument in a call
- 1 for Unused import
This pull request introduces 1 alert when merging 052a97a7f59474fdd38f89f9f37d48f48f81acbd into 0b49293d4d67dfecfe58e710616094b30a90130f - view on LGTM.com
new alerts:
- 1 for Wrong name for an argument in a call
Just had a review with @vepadulano , congratulations on this very elegant and modular solution for collecting the profiling information!
Starting build on ROOT-debian10-i386/soversion, ROOT-performance-centos8-multicore/cxx17, ROOT-ubuntu18.04/nortcxxmod, ROOT-ubuntu2004/python3, mac1015/cxx17, mac11/cxx14, windows10/cxx14
How to customize builds
Build failed on ROOT-ubuntu2004/python3. Running on root-ubuntu-2004-1.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build See console output.
Failing tests:
Build failed on mac11/cxx14. Running on macphsft20.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build See console output.
Failing tests:
- projectroot.runtutorials.tutorial_dataframe_distrdf001_spark_connection_py
- projectroot.runtutorials.tutorial_dataframe_distrdf002_dask_connection_py
- projectroot.roottest.python.distrdf.spark.roottest_python_distrdf_spark_test_all
- projectroot.roottest.python.distrdf.dask.roottest_python_distrdf_dask_test_all
Build failed on mac1015/cxx17. Running on macitois22.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build See console output.
Failing tests:
- projectroot.roottest.python.distrdf.spark.roottest_python_distrdf_spark_test_all
- projectroot.roottest.python.distrdf.dask.roottest_python_distrdf_dask_test_all
Starting build on ROOT-debian10-i386/soversion, ROOT-performance-centos8-multicore/cxx17, ROOT-ubuntu18.04/nortcxxmod, ROOT-ubuntu2004/python3, mac1015/cxx17, mac11/cxx14, windows10/cxx14
How to customize builds
Build failed on ROOT-ubuntu2004/python3. Running on root-ubuntu-2004-3.cern.ch:/home/sftnight/build/workspace/root-pullrequests-build See console output.
Errors:
- [2022-10-13T13:39:57.002Z] CMake Error at /home/sftnight/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1101 (message):
Build failed on windows10/cxx14. Running on null:C:\build\workspace\root-pullrequests-build See console output.
Errors:
- [2022-10-13T13:40:38.385Z] CMake Error at C:/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1101 (message):
Build failed on ROOT-performance-centos8-multicore/cxx17. Running on olbdw-01.cern.ch:/data/sftnight/workspace/root-pullrequests-build See console output.
Errors:
- [2022-10-13T14:00:27.347Z] CMake Error at /data/sftnight/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1101 (message):
Build failed on ROOT-debian10-i386/soversion. Running on pcepsft10.dyndns.cern.ch:/build/workspace/root-pullrequests-build See console output.
Errors:
- [2022-10-13T14:21:39.301Z] CMake Error at /home/sftnight/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1101 (message):
Build failed on mac11/cxx14. Running on macphsft20.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build See console output.
Errors:
- [2022-10-13T14:28:07.706Z] CMake Error at /Users/sftnight/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1101 (message):
Build failed on mac1015/cxx17. Running on macitois22.dyndns.cern.ch:/Users/sftnight/build/workspace/root-pullrequests-build See console output.
Errors:
- [2022-10-13T14:30:06.307Z] CMake Error at /Volumes/HD2/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1101 (message):
Build failed on ROOT-ubuntu18.04/nortcxxmod. Running on sft-ubuntu-1804-3.cern.ch:/build/workspace/root-pullrequests-build See console output.
Errors:
- [2022-10-13T15:20:10.126Z] CMake Error at /mnt/build/workspace/root-pullrequests-build/rootspi/jenkins/root-build.cmake:1101 (message):