root icon indicating copy to clipboard operation
root copied to clipboard

[DF] Allow calling RunGraphs and VariationsFor on distrdf results

Open eguiraud opened this issue 2 years ago • 18 comments

Companion roottest PR at https://github.com/root-project/roottest/pull/980 .

eguiraud avatar Jul 13 '23 19:07 eguiraud

Test Results

         9 files           9 suites   1d 20h 28m 56s :stopwatch:   2 472 tests   2 465 :heavy_check_mark: 0 :zzz:   7 :x: 21 147 runs  21 129 :heavy_check_mark: 0 :zzz: 18 :x:

For more details on these failures, see this check.

Results for commit 192143fc.

:recycle: This comment has been updated with latest results.

github-actions[bot] avatar Jul 13 '23 21:07 github-actions[bot]

With this patch, this code:

import ROOT
from dask.distributed import Client, LocalCluster

if __name__ == "__main__":
    client = Client(LocalCluster(n_workers=1, processes=False, threads_per_worker=1))
    df = ROOT.RDF.Experimental.Distributed.Dask.RDataFrame(10, daskclient=client)
    df = ROOT.RDataFrame(10)
    c = df.Count()
    cs = ROOT.RDF.Experimental.VariationsFor(c)
    print(cs["nominal"])

breaks with this error:

Traceback (most recent call last):
  File "/tmp/distrdf.py", line 9, in <module>
    cs = ROOT.RDF.Experimental.VariationsFor(c)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/blue/ROOT/master-with-venv/cmake-build/install/lib/ROOT/_facade.py", line 96, in variationsfor
    return rdf_variationsfor(resptr)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Template method resolution failed:
  ROOT::RDF::RResultPtr<ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager,void> > ROOT::RDF::Experimental::VariationsFor(ROOT::RDF::RResultPtr<ROOT::RDF::RInterface<ROOT::Detail::RDF::RLoopManager,void> > resPtr) =>
    TypeError: could not convert argument 1
  Failed to instantiate "VariationsFor(ROOT::RDF::RResultPtr<ULong64_t>*)"

Probably some issue with monkey-patching a template function, but I don't understand what goes wrong exactly or how to fix it. Needs debugging.

eguiraud avatar Jul 18 '23 22:07 eguiraud

I opened a PR that practically supersedes this (https://github.com/root-project/root/pull/16681). It introduces the fixes necessary for the issues highlighted here, plus also unifies the local and distributed RDataFrame constructors. We can move further conversation over to the new PR.

vepadulano avatar Oct 14 '24 23:10 vepadulano

Superseded by the following PR, as explained in https://github.com/root-project/root/pull/13248#issuecomment-2412528571:

  • https://github.com/root-project/root/pull/16681

guitargeek avatar May 29 '25 06:05 guitargeek