Iaroslav Igoshev
Iaroslav Igoshev
@RehanSD, what is the status here? Are there any perf numbers?
> @RehanSD, what is the status here? Are there any perf numbers? @mvashishtha, if you are going to take over this PR, could you respond to my question?
> @YarShev, @RehanSD is going to fix some correctness bugs on the branch, then do performance testing. AFAIK he doesn't know have any perf numbers yet. I see, thanks! Look...
Is there a chance to put this in 0.16?
Yes, I didn't see any perf gain from that by profiling our use case so I don't think it is worth making a PR for now.
As to the performance penalties I said earlier, you can see that in the following simple example. ```python if __name__ == "__main__": import modin.pandas as pd import numpy as np...
> > Also, I was wondering if we could do preprocessing (ray.put. client.scatter) for the arguments to be passed in a ray/dask remote function? That may drastically increase performance. >...
@noloerino, please note that you are getting the error because f1 didn't get enough arguments. The following message says about that - `Exception: 'TypeError("f1() missing 1 required positional argument: \'arg\'")'`....
Also, note that Dask may not work with tuples passed in to a remote function. That's why we have used a list for the call queue so far.
> the `in f2` print statement is hit before `f1` is even called, which seems to imply that Dask is depth-first evaluating functions within tuples I don't see this behavior....