Dagger.jl icon indicating copy to clipboard operation
Dagger.jl copied to clipboard

Add FileTrees reverse CI

Open jpsamaroo opened this issue 3 years ago • 3 comments

@DrChainsaw @shashi we should add FileTrees reverse CI to Dagger to prevent me from breaking things :smile:

jpsamaroo avatar Feb 19 '22 19:02 jpsamaroo

I suppose more testing is always good, but I guess there is a cost to it. FileTrees pretty much only makes this call to Dagger:

compute(ctx, delayed((xs...)->[xs...]; meta=true)(thunks...))

Which I think might be covered by Daggers existing test suite.

Btw, that splatting is probably not so good for performance (thunks is an array with all lazy values in the tree, so it can be very large). Any suggestions on how to rewrite (I know I asked the same question in https://github.com/shashi/FileTrees.jl/issues/63 but the discussion went on there for a while so you might have missed it)? I think that the vcating is unnecessary, but is there a way to kick of the parallel computation without splatting thunks?

DrChainsaw avatar Feb 21 '22 13:02 DrChainsaw

Even if the invocation is simple, Dagger itself is pretty complex, and that particular API may change going forward, so it might be good to have it just in case.

w.r.t splatting, can you provide some performance numbers and an MWE in a separate issue?

jpsamaroo avatar Feb 22 '22 18:02 jpsamaroo

Sure, I suppose no sane downstream maintainer would say no to upstream CI :)

Splatting issue coming up!

DrChainsaw avatar Feb 22 '22 21:02 DrChainsaw