Jerome Kelleher
Jerome Kelleher
Nice - hopefully this isn't too intrusive perf-wise. Note for compatibilty, I think we should add a flag to treeseq_init like TSK_COMPUTE_MUTATION_PARENTS or something, that just calls the function unconditionally...
I think this is a good path. We can whittle the time down another bit later, as you say.
Looks very nice, that's about as non-intrusive as it'll get I'd say. Maybe we should try this out in a SLiM build and see if it breaks anything @petrelharp?
> We're now at 0.7s for main and 0.9s for this branch! Excellent - with #3218 we'll get this down to something negligible. Looks like there's a few error conditions...
> One more thought here - there will/may be tree sequences on disk that will now error on load. They will have to be loaded as tables and the parents...
> @jeromekelleher what's the timeframe for needing feedback on this? Is there urgency here? No major hurry, we can leave this sit for a few weeks.
This seems like an important point @hyanwong, we should keep a separate issue to track
Let's take the simple path here and raise an error rather than try to implement the right thing
Very interesting, thanks @percyfal! Can you give more details about how you got these numbers please?
I see, this is coming from Dask. Unfortunately our experience has been that it's quite hard to keep a handle on Dask's memory usage - this is one of the...