Disk-based data

[willtownes]

It would be awesome to be able to run sctransform on data that are too big to fit in memory (eg, HDF5 files accessed through something like DelayedArray). Is this a feature that would be considered for the package in the future? We are hoping to wrap sctransform to facilitate interoperability with SingleCellExperiment objects via the scry package (kstreet13/scry#14 )

[ChristophH]

I agree that it would be awesome, but so far this feature has only been tested in a previous development version of sctransform (through loomR). I will look into the DelayedArray and HDF5Array packages to figure out what I need to do to make sctransform work with on-disk data.

[PedroMilanezAlmeida]

I believe glmGamPoi can handle DelayedArray out-of-the box, right?

Would it be of interest to have at least the "v2 flavor" workflow compatible with DelayedArray?

[saketkc]

Thanks for the suggestion! We will have some updates soon.