webknossos-libs icon indicating copy to clipboard operation
webknossos-libs copied to clipboard

Allow cubing huge images without major slow down

Open philippotto opened this issue 4 years ago • 0 comments

If input images are too big to hold 32 of them in RAM, we currently have to reduce the batch size below 32 which makes cubing way slower. Some sort of intermediate representation would probably help here. For example, chunking a tiff into tiles and then cubing these. There might be better ways to do this, though.

As an example: I currently have sections which are 6 GB each. 32 of them are not enough. I currently have to use a batch size of 4 and cubing 4 slices takes up to 10h which makes cubing the entire dataset not feasible.

philippotto avatar Oct 07 '21 07:10 philippotto