spatialdata icon indicating copy to clipboard operation
spatialdata copied to clipboard
verified publisher icon scverse

improves dataloader performance

Open giovp opened this issue 1 year ago • 1 comments

replaces #565 #622

iteration over 20 batches, single worker

new implementation image

main image

one annoying thing is that the "apply" method of the dataframe to get the bounding box selection is quite slow.

giovp avatar Aug 21 '24 18:08 giovp

quick push to try #699 where tiling is vectorized, removed the need for pandas.DataFrame.apply. Quite big speedup

image

giovp avatar Sep 03 '24 00:09 giovp