tomviz icon indicating copy to clipboard operation
tomviz copied to clipboard

Resize data at load time

Open anderskaestner opened this issue 6 years ago • 4 comments

Is your feature request related to a problem? Please describe. Tomviz tends to crash when I load and work with large data sets (close to the limits given by the available memory)

Describe the solution you would like Usually, it is not needed to work with the full data set. Therefore it would make sense to have a dialog at load time that allows the user to crop and downsize the data before loading. This would reduce the stress on the available memory.

Describe alternatives you have considered None

Additional context None

anderskaestner avatar Nov 03 '19 11:11 anderskaestner

If you try the latest nightly builds we have this implemented for HDF5-based formats - supporting subvolumes or striding. How large is the data you mention, what format are you using?

cryos avatar Nov 04 '19 13:11 cryos

The data I used was 2600 tiff images (1200x1200pixels, 16-bit) ~ 8Gb on a machine with 32Gb RAM. The data opened, but Tomviz crashed soon after the display of a volume rendering. BTW, we rarely have hdf5 files (this will change, but for now tiff slices are what we have)

anderskaestner avatar Nov 04 '19 14:11 anderskaestner

If you have HDF5 with a volume we now have a generalized 3D array interface discussed here, and you can even resample. We have not generalized it to TIFF, it is something I would like to assess in the future but we were able to reuse the HDF5 routines here.

It would be good to try and understand the crash, is this the latest version of Tomviz? I don't think I have anything with quite so many projections.

cryos avatar Nov 04 '19 15:11 cryos

Yes 😊 This is what I’m looking for.

From: "Marcus D. Hanwell" [email protected] Reply-To: OpenChemistry/tomviz [email protected] Date: Monday, 4 November 2019 at 16:05 To: OpenChemistry/tomviz [email protected] Cc: Kaestner Anders [email protected], Author [email protected] Subject: Re: [OpenChemistry/tomviz] Resize data at load time (#1986)

If you have HDF5 with a volume we now have a generalized 3D array interface discussed herehttps://tomviz.readthedocs.io/en/latest/data/#hdf5-subsampling, and you can even resample. We have not generalized it to TIFF, it is something I would like to assess in the future but we were able to reuse the HDF5 routines here.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/OpenChemistry/tomviz/issues/1986?email_source=notifications&email_token=ACVIDXBZJHTRMCHJ2QZLGATQSA2YDA5CNFSM4JIK42HKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEC7RYEQ#issuecomment-549395474, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACVIDXCINRFNZ5W4ABQN2XLQSA2YDANCNFSM4JIK42HA.

anderskaestner avatar Nov 04 '19 15:11 anderskaestner