ilastik
Carving, MemoryError on Preprocessing stage
Please help with passing the pre-processing stage. I am on a Linux system with Ubuntu 20.04, 16 gig ram memory.
I have a 3D data 2560 x 2560 x 1450 that I am trying to label using ilastik. I tried all the released version and it always crashes at the pre-processing stage regardless of what filter scale I use. I tried all the way up to 99 from .9 and it still crashes every time I hit "run". I small window with "Error " pops out right after and below is the message displayed in terminal. It seems similar to the issue described in #1852 and #1951 but I do not understand the solution though the issue is closed over there. Thank you very much.
ERROR 2021-05-29 16:43:26,229 log_exception 43172 140437953783424 Traceback (most recent call last):
File "/home/user/Downloads/ilastik-1.2.2post2-Linux/ilastik-meta/lazyflow/lazyflow/request/request.py", line 312, in _execute
self._result = self.fn()
File "/home/user/Downloads/ilastik-1.2.2post2-Linux/ilastik-meta/lazyflow/lazyflow/slot.py", line 876, in __call__
result_op = self.operator.execute(self.slot, (), self.roi, destination)
File "/home/user/Downloads/ilastik-1.2.2post2-Linux/ilastik-meta/ilastik/ilastik/workflows/carving/opPreprocessing.py", line 405, in execute
mst = self._opMstProvider.MST.value
File "/home/I am on a Linux system with Ubuntu 20.04, 16 gig ram memory. /Downloads/ilastik-1.2.2post2-Linux/ilastik-meta/lazyflow/lazyflow/slot.py", line 1100, in value
temp = self[:].wait()
File "/home/I am on a Linux system with Ubuntu 20.04, 16 gig ram memory. /Downloads/ilastik-1.2.2post2-Linux/ilastik-meta/lazyflow/lazyflow/request/request.py", line 487, in wait
return self._wait(timeout)
File "/home/user/Downloads/ilastik-1.2.2post2-Linux/ilastik-meta/lazyflow/lazyflow/request/request.py", line 513, in _wait
self._wait_within_request( current_request )
File "/home/user/Downloads/ilastik-1.2.2post2-Linux/ilastik-meta/lazyflow/lazyflow/request/request.py", line 312, in _execute
self._result = self.fn()
File "/home/user/Downloads/ilastik-1.2.2post2-Linux/ilastik-meta/lazyflow/lazyflow/slot.py", line 876, in __call__
result_op = self.operator.execute(self.slot, (), self.roi, destination)
File "/home/user/Downloads/ilastik-1.2.2post2-Linux/ilastik-meta/ilastik/ilastik/workflows/carving/opPreprocessing.py", line 219, in execute
volume_feat = self.Image( *roiFromShape( self.Image.meta.shape ) ).wait()
File "/home/user/Downloads/ilastik-1.2.2post2-Linux/ilastik-meta/lazyflow/lazyflow/request/request.py", line 487, in wait
return self._wait(timeout)
File "/home/user/Downloads/ilastik-1.2.2post2-Linux/ilastik-meta/lazyflow/lazyflow/request/request.py", line 513, in _wait
self._wait_within_request( current_request )
File "/home/user/Downloads/ilastik-1.2.2post2-Linux/ilastik-meta/lazyflow/lazyflow/request/request.py", line 312, in _execute
self._result = self.fn()
File "/home/user/Downloads/ilastik-1.2.2post2-Linux/ilastik-meta/lazyflow/lazyflow/slot.py", line 861, in __call__
destination = self.slot.stype.allocateDestination(self.roi)
File "/home/user/Downloads/ilastik-1.2.2post2-Linux/ilastik-meta/lazyflow/lazyflow/stype.py", line 109, in allocateDestination
storage = numpy.ndarray(shape, dtype=self.slot.meta.dtype)
MemoryError
Hi @sitesite53,
sorry that you ran into this problem!
Your data is quite big - how much RAM do you have available? (The memory error is raised once more memory is requested than is available). The size of the filter does not have any effect on this issue.
Version-wise I'd suggest to use the latest beta (currently 1.4.0b15)
In order to get you going quickly I'd suggest to consider one or more of the following suggestions:
- is whatever you want to segment spread across the whole volume? Maybe you can cutout a smaller subvolume, that will not result in such a high memory usage
- move to a computer that has more ram available
- would it be possible to downsample (e.g. Image > Scale… and Image > Adjust > Size in imagj/fiji) the data a bit? (Is the structure you are trying to segment very large - do you need full resolution detail?)
- what file format is your data? if it is some sort of tif, ilastik might read it inefficiently and cause some memory spiking
- Do you have any other "heavy" processes on your computer running (for me, Firefox is a big RAM hog...).
Cheers Dominik
Thank you for your response Dominik. I have a 16 gb ram and have just cut the bigger volume into smaller ones of size 512 x 512 x 512 and trying to generate the labels on each of those. Here too it seems to be stuck on the same pre-processing stage for hours now.
I am using the 1.3.2 version because I have noticed that the 1.4 version did not seem to allow me to create more than 2 labels (it has background and object only when I tried with a single 2D image on the pixel classification side). I would like to have background, object1, object2, ....., objectn instead if possible on a 3D volume.
I also can't seem to go lower than 512 x 512 x 512 as the resulting volumes become really pixelated. Any further recommendation will help. Thank you so much
Hello @sitesite53,
It is hard to answer the question without seeing any data. Also note, that for these kinds of analysis advice questions forum.image.sc is a better place - there are other users around that might have had a similar task.
Most workflows work with up to 5 dimensions (3D + time + channels). Depending on your data (and your exact goal) you have various options:
- Do Pixel classification for foreground/background segmentation, let's say you have some cells that you can clearly label as foreground, and some background that is distinguishable. This will give you a background/foreground probability map. With this you can go to object classification, where you'd do thresholding on the foreground channel to get your instances (the different objects) out. If all your objects are of the same kind, you can still use the object classifier to find outliers, or badly segmented objects and mark them. I think our youtube videos on pixel classification (some steps there, like
subset features, are optional) and object classification. Both workflows also work with 3D data. - Carving: carving implies there is a boundary between your objects, like in neuron em data. objects are segmented one by one. You always only work on a single object at a time, then save it, then work on the next, see the relevant youtube video.
- Multicut: Multicut also does instance segmentation on boundary separated objects. It requires you to supply a boundary prediction map, that you can generate in pixel classification. There's also a youtube video demonstrating it.
Note: turn on subtitles for all videos.
Cheers Dominik
edit: no version of ilastik lets you add more than two labels (foreground and baground) in carving. So I really encourage using the latest one.
Thank you Dominik, I realized I have not provided the error returned here. What will be on average the typical volume dimensions that will be recommended for carving on a 16 gb ram machine?
QObject::connect: Cannot queue arguments of type 'QVector
Thank you very much