steinbock
steinbock copied to clipboard
Published
20 hours ago •
BodenmillerGroup
BodenmillerGroup
Error when running `steinbock classify ilastik run`
I re-ran the bash script shown here:
#!/usr/bin/env bash
BASEDIR=$(cd -- "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)
cd "${BASEDIR}"
# raw data collection
mkdir raw
wget https://zenodo.org/record/7997296/files/IMCWorkflow.ilp
wget https://zenodo.org/record/7997296/files/analysis.zip
unzip analysis.zip
rm analysis.zip
rm -r analysis/cpinp
rm -r analysis/cpout
rm -r analysis/histocat
rm -r analysis/ilastik
rm -r analysis/ometiff
cd raw
wget https://zenodo.org/record/7575859/files/panel.csv
wget https://zenodo.org/record/7575859/files/Patient1.zip
wget https://zenodo.org/record/7575859/files/Patient2.zip
wget https://zenodo.org/record/7575859/files/Patient3.zip
wget https://zenodo.org/record/7575859/files/Patient4.zip
cd ${BASEDIR}
# steinbock alias setup
shopt -s expand_aliases
alias steinbock="docker run -v ${BASEDIR}:/data -u $(id -u):$(id -g) ghcr.io/bodenmillergroup/steinbock:0.16.0"
# raw data preprocessing
steinbock preprocess imc panel --namecol Clean_Target
steinbock preprocess imc images --hpf 50
# random forest-based segmentation using Ilastik/CellProfiler
steinbock classify ilastik prepare --cropsize 500 --seed 123
rm pixel_classifier.ilp && mv IMCWorkflow.ilp pixel_classifier.ilp
rm -r ilastik_crops && mv analysis/crops ilastik_crops
steinbock classify ilastik fix --no-backup
steinbock classify ilastik run
steinbock segment cellprofiler prepare
steinbock segment cellprofiler run -o masks_ilastik
In the past (using an older version of the ImcSegmentationPipeline output) worked fine but now I'm getting the following error:
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/app/steinbock/steinbock/__main__.py", line 17, in <module>
steinbock_cmd_group(prog_name="steinbock")
File "/opt/steinbock-venv/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/opt/steinbock-venv/lib/python3.8/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/opt/steinbock-venv/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/steinbock-venv/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/steinbock-venv/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/steinbock-venv/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/steinbock-venv/lib/python3.8/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/app/steinbock/steinbock/_cli/utils.py", line 34, in wrapper
return func(*args, **kwargs)
File "/app/steinbock/steinbock/classification/_cli/ilastik.py", line 336, in fix_cmd
ilastik.fix_ilastik_project_file_inplace(
File "/app/steinbock/steinbock/classification/ilastik/_ilastik.py", line 390, in fix_ilastik_project_file_inplace
_fix_pixel_classification_group_inplace(
File "/app/steinbock/steinbock/classification/ilastik/_ilastik.py", line 438, in _fix_pixel_classification_group_inplace
block = np.transpose(block, axes=transpose_axes)
File "<__array_function__ internals>", line 180, in transpose
File "/opt/steinbock-venv/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 660, in transpose
return _wrapfunc(a, 'transpose', axes)
File "/opt/steinbock-venv/lib/python3.8/site-packages/numpy/core/fromnumeric.py", line 57, in _wrapfunc
return bound(*args, **kwds)
ValueError: axes don't match array
/opt/ilastik/ilastik-meta/lazyflow/lazyflow/classifiers/__init__.py:20: UserWarning: init: Could not import tiktorch classifier
warnings.warn("init: Could not import tiktorch classifier")
INFO ilastik_main: Starting ilastik from "/opt/ilastik".
Starting ilastik from "/opt/ilastik".
WARNING 2023-06-12 13:35:28,748 opConservationTracking 31 139775893624640 Could not find any ILP solver
WARNING 2023-06-12 13:35:28,756 opStructuredTracking 31 139775893624640 Could not find any ILP solver
WARNING 2023-06-12 13:35:28,758 structuredTrackingWorkflow 31 139775893624640 Could not find any learning solver. Tracking will use flow-based solver (DPCT). Learning for tracking will be disabled!
INFO ilastik.shell.projectManager: Opening Project: pixel_classifier.ilp
WARNING dataset.py(313): H5pyDeprecationWarning: dataset.value has been deprecated. Use dataset[()] instead.
WARNING opSimpleBlockedArrayCache.py(43): FutureWarning: Conversion of the second argument of issubdtype from `dtype` to `np.generic` is deprecated. In future, it will be treated as `np.object_ == np.dtype(dtype).type`.
WARNING opSlicedBlockedArrayCache.py(141): FutureWarning: Conversion of the second argument of issubdtype from `dtype` to `np.generic` is deprecated. In future, it will be treated as `np.object_ == np.dtype(dtype).type`.
ERROR 2023-06-12 13:35:31,985 log_exception 31 139775893624640 Traceback (most recent call last):
File "/opt/ilastik/ilastik-meta/ilastik/ilastik/shell/projectManager.py", line 460, in _loadProject
serializer.deserializeFromHdf5(self.currentProjectFile, projectFilePath, self._headless)
File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 1151, in deserializeFromHdf5
ss.deserialize(topGroup)
File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/pixelClassification/pixelClassificationSerializer.py", line 100, in deserialize
super().deserialize(group)
File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 264, in deserialize
self._deserialize(group[self.name], self.inslot)
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/utility/timer.py", line 153, in wrapper
return func(*args, **kwargs)
File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 622, in _deserialize
self.inslot[index][slicing] = blockArray
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 992, in __setitem__
p[key] = value
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 992, in __setitem__
p[key] = value
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 992, in __setitem__
p[key] = value
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 988, in __setitem__
self.operator.setInSlot(self.top_level_slot, self.subindex, roi, value)
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/operators/opCompressedUserLabelArray.py", line 394, in setInSlot
self._setInSlotInput(slot, subindex, roi, new_pixels)
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/operators/opCompressedUserLabelArray.py", line 416, in _setInSlotInput
block_rois = getIntersectingRois(self.Output.meta.shape, self._blockshape, (roi.start, roi.stop))
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/roi.py", line 639, in getIntersectingRois
block_rois = list(map(partial(getBlockBounds, dataset_shape, blockshape), block_starts))
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/roi.py", line 667, in getBlockBounds
block_bounds = getIntersection(block_bounds, entire_dataset_roi)
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/roi.py", line 402, in getIntersection
assert ((stop - start) > 0).all(), "Rois do not intersect: {} and {}".format(roiA, roiB)
AssertionError: Rois do not intersect: (array([0, 0, 2]), array([253, 252, 3])) and ([0, 0, 0], [500, 500, 1])
ERROR 2023-06-12 13:35:31,985 log_exception 31 139775893624640 Project could not be loaded due to the exception shown above.
Aborting Project Open Action
ERROR 2023-06-12 13:35:31,986 excepthooks 31 139775893624640 Unhandled exception in thread: 'MainThread'
ERROR 2023-06-12 13:35:31,988 excepthooks 31 139775893624640 Traceback (most recent call last):
File "/opt/ilastik/ilastik-meta/ilastik/ilastik.py", line 134, in <module>
main()
File "/opt/ilastik/ilastik-meta/ilastik/ilastik.py", line 128, in main
hShell = ilastik_main.main(parsed_args, workflow_cmdline_args)
File "/opt/ilastik/ilastik-meta/ilastik/ilastik_main.py", line 170, in main
f(shell)
File "/opt/ilastik/ilastik-meta/ilastik/ilastik_main.py", line 347, in loadProject
shell.openProjectFile(path, parsed_args.readonly)
File "/opt/ilastik/ilastik-meta/ilastik/ilastik/shell/headless/headlessShell.py", line 125, in openProjectFile
self.projectManager._loadProject(hdf5File, projectFilePath, readOnly)
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/utility/timer.py", line 153, in wrapper
return func(*args, **kwargs)
File "/opt/ilastik/ilastik-meta/ilastik/ilastik/shell/projectManager.py", line 460, in _loadProject
serializer.deserializeFromHdf5(self.currentProjectFile, projectFilePath, self._headless)
File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 1151, in deserializeFromHdf5
ss.deserialize(topGroup)
File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/pixelClassification/pixelClassificationSerializer.py", line 100, in deserialize
super().deserialize(group)
File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 264, in deserialize
self._deserialize(group[self.name], self.inslot)
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/utility/timer.py", line 153, in wrapper
return func(*args, **kwargs)
File "/opt/ilastik/ilastik-meta/ilastik/ilastik/applets/base/appletSerializer.py", line 622, in _deserialize
self.inslot[index][slicing] = blockArray
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 992, in __setitem__
p[key] = value
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 992, in __setitem__
p[key] = value
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 992, in __setitem__
p[key] = value
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/slot.py", line 988, in __setitem__
self.operator.setInSlot(self.top_level_slot, self.subindex, roi, value)
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/operators/opCompressedUserLabelArray.py", line 394, in setInSlot
self._setInSlotInput(slot, subindex, roi, new_pixels)
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/operators/opCompressedUserLabelArray.py", line 416, in _setInSlotInput
block_rois = getIntersectingRois(self.Output.meta.shape, self._blockshape, (roi.start, roi.stop))
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/roi.py", line 639, in getIntersectingRois
block_rois = list(map(partial(getBlockBounds, dataset_shape, blockshape), block_starts))
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/roi.py", line 667, in getBlockBounds
block_bounds = getIntersection(block_bounds, entire_dataset_roi)
File "/opt/ilastik/ilastik-meta/lazyflow/lazyflow/roi.py", line 402, in getIntersection
assert ((stop - start) > 0).all(), "Rois do not intersect: {} and {}".format(roiA, roiB)
AssertionError: Rois do not intersect: (array([0, 0, 2]), array([253, 252, 3])) and ([0, 0, 0], [500, 500, 1])
I'm not quite sure where this is coming from. What I changed was the version of the ImcSegmentationPipeline that I used for generating the ilastik crops and I upgraded from Ilastik v1.3.3 to v1.4.0. A similar issue was reported in #188 so it would be good to investigate where this is coming from.
Steinbock docker container is fixed to ilastik-1.3.3, it appears that classifiers from ilastik 14.0 and above are not compatible with the older versions. That is the cause of this and #188.
Given above is this actually a bug? Shall we update ilastik with the next release? @jwindhager