[BUG]: Cannot decompress jpeg error for SpaceNet Rio imagery

[jacquesmats]

Thank you for helping us improve solaris!

Summary of the bug

I'm receiving ValueError: cannot decompress jpeg after trying to run a command. I'm point the .yml file to 500x500px .tif files. Just as the API Tiling Tutorial.

I'm not sure what should I put as input for this command. TIFF, PNG, JPG images? From which size?

Steps to reproduce the bug

import solaris as sol
config = sol.utils.config.parse('path/to/file.yml')
inferer = sol.nets.infer.Inferer(config)
inference_data = sol.nets.infer.get_infer_df(config)
inferer(inference_data)

Steps to reproduce the behavior: Run this on Jupyter after installing solaris.

Buggy behavior and/or error message

Please describe the buggy behavior and/or paste output here.

ValueError                                Traceback (most recent call last)
<ipython-input-7-cd090e093ecc> in <module>
      3 inferer = sol.nets.infer.Inferer(config)
      4 inference_data = sol.nets.infer.get_infer_df(config)
----> 5 inferer(inference_data)

~/miniconda3/envs/project-v1-nogpu/lib/python3.6/site-packages/solaris/nets/infer.py in __call__(self, infer_df)
     62         for idx, im_path in enumerate(infer_df['image']):
     63             inf_input, idx_refs, (
---> 64                 src_im_height, src_im_width) = inf_tiler(im_path)
     65 
     66             if self.framework == 'keras':

~/miniconda3/envs/project-v1-nogpu/lib/python3.6/site-packages/solaris/nets/datagen.py in __call__(self, im)
    259         # read in the image if it's a path
    260         if isinstance(im, str):
--> 261             im = imread(im)
    262         # determine how many samples will be generated with the sliding window
    263         src_im_height = im.shape[0]

~/miniconda3/envs/project-v1-nogpu/lib/python3.6/site-packages/solaris/utils/io.py in imread(path, make_8bit, rescale, rescale_min, rescale_max)
     50 
     51     """
---> 52     im_arr = skimage.io.imread(path)
     53     # check dtype for preprocessing
     54     if im_arr.dtype == np.uint8:

~/miniconda3/envs/project-v1-nogpu/lib/python3.6/site-packages/skimage/io/_io.py in imread(fname, as_gray, plugin, flatten, **plugin_args)
     59 
     60     with file_or_url_context(fname) as fname:
---> 61         img = call_plugin('imread', fname, plugin=plugin, **plugin_args)
     62 
     63     if not hasattr(img, 'ndim'):

~/miniconda3/envs/project-v1-nogpu/lib/python3.6/site-packages/skimage/io/manage_plugins.py in call_plugin(kind, *args, **kwargs)
    208                                (plugin, kind))
    209 
--> 210     return func(*args, **kwargs)
    211 
    212 

~/miniconda3/envs/project-v1-nogpu/lib/python3.6/site-packages/skimage/io/_plugins/tifffile_plugin.py in imread(fname, dtype, **kwargs)
     36     # read and return tiff as numpy array
     37     with TiffFile(fname, **kwargs_tiff) as tif:
---> 38         return tif.asarray(**kwargs)

~/miniconda3/envs/project-v1-nogpu/lib/python3.6/site-packages/skimage/external/tifffile/tifffile.py in asarray(self, key, series, memmap, tempdir)
   1505                                      colormapped=False, squeeze=False)
   1506         elif len(pages) == 1:
-> 1507             result = pages[0].asarray(memmap=memmap)
   1508         elif self.is_ome:
   1509             assert not self.is_indexed, "color mapping disabled for ome-tiff"

~/miniconda3/envs/project-v1-nogpu/lib/python3.6/site-packages/skimage/external/tifffile/tifffile.py in asarray(self, squeeze, colormapped, rgbonly, scale_mdgel, memmap, reopen, maxsize)
   2482                 self.sample_format, self.bits_per_sample))
   2483         if self.compression not in TIFF_DECOMPESSORS:
-> 2484             raise ValueError("cannot decompress %s" % self.compression)
   2485         if 'sample_format' in self.tags:
   2486             tag = self.tags['sample_format']

ValueError: cannot decompress jpeg

Environment information

• OS: Ubuntu 18.10
• solaris version: 0.1.2
• python version: 3.7
• version of any relevant dependencies (optional - we may ask for this information later if not provided)

[jacquesmats]

When feeding it with png files, I receive the following: IndexError: index 3 is out of bounds for axis 2 with size 3

Full error:

/home/jacquesmats/miniconda3/envs/project-v1-nogpu/lib/python3.6/site-packages/torch/cuda/__init__.py:118: UserWarning: 
    Found GPU0 GeForce GT 740M which is of cuda capability 3.0.
    PyTorch no longer supports this GPU because it is too old.
    The minimum cuda capability that we support is 3.5.
    
  warnings.warn(old_gpu_warn % (d, name, major, capability[1]))
Traceback (most recent call last):
  File "/home/jacquesmats/miniconda3/envs/project-v1-nogpu/bin/solaris_run_ml", line 10, in <module>
    sys.exit(main())
  File "/home/jacquesmats/miniconda3/envs/project-v1-nogpu/lib/python3.6/site-packages/solaris/bin/solaris_run_ml.py", line 34, in main
    inferer(inf_df)
  File "/home/jacquesmats/miniconda3/envs/project-v1-nogpu/lib/python3.6/site-packages/solaris/nets/infer.py", line 64, in __call__
    src_im_height, src_im_width) = inf_tiler(im_path)
  File "/home/jacquesmats/miniconda3/envs/project-v1-nogpu/lib/python3.6/site-packages/solaris/nets/datagen.py", line 287, in __call__
    subarr = self.aug(image=subarr)['image']
  File "/home/jacquesmats/miniconda3/envs/project-v1-nogpu/lib/python3.6/site-packages/albumentations/core/composition.py", line 194, in __call__
    data = t(force_apply=force_apply, **data)
  File "/home/jacquesmats/miniconda3/envs/project-v1-nogpu/lib/python3.6/site-packages/albumentations/core/transforms_interface.py", line 65, in __call__
    res[key] = target_function(arg, **dict(params, **target_dependencies))
  File "/home/jacquesmats/miniconda3/envs/project-v1-nogpu/lib/python3.6/site-packages/solaris/nets/transform.py", line 101, in apply
    return np.delete(im_arr, self.idx, self.axis)
  File "<__array_function__ internals>", line 6, in delete
  File "/home/jacquesmats/miniconda3/envs/project-v1-nogpu/lib/python3.6/site-packages/numpy/lib/function_base.py", line 4382, in delete
    "size %i" % (obj, axis, N))
IndexError: index 3 is out of bounds for axis 2 with size 3

[nrweir]

@jacquesmats,

See #212 for a solution to the second problem. Without seeing the image file you're passing in, I can't do much to help with the first.

I assume you're updating the path in config = sol.utils.config.parse('path/to/file.yml') to a real path, as that's not a valid path, but a placeholder provided in the tutorial.

[jacquesmats]

Hey @nrweir, thanks! Using .png files and setting p = 0 really worked. Now I'm getting a error because of my GPU.

Anyway, the image file I was passing was a TIFF image, from the Rio dataset, preprocessed in this tutorial.

[nrweir]

Thanks @jacquesmats for the clarification and glad it worked.

I'm going to re-name the issue and flag it to help with tracking. My guess is that the issue is related to the formatting of those images being incompatible with scikit-image's default TIFF loading schema. I'll try to explore what's going on if I get around to it, but feel free to explore yourself and drop your findings here.

[AH-Merii]

@nrweir I was facing the same issue, in order to resolve it I pip installed imagecodecs. Note: I had to pip install it as it wasn't being resolved when I attempted to conda install. After installing imagecodecs, everything seemed to work as expected.

I myself am developing using the docker environment that you provided, so in I added this requirement to the Dockerfile by doing the following:

# activate conda environment
SHELL ["conda", "run", "-n", "solaris", "/bin/bash", "-c"]

# pip install imagecodecs in conda environment
RUN pip install imagecodecs