TypeError appears when following "getting started" guide

[nwmorten]

Hello! I encountered the following TypeError while following the getting started guide, using the MMNIST data set as an example:

$ python tools/visualizations/vis_video.py -d mmnist -w work_dirs/mmnist_simvp_gsta --index 0 --save_dirs fig_mmnist_vis

mmnist_simvp_gsta (10, 1, 64, 64)
Traceback (most recent call last):
  File "/home/admin/miniconda3/envs/OpenSTL/lib/python3.10/site-packages/PIL/Image.py", line 3095, in fromarray
    mode, rawmode = _fromarray_typemap[typekey]
KeyError: ((1, 1, 1), '<f4')

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/admin/Documents/Code/OpenSTL/tools/visualizations/vis_video.py", line 142, in <module>
    main()
  File "/home/admin/Documents/Code/OpenSTL/tools/visualizations/vis_video.py", line 127, in main
    show_video_gif_single(inputs.copy(), use_rgb=use_rgb,
  File "/home/admin/Documents/Code/OpenSTL/openstl/utils/visualization.py", line 172, in show_video_gif_single
    imageio.mimsave(out_path, images)
  File "/home/admin/miniconda3/envs/OpenSTL/lib/python3.10/site-packages/imageio/v2.py", line 495, in mimwrite
    return file.write(ims, is_batch=True, **kwargs)
  File "/home/admin/miniconda3/envs/OpenSTL/lib/python3.10/site-packages/imageio/plugins/pillow.py", line 425, in write
    pil_frame = Image.fromarray(frame, mode=mode)
  File "/home/admin/miniconda3/envs/OpenSTL/lib/python3.10/site-packages/PIL/Image.py", line 3098, in fromarray
    raise TypeError(msg) from e
TypeError: Cannot handle this data type: (1, 1, 1), <f4

Setup: I followed all the steps in the installation and getting started guide.

Possible Solution: I amended .../OpenSTL/tools/visualizations/vis_video.py by adding np.squeeze() to the function show_video_gif_single:

if len(data.shape) > 3:
        data=data.swapaxes(1, 2).swapaxes(2, 3)
        data=np.squeeze(data)

Executing the terminal command mentioned at the top of this issue again yields no errors after the above mentioned fix. The generated mmnist_mmnist_simvp_gsta_0.gif works fine. However, the gif for input, pred and true show a black unchanging picture.

I would love to get some input

1. if anyone has had this problem before,
2. if my proposed solution is viable, and
3. if you might know why the three gifs mentioned are faulty.

Thanks in advance!

[Lupin1998]

Hi, @nwmorten. Sorry for the late reply. Have I solved this issue yet?

[nwmorten]

Not that I'm aware of.

To fix the Gifs of input, pred and true produced by the function show_video_gif_single() in openstl/utils/visualization.py, i made the following changes:

def show_video_gif_single(data, out_path=None, use_rgb=False):
    """generate gif with a video sequence"""
    if len(data.shape) > 3:
        data=data.swapaxes(1, 2).swapaxes(2, 3) # (10,1,64,64) --> (10,64,64,1)
        data=np.clip(data, 0, 1) # prediction has some values slightly outside of [0,1] bounds. this clips them
        data=np.squeeze(data) * 255 # (10,64,64,1) --> (10,64,64). multiply by 255 to convert range [0,1] to grayscale [0,255]
        data = data.astype('u1')

    images = []
   ...

This seems more like a hack than an out right solution. Any idea how this worked with grayscale images before?