dianna icon indicating copy to clipboard operation
dianna copied to clipboard

Enable DIANNA to take images with multiple channels as input

Open elboyran opened this issue 2 years ago • 2 comments

DIANNA should be able to process images as inputs and models trained with more than 3 (e.g. RGB) channels.

Inspired by the Lorentz workshop CNN-post-processing use case. For example data and quick fix, see the link in the Lorenz code usecase branch by @gcroci2. Have in mind that the model in this code also accepts multiple (in this case 2) inputs. The latter is related to issue #333.

elboyran avatar Sep 08 '22 14:09 elboyran

This would be great indeed! I did some quick tests during the workshop: RISE works as-is with more than 3 channels, but it's good to know that it applies the same mask to every channel. LIIME doesn't work: the lime explainer (i.e. the external package we use) only accepts 3 channels. For greyscale we fixed this by duplicating the channels axis. For more than 3 channels we would need to submit a PR to lime or reimplement it ourselves. I haven't tested KernelSHAP.

loostrum avatar Sep 09 '22 07:09 loostrum

Also follows from #406

elboyran avatar Dec 01 '22 09:12 elboyran