Length of scale parameter inconsistent with RGB images

[tlambert03]

🐛 Bug Report

not a bug per-se, but perhaps surprising in some cases. It's usually ok to give a layer scale of [1] * data.ndim, however, if the array has a trailing shape of 3 or 4, it will get auto-interpreted as a RGB, and then a scale matching the length of the data shape will raise an exception

💡 Steps to Reproduce

import napari
import numpy as np

data = np.empty((2, 128, 128, 3))

viewer = napari.Viewer()
viewer.add_image(data, scale=[1.0] * data.ndim)  # fine if we explicitly provide rgb=False

napari.run()

Traceback (most recent call last):
  File "/Users/talley/dev/self/napari-micromanager/x.py", line 7, in <module>
    viewer.add_image(data, scale=[1.0] * data.ndim)
  File "/Users/talley/miniforge3/envs/napari-micromanager/lib/python3.11/site-packages/napari/utils/migrations.py", line 101, in _update_from_dict
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/talley/miniforge3/envs/napari-micromanager/lib/python3.11/site-packages/napari/components/viewer_model.py", line 989, in add_image
    layer = Image(data, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/talley/miniforge3/envs/napari-micromanager/lib/python3.11/site-packages/napari/layers/base/base.py", line 114, in __call__
    obj = super().__call__(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/talley/miniforge3/envs/napari-micromanager/lib/python3.11/site-packages/napari/utils/migrations.py", line 101, in _update_from_dict
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/talley/miniforge3/envs/napari-micromanager/lib/python3.11/site-packages/napari/layers/image/image.py", line 277, in __init__
    super().__init__(
  File "/Users/talley/miniforge3/envs/napari-micromanager/lib/python3.11/site-packages/napari/layers/intensity_mixin.py", line 31, in __init__
    super().__init__(*args, **kwargs)
  File "/Users/talley/miniforge3/envs/napari-micromanager/lib/python3.11/site-packages/napari/layers/_scalar_field/scalar_field.py", line 224, in __init__
    super().__init__(
  File "/Users/talley/miniforge3/envs/napari-micromanager/lib/python3.11/site-packages/napari/layers/base/base.py", line 420, in __init__
    CompositeAffine(
  File "/Users/talley/miniforge3/envs/napari-micromanager/lib/python3.11/site-packages/napari/utils/transforms/transforms.py", line 853, in __init__
    self._scale = scale_to_vector(scale, ndim=ndim)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/talley/miniforge3/envs/napari-micromanager/lib/python3.11/site-packages/napari/utils/transforms/transform_utils.py", line 138, in scale_to_vector
    scale_arr[-len(scale) :] = scale
    ~~~~~~~~~^^^^^^^^^^^^^^^
ValueError: could not broadcast input array from shape (4,) into shape (3,)

💡 Expected Behavior

dunno... maybe swallow the last element in the scale array if it's off by one, or consider a more specific exception with auto-detected RGB images and a scale of length data.ndim + 1?

🌎 Environment

napari: 0.5.3 Platform: macOS-15.0-arm64-arm-64bit System: MacOS 15.0 Python: 3.11.10 | packaged by conda-forge | (main, Sep 22 2024, 14:11:13) [Clang 17.0.6 ] Qt: 6.7.1 PyQt6: NumPy: 2.1.1 SciPy: 1.14.1 Dask: 2024.9.0 VisPy: 0.14.3 magicgui: 0.9.1 superqt: 0.6.7 in-n-out: 0.2.1 app-model: 0.2.8 npe2: 0.7.7

OpenGL:

• GL version: 2.1 Metal - 89.3
• MAX_TEXTURE_SIZE: 16384
• GL_MAX_3D_TEXTURE_SIZE: 2048

Screens:

• screen 1: resolution 2560x1440, scale 2.0

Optional:

• numba not installed
• triangle not installed
• napari-plugin-manager not installed

Settings path:

• /Users/talley/Library/Application Support/napari/napari-micromanager_63091fb16e63a43cabe1f8066a250f46de5827f3/settings.yaml Plugins:
• napari: 0.5.3 (81 contributions)
• napari-console: 0.0.9 (0 contributions)
• napari-micromanager: 0.1.3.dev0+g31d973d.d20240926 (2 contributions)
• napari-svg: 0.2.0 (2 contributions)

💡 Additional Context

No response

[Czaki]

next place to disable magic

[DragaDoncila]

dunno... maybe swallow the last element in the scale array if it's off by one,

I would really hate this 😆

consider a more specific exception with auto-detected RGB images and a scale of length data.ndim + 1?

I think we should throw a specific exception here and ask that the user split up the image themselves if they wish to set scale. I think asking them to pass a scale but omit the last dim is wild...

[tlambert03]

i would really hate this 😆

Yep, it's more magic, but not all that different from magically inferring RGB from a trailing size of 3. It's just that it's done for data but not scale, it's like "half magic"

ask that the user split up the image themselves if they wish to set scale.

Wait... disallow scale on RGB? I would say require the user to set RGB=true if you don't want to do "full" magic on RGB-from-shape.

[DragaDoncila]

Yep, it's more magic, but not all that different from magically inferring RGB from a trailing size of 3

Yeah, I mean, I kinda don't love that either, it's just that deprecating it is going to be a massive pain.

Wait... disallow scale on RGB? I would say require the user to set RGB=true if you don't want to do "full" magic on RGB-from-shape.

Yeah ok, I guess I would be fine with the exception saying something like

We are interpreting this image as RGB but you have passed scale for all dimensions. 
To correctly set the scale, set `rgb=True` and pass scale values excluding the final dimension.

[jni]

I'm a little confused about the desired behaviour here... @tlambert03 (👋!) are you saying that you want your invocation to Just Work? Or that you want the image to not be interpreted as RGB? My initial impression is that it's straightforwardly user error to pass in too many scales for the dimensionality of the image. As a thought experiment, what if the value was not 1 for that dimension — would you still expect it to be silently swallowed?

But of course the error message can be much improved.

[tlambert03]

I said "dunno" cause I'm really not sure what I think it should do, but I think if it "just works" to change a 6d data array to a 5d RGB source without any additional input, then perhaps those other shaped inputs should go along with that? And if not, then the exception could explain that the reason for the exception is that the last dimension of the input data has been automatically changed, and the user needs to change the other shaped inputs accordingly

[tlambert03]

straightforwardly user error to pass in too many scales for the dimensionality of the image.

Absolutely true, but the user didn't pick RGB here either, and the length of scales matches the length of the data

[jni]

Absolutely true, but the user didn't pick RGB here either, and the length of scales matches the length of the data

Yeah, I'm ok with this argument except the user (in this case you) actually wants us to interpret the layer as RGB, so if they explicitly override our default scale (we are of course happy to give you a scale of 1 of the appropriate dimensionality by default), then I think it's reasonable for us to assume that they knew what they were doing and that that scale argument should match our expectation. Like, if you were saying "turn off this abominable auto-RGB so I don't see this error", I'd be generally fine with that argument.

Anyway, I think the consensus here is to make a better error message. However the dimension check is super deep in the base layer so it's a bit tricky — it looks like we'd have to wrap a try-except around the whole super().__init__ of the Image layer (here), which is a bit nasty.