Reported issue with bounding box query and rotated images

[LucaMarconato]

Reported via email by a user.

I contact you because of a minor issue, though. I have an image of a tissue section and I would like to crop an area of interest. The best way for me to select that area is actually by rotating the image and then just drawing a rectangle around. However, I find that the cropping is not performed on the rotated, new coordinates. Could you help me with any suggestion about how to address this issue?

I write down below the key code snippets, but happy to share with you a jupyter notebook with a reproducible example.

# Using data from registry transcriptomics/resolve/mouse/20272_slide1_A1-1_DAPI.tiff


OUTPUT_DIR = tempfile.gettempdir()
img_layer = "raw_image"
path = None
registry = get_registry(path=path)
path_image = registry.fetch("transcriptomics/resolve/mouse/20272_slide1_A1-1_DAPI.tiff")
zarr_path = os.path.join(OUTPUT_DIR, f"sdata_{uuid.uuid4()}.zarr")

sdata = sp.io.create_sdata(
   input=path_image,
   output_path=zarr_path,
   img_layer=img_layer,
   chunks=1024,

# Rotate
import math

import matplotlib.pyplot as plt
import spatialdata
import spatialdata_plot
from spatialdata.transformations import (
   Affine,
   MapAxis,
   Scale,
   Sequence,
   Translation,
   get_transformation,
   set_transformation,
)

theta = math.radians(45)
rotation = Affine(
   [
       [math.cos(theta), -math.sin(theta), 0],
       [math.sin(theta), math.cos(theta), 0],
       [0, 0, 1],
   ],
   input_axes=("x", "y"),
   output_axes=("x", "y"),
)

set_transformation(sdata.images["raw_image"], rotation, to_coordinate_system="rotation_test")

sdata.pl.render_images().pl.render_labels().pl.render_shapes().pl.show()

# Bounding box
bb_xmin = -4000
bb_ymin = 5000
bb_w = 2500
bb_h = 7000
bb_xmax = bb_xmin + bb_w
bb_ymax = bb_ymin + bb_h

f, ax = plt.subplots(figsize=(7, 7))
sdata.pl.render_images('raw_image').pl.show(ax=ax, coordinate_systems=['rotation_test'])
#sdata.pl.render_shapes().pl.show(ax=ax)
rect = patches.Rectangle((bb_xmin, bb_ymin), bb_w, bb_h, linewidth=5, edgecolor="red", facecolor="none")
ax.add_patch(rect)

sdata_cropped = sdata.query.bounding_box(
min_coordinate=[bb_xmin, bb_ymin],
max_coordinate=[bb_xmax, bb_ymax],
axes=("x", "y"),
target_coordinate_system="rotation_test")

# Cropping not performed as intended:
sdata_cropped.pl.render_images('raw_image').pl.show(coordinate_systems=['global', 'rotation_test'])

[LucaMarconato]

Two comments in my answer to the email (the next follow ups will be here in GitHub). I haven't tried the code yet.

These behavior could be the reason behind the reported issue.

1. a bounding box query on a rotated image will return the bounding box of the backward-transformed bounding box in the intrinstic coordinate system. This design is documented here: https://github.com/scverse/spatialdata/pull/151#issuecomment-1444609101.
2. if you use a polygon_query function (passing a rotated bounding box on the unrotated image) you will obtain the same result as above, because the polygon_query function will call the bounding_box query on the bounding box of the polygon.