Incorrect Visualization & Question about ‘read’ function

[Hooray-Lee]

Hi！ Thanks a lot for the excellent work! I am writing to report 3 issues I encountered while using your project.

Issue 1: I used the following specific parameters for DRR projection, and I specified the translation y as 595. Why is the translation of the camera relative to the volume data manifested on the x-axis?

    rot = torch.tensor([[90.0, 0.0, 0.0]], device=device) / 180 * torch.pi
    xyz = torch.tensor([[0.0, 595.0, 0.0]], device=device)
    pose = convert(rot, xyz, parameterization="euler_angles", convention="ZXY")

Issue 2: I tried to test the results of perspective_projection. It seems not right. Did I do it wrong?

    points = torch.tensor([[[0., 0., 0.,],
                        [-100., 0., 0.]]], device=device)
    points_det = drr.perspective_projection(pose, points)
    print(points_det)
========================================
    tensor([[[150.0000, 150.0000],
             [150.0000, 150.0000]]], device='cuda:0')

Issue 3: I don't understand what AP and PA specifically represent in the 'read' function? Why is it necessary to rotate around two coordinate axes? And which coordinate system are the coordinate axes referring to here? Can you please help me explain this?

    # Frame-of-reference change
    if orientation == "AP":
        # Rotates the C-arm about the x-axis by 90 degrees
        # Rotates the C-arm about the z-axis by -90 degrees
        reorient = torch.tensor(
            [
                [0.0, 1.0, 0.0, 0.0],
                [0.0, 0.0, -1.0, 0.0],
                [-1.0, 0.0, 0.0, 0.0],
                [0.0, 0.0, 0.0, 1.0],
            ]
        )
    elif orientation == "PA":
        # Rotates the C-arm about the x-axis by 90 degrees
        # Rotates the C-arm about the z-axis by 90 degrees
        reorient = torch.tensor(
            [
                [0.0, 1.0, 0.0, 0.0],
                [0.0, 0.0, 1.0, 0.0],
                [-1.0, 0.0, 0.0, 0.0],
                [0.0, 0.0, 0.0, 1.0],
            ]
        )
    elif orientation is None:
        # Identity transform
        reorient = torch.tensor(
            [
                [1.0, 0.0, 0.0, 0.0],
                [0.0, 1.0, 0.0, 0.0],
                [0.0, 0.0, 1.0, 0.0],
                [0.0, 0.0, 0.0, 1.0],
            ]
        )

Thank you for your time and consideration.

[eigenvivek]

Hi @Hooray-Lee , sorry for the slow reply.

Issue 1

That result is surprising. What's the affine matrix for your volume? Also what orientation are you using in read? If you use AP, I would expect the camera to be drawn back along the y-direction

Issue 2

The output of perspective_projection is 2D points that can be plotted on the DRR you rendered. What leads you to believe those outputs are wrong?

Issue 3

• AP means the X-ray camera is facing the anterior side of the CT
• PA means the X-ray camera is facing the posterior side of the CT

It's illustrative to use orientation=None to see why this argument is useful. By default, the X-ray camera points in the positive Z-direction. For most CTs, this is akin to point from the caudal-to-cranial axis, which does not produce useful DRRs. Since we usually want frontal/lateral projections, AP and PA reorients the coordinate frame in a way such that the Euler angle pose inputs are human-readable. This way, pose parameters in DiffDRR directly correspond to radiologic conventions (e.g., if you've ever heard the term LAO45, this setup makes it very easy to parameterize such a pose in DiffDRR with minimal effort).

I implemented both AP and PA for cases when patients are supine and prone, respectively.