Fixes near2far projection for 2d simulations

[QimingFlex]

Implemented a preliminary fix for the 'near2far' in 2D. The key differences between 2D and 3D of 'near2far' in math are as follows:

1. Field Components - In 3D, they include Er, Ephi, Etheta, Hr, Hphi, Htheta; in 2D, for TMz mode, they are Ez and Hphi, and for TEz mode, Ephi and Hz.
2. Parameters - In 3D, these are theta, phi, and f; in 2D, phi and f.
3. Propagation phase.
4. RCS coefficient.

Given these differences, my initial idea was to develop a 2D variant of the FieldProjectionAngleMonitor, omitting the theta parameter and inheriting from the AbstractFieldProjectionMonitor. However, since the AbstractFieldProjectionMonitor is heavily designed for 3D features and depends on theta, it seems a 2D monitor would require a ground-up redesign. My current approach involves a rudimentary adaptation where the 3D monitor is used as 2D by setting theta = pi/2, and I added a 2D version of the far-field computation. This mirrors the 3D code with adjustments for the propagation phase and the RCS coefficient. I am relatively new to contributing to a large project, any comments and suggestions on how to more efficiently integrate these changes would be greatly appreciated.

[momchil-flex]

Thanks @QimingFlex!

I think it should be possible to reuse our monitors as they are. I think you haven't yet familiarized yourself with pydantic validators, which is what we use ubiquitously in the code to ensure the user can only set things the way we want them to. I think that's what we can use here and you should start looking through some examples of those in our code (@dmarek-flex could help if needed).

I think what you describe, setting theta = pi/2, works specifically because the simulation 0-dim is set to z. If we were to instead rotate everything such that e.g. the 0-dim is x, then we would do something like phi = 0 and scan theta from 0 to 2*pi right? So I think what we can do is

1. Use a validator to enforce that no theta is provided to the user if a 2D simulation. Explain in the docstring that in 2D simulations, only phi is needed, and is always in the plane of the simulation.
2. Under the hood, convert the phi-s provided by the user, with the definition above, to a (theta, phi) to be passed to a regular 3D monitor.
3. Finally reframe the data back to that reference frame.

I think this can be done with the field data too. We don't necessarily want to introduce whole new objects to store the 2D data. We can use the same, and Etheta and Htheta will just always be zeros. We also don't currently explicitly split 2D sims into TE or TM. Depending on the exciting source, those could even be mixed. So it would just be again a matter of rotating everything from the global reference frame where z is the polar axis, to the 2D refarence frame that only has r and phi. Does that make sense?

[QimingFlex]

Thanks for the detailed explanation and guidance @momchil-flex. I'll start with the pydantic validators and consult Damian. Your approach to simulations and data transformations is clear and helpful!

[momchil-flex]

Actually, on second thought I feel like everything should be handled internally. That is to say, there will be no difference in the reference frame used from the user perspective depending on whether the simulation is 2D or 3D. The reason why I'm thinking this may make the most sense is so that users easily get consistent results between 2D and 3D sim setups. For example here I want to be able to just change the simulation size along y to 0, nothing else, and have everything working. That means that in the 2D sim I will still get the far field w.r.t. theta, and theta is defined w.r.t. the z axis.

I feel like from a usability perspective this is maybe better than requiring the user to rotate the axes based on the simulation 0-sized-dim? @tomflexcompute any input here?

[tomflexcompute]

Actually, on second thought I feel like everything should be handled internally. That is to say, there will be no difference in the reference frame used from the user perspective depending on whether the simulation is 2D or 3D. The reason why I'm thinking this may make the most sense is so that users easily get consistent results between 2D and 3D sim setups. For example here I want to be able to just change the simulation size along y to 0, nothing else, and have everything working. That means that in the 2D sim I will still get the far field w.r.t. theta, and theta is defined w.r.t. the z axis.

I feel like from a usability perspective this is maybe better than requiring the user to rotate the axes based on the simulation 0-sized-dim? @tomflexcompute any input here?

Yeah totally agree.

[momchil-flex]

On the flip side though if someone is doing scattering off a Cylinder they would have to set different settings in the projection monitor (different angles, specifically) depending on which axis is the zero-size dim. Still, I feel like this is the better way.

[QimingFlex]

Hi @momchil-flex , I just updated the revision. After some thought, I believe it's better to conduct near-to-far in the x-y plane for the following reasons:

1. This conforms to the standards of cylindrical coordinates where phi starts from the x-axis and the z-component is along the infinite direction. I guess it's less common to consider a case where phi starts from the y-axis with the x-component along the infinite direction?
2. Such a revision would require much more effort to me at this moment as I still need some more time to get familiar with the code. Perhaps the directivity monitor will help me understand the code more so this won't be a problem later.

In this revision, the simulation size along the z-direction determines whether it's a 2D or 3D simulation. If a 2D simulation is determined, theta is set to pi/2, and the propagation constant is related to the Hankel function. These computations are handled internally. However, I encountered an issue with computing the RCS. It seems related to the monitor, which is a 3D monitor and cannot determine if it's a 2D or 3D simulation. To address this, I introduced radar_cross_section_2d.

[momchil-flex]

Thank you for the comments, @momchil-flex ! To achieve that goal for a general case, I'm considering how to modify the monitor if we allow 0-dim for every dimension. For example, Ez/Hphi and Hz/Ephi are generated when z is 0-dim, Ey/Hphi and Hy/Ephi are generated when y is 0-dim, and the same applies for x being 0-dim. Therefore, we would need to introduce new fields in the field monitor to consider all these three cases. Is this correct?

You should use the exact same data structures as for the 3D monitors, i.e. Er, Ephi, Etheta, with the polar coordinates defined w.r.t. the global z axis. For e.g. a 2D sim in the xy-plane then, you would have theta = pi / 2 and what you call Ez and Hphi for a TM sim would actually be Etheta and Hphi. All the other 3D components will be zero. If the plane is oriented along yz, then phi is set to 0, and what would be now Ex/Hphi in a purely 2D case correspond to Ephi/Htheta of the 3D data structure. For an xz plane, phi = pi/2 and Ey/Hphi again correspond to Ephi/Htheta. Try to visualize this and let me know if it makes sense to you.

I mean, as I'm writing this I do realize that it sounds convoluted. But like I mentioned above, the nice thing would be that I can just swap 2D and 3D sims without needing to change anything else in my scripts, i.e. take a 3D sim and choose to make one of the axes 0D and compare the projection results directly. I am not sure how much this complicates everything internally. Like, I feel like it should be possible to write the 2D projection as a special case of the 3D projection, and not have to do too much going back and forth between different coordinates. Conceptually, it should be possible to write the Green's function for a 2D sim with arbitrary normal axis as a 3D Green's function in the polar coordinates with a global z axis, i.e. in principle you can still write G(x, y) as a G(r, theta, phi), assuming G(x, y) is just constant with z. Now, practically there are a couple extra steps to actually put this in the code, but I still think eventually producing results in the same coordinate system as the 3D monitors would be best.

[tylerflex]

What's the status of this PR? is it for 2.7? @QimingFlex @weiliangjin2021

[QimingFlex]

Thank you for the comments, @momchil-flex ! To achieve that goal for a general case, I'm considering how to modify the monitor if we allow 0-dim for every dimension. For example, Ez/Hphi and Hz/Ephi are generated when z is 0-dim, Ey/Hphi and Hy/Ephi are generated when y is 0-dim, and the same applies for x being 0-dim. Therefore, we would need to introduce new fields in the field monitor to consider all these three cases. Is this correct?

You should use the exact same data structures as for the 3D monitors, i.e. Er, Ephi, Etheta, with the polar coordinates defined w.r.t. the global z axis. For e.g. a 2D sim in the xy-plane then, you would have theta = pi / 2 and what you call Ez and Hphi for a TM sim would actually be Etheta and Hphi. All the other 3D components will be zero. If the plane is oriented along yz, then phi is set to 0, and what would be now Ex/Hphi in a purely 2D case correspond to Ephi/Htheta of the 3D data structure. For an xz plane, phi = pi/2 and Ey/Hphi again correspond to Ephi/Htheta. Try to visualize this and let me know if it makes sense to you.

I mean, as I'm writing this I do realize that it sounds convoluted. But like I mentioned above, the nice thing would be that I can just swap 2D and 3D sims without needing to change anything else in my scripts, i.e. take a 3D sim and choose to make one of the axes 0D and compare the projection results directly. I am not sure how much this complicates everything internally. Like, I feel like it should be possible to write the 2D projection as a special case of the 3D projection, and not have to do too much going back and forth between different coordinates. Conceptually, it should be possible to write the Green's function for a 2D sim with arbitrary normal axis as a 3D Green's function in the polar coordinates with a global z axis, i.e. in principle you can still write G(x, y) as a G(r, theta, phi), assuming G(x, y) is just constant with z. Now, practically there are a couple extra steps to actually put this in the code, but I still think eventually producing results in the same coordinate system as the 3D monitors would be best.

I just went through the code for both Monitors and MonitorData today and got a better understanding of the monitor classes. What you posted makes sense to me. I will test all three 2D scenarios to ensure we get consistent results.

[QimingFlex]

What's the status of this PR? is it for 2.7? @QimingFlex @weiliangjin2021

It still needs some time to be finished. I'm not sure if it is for 2.7, any ideas? @tomflexcompute @momchil-flex

[tomflexcompute]

What's the status of this PR? is it for 2.7? @QimingFlex @weiliangjin2021

It still needs some time to be finished. I'm not sure if it is for 2.7, any ideas? @tomflexcompute @momchil-flex

If it's too much of a rush, I think we can certainly leave it for a later version release or pre-release.

[tylerflex]

Ok, for now I'll tag it for 2.8 and mark as a draft just to focus on the PRs that are in their final stages.

[QimingFlex]

@momchil-flex I've fixed the code for a general 2D farfield projection and tested it with RCS for all three scenarios in 2d. For a 0-dim along the x-direction, users should set φ = π/2 in the farfield monitor to get the desired farfield. For a 0-dim along the y-direction, users should set φ = 0 . For a 0-dim along the z-direction, users should set θ = π/2. This assumes users are familiar with the standard definition of spherical coordinates.

I have one question regarding the RCS calculation, which is a class method of the AbstractFieldProjectionData. In this case, I haven't found a way to determine if it is a 2D or 3D simulation based solely on the data class. Do you have any ideas on this?

[momchil-flex]

@momchil-flex I've fixed the code for a general 2D farfield projection and tested it with RCS for all three scenarios in 2d. For a 0-dim along the x-direction, users should set φ = π/2 in the farfield monitor to get the desired farfield. For a 0-dim along the y-direction, users should set φ = 0 . For a 0-dim along the z-direction, users should set θ = π/2. This assumes users are familiar with the standard definition of spherical coordinates.

Nice! We should have a Simulation level validator that checks this and explains it to the user if it is not set correctly. Namely, if simulation is 2D, iterate over monitors, and for every field projection monitor make sure that the angles are set correctly. Checking if the simulation is 2D is a bit tricky on the validator level actually as you don't have access to the grid as the Simulation is not yet fully initialized. So it could be just a size == 0 along some direction check. If you want to use sim.grid.num_cells == 0 then you have to have it as a pre-upload validator, which is also possible, but not sure if needed. Generally, I think we prefer 2D sims to be set with a size strictly 0. But that may not be what users always do... and I think in the case of Bloch boundaries along the 0 dim, we actually do currently need a non-zero size to be set.

I have one question regarding the RCS calculation, which is a class method of the AbstractFieldProjectionData. In this case, I haven't found a way to determine if it is a 2D or 3D simulation based solely on the data class. Do you have any ideas on this?

You could add a 2d_normal_axis: Axis = None Field to the data structure to store whether this was associated with a 2D sim, and which axis was normal to it?

[momchil-flex]

I feel like to avoid confusion we shouldn't merge this before we also fix the server-side projections. Those are much more commonly used anyway. @dmarek-flex now has very good understanding of how they work, maybe you can work together to see what will need to change, at least to scope it to decide whether it will be a lot of effort or not, and we can decide what to do.

Note that both the exact and the approximate projections will need to be fixed there.

[QimingFlex]

I feel like to avoid confusion we shouldn't merge this before we also fix the server-side projections. Those are much more commonly used anyway. @dmarek-flex now has very good understanding of how they work, maybe you can work together to see what will need to change, at least to scope it to decide whether it will be a lot of effort or not, and we can decide what to do.

Note that both the exact and the approximate projections will need to be fixed there.

I agree. I will prioritize the server-side computation.

[QimingFlex]

@momchil-flex I added validators in the frontend. With this fix, users can compute far field components Ephi, Etheta, Hphi, and Htheta in 2D. There are two remaining tasks for a full fix:

1. Frontend computation.
2. Update the monitor data class to indicate a 2D simulation for the correct constant for RCS computation in 2d.

[QimingFlex]

Hi @momchil-flex So far, we have fixed the FieldAngleProjectionMonitor for both frontend and backend with far-field approximation. For the remaining work, I propose the following:

1. Updates to Cartesian space and k-space (any references on k-space would be helpful).
2. Solve for the exact projection.

[momchil-flex]

Hi @momchil-flex So far, we have fixed the FieldAngleProjectionMonitor for both frontend and backend with far-field approximation. For the remaining work, I propose the following:

1. Updates to Cartesian space and k-space (any references on k-space would be helpful).
2. Solve for the exact projection.

Sounds good. We already validate 2. for an angle projection monitor. We should validate out 1. if it's not yet supported, i.e. in your validator you should add a check if k-space or cartesian monitor + 2D sim and error out.

Regarding k-space, it's conceptually exactly the same as angle, except the angles are defined not on a uniform grid in (theta, phi), but through a uniform grid of (kx, ky). The main difference is that in the k-space definition the angles are frequency dependent as they are basically determined through kx, ky and |k| = omega * n / c through simple trigonometry. Does that clear it up for you?

[momchil-flex]

Having said that it seems like making the 2D KSpace monitors work should be quite simple, but see if you want to try it right away or keep it for later (and validate them out then).

[QimingFlex]

Regarding k-space, it's conceptually exactly the same as angle, except the angles are defined not on a uniform grid in (theta, phi), but through a uniform grid of (kx, ky). The main difference is that in the k-space definition the angles are frequency dependent as they are basically determined through kx, ky and |k| = omega * n / c through simple trigonometry. Does that clear it up for you?

Thanks. I have one more question regarding the proj_axis for both Cartesian space and k-space. The docstring says it is normal to the monitor. In the example in the docstrings, which uses a box monitor with proj_axis = 2, how does this proj_axis relate to the monitor? Since the monitor has 6 faces, only 2 of them have normals in the z-direction?

[QimingFlex]

Having said that it seems like making the 2D KSpace monitors work should be quite simple, but see if you want to try it right away or keep it for later (and validate them out then).

Let's keep that for later? For now, let's check if k-space or Cartesian monitor + 2D sims is used, and handle any errors accordingly.

[QimingFlex]

@weiliangjin2021 It's validated here within the loop for monitors.