Add accelerator lattice, starting with quadrupoles

[dpgrote]

This PR is the initial implementation of an accelerator lattice, starting with hard edge quadrupoles. The implementation follows the scheme in Warp, where the lattice is laid out along the z axis. Each lattice element is specified by an extent in z and information about its field structure.

To locate which element particles are in, a grid is laid out in z and at each grid cell, the index of the nearest element is saved. Then for each particle, a look up is done from the table. This grid is updated as needed in a moving frame and with load balancing.

For the GPU, host level and device level classes are defined. The data is read in and setup in the host level class and the needed data copied to the GPU. The device level class only holds pointers to the needed data. This hasn't been tested but does compile.

To do:

• [x] Add a CI test in the moving window
• [x] Test on GPU
• [x] Add handling of the boosted frame
• [x] Add plasma lens using the new lattice infrastructure
• [x] Add documentation

This PR assumes that elements within a type do not overlap each other. A future PR can be added to allow the overlapping.

[lgtm-com[bot]]

This pull request introduces 2 alerts when merging fa215019b7e40bd4237f76e9aa9496a83b8d9591 into 8eb3163d865fbabb22c227b8f1810db079cca8f3 - view on LGTM.com

new alerts:

• 1 for Unused import
• 1 for Variable defined multiple times

[EZoni]

@dpgrote Could you please rebase on development in order to include the bug fix of #3253, so that we can proceed with testing this PR? Thank you!

[EZoni]

@dpgrote The commit history might have been corrupted by the last merge, would you have time to check if this is the case? I see many more lines changed than before.

[dpgrote]

I merged again and it seems to have cleaned up the problem.

[dpgrote]

Thanks for your comments @EZoni ! For your second question, perhaps to provide full coverage, an RZ test could be added, but I don't think it is necessary. The code treats the particles as if they are in 3D Cartesian space and the dimensionality of the rest of the code doesn't come into play. The lattice only sees the particles using GetParticlePosition, so it doesn't know anything about the dimensionality they might have and only has their (x,y,z) coordinate.

[RTSandberg]

I used the accelerator in one case, with the attached input deck (or something very close). inputs_lattice.txt

In release mode, I find the following timings running on 1 node on Summit: lattice, this PR: 0.025 s / step, 158.4 s total

No lattice, this PR: 0.0245 s / step, 153 s total

No lattice, dev branch: 0.0243 s / step, 152 s total

In debug mode, I found that this PR was 10x slower (0.5 s / step) than the current development branch

[dpgrote]

Thanks for the timings @RTSandberg. It's nice to know that timing was not affected much with the new code added when optimized.

Did you see any indication about which part of the code is slowing things down when running in debug mode?

[ax3l]

Let's take this up once #3402 is merged, so we can avoid runtime costs if an accelerator lattice is not used.