Integrate LLAMA

[bernhardmgruber]

This PR integrates LLAMA for handling the memory layout of several core data structures.

Open tasks:

• [ ] Integrate the LLAMA library as a subtree. For this, run the command: GIT_AUTHOR_NAME="Third Party" GIT_AUTHOR_EMAIL="[email protected]" git subtree pull --prefix thirdParty/llama [email protected]:alpaka-group/llama.git develop --squash
• [x] ~Pull upgrade to Boost 1.70 into separate PR~ (done in #4144)
• [x] Find a better way to choose which memory layout to use in the param files
• [x] Make IO compile
• [x] Test IO works
• [ ] Test performance is the same as before

After checking out this branch, you need to run git submodule update to get LLAMA.

[BrianMarre]

Also please note that PIConGPU already moved on to boost 1.74

[bernhardmgruber]

Also please note that PIConGPU already moved on to boost 1.74

Awesome, I updated the PR description!

[BrianMarre]

@sbastrakov or @psychocoderHPC if you a little bit of free time could you take a look, I am not yet qualified to review this ;)

[BrianMarre]

@bernhardmgruber you also should execute something like, after having load/installed clang format, (or an llvm packet containing it)

find share test include -iname "*.def" -o -iname "*.h" -o -iname "*.cpp" -o -iname "*.cu" -o -iname "*.hpp" -o -iname "*.tpp" -o -iname "*.kernel" -o -iname "*.loader" -o -iname "*.param" -o -iname "*.unitless" | xargs clang-format -i

on your pull request to do a clang format, and fix the formatting errors the CI is complaining about.

Note: clang-fromat version is important, PIConGPU uses 12.0.1

[sbastrakov]

I believe the plan is that @bernhardmgruber comes for some time to Dresden relatively soon to finish integratation of LLAMA to PIConGPU.

[bernhardmgruber]

I will resume on this work starting on the 28th of November. I will be at HZDR for two weeks to complete this work.

[bernhardmgruber]

So, I updated the branch to LLAMA develop and picongpu dev. I can compile the SPEC example again.

[psychocoderHPC]

could you run clang-format-12 that we can see what the CI is saying

find src/include/mallocMC example share tests include test  -iname "*.def" \
  -o -iname "*.h" -o -iname "*.cpp" -o -iname "*.cu" \
  -o -iname "*.hpp" -o -iname "*.tpp" -o -iname "*.kernel" \
  -o -iname "*.loader" -o -iname "*.param" -o -iname "*.unitless" \
  | xargs clang-format-12 -i

[bernhardmgruber]

could you run clang-format-12 that we can see what the CI is saying

I know, I only have clang-format-15 locally on Windows. On my Ubuntu 22.10, it is also no longer available:

bgruber@dixxi3:/mnt/c/dev/picongpu$ sudo apt install clang-format-12
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package clang-format-12

LLAMA apt repository only has versions dev, 15 and 14 (secretly, the older ones might still be reachable, I did not check).

How about you update to clang-format-14? :)

[psychocoderHPC]

could you run clang-format-12 that we can see what the CI is saying

I know, I only have clang-format-15 locally on Windows. On my Ubuntu 22.10, it is also no longer available:

bgruber@dixxi3:/mnt/c/dev/picongpu$ sudo apt install clang-format-12
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
E: Unable to locate package clang-format-12

LLAMA apt repository only has versions dev, 15 and 14 (secretly, the older ones might still be reachable, I did not check).

How about you update to clang-format-14? :)

Unfortunately updating the formation requirement has low priority. The only nice feature which would come with clang-format-14 is the east cost const.

At the moment lifting the requirements is also hard because we have a few students currently trying to bring their extensions to PIConGPU they developed. Switching now the code style would maybe introduce too much pain for them.

[psychocoderHPC]

I can pull your branch and focre push the formatted version if you like.

[bernhardmgruber]

I can pull your branch and focre push the formatted version if you like.

We can do that once I am finished with everything else. I am actively working on this branch ATM to make the IO work as well.

[BrianMarre]

Our dev server still has a clang-format 12.0.1, in general you can install/load it using spack via the corresponding llvm pack

Also be aware that picongpu is even patch level sensitive, you need clang-format 12.0.1 specifically

[bernhardmgruber]

@psychocoderHPC informed me that there is currently a bug in the IO on dev. I am blocked on the IO by this bug, since the LLAMA integration is basically done.

[PrometheusPi]

@bernhardmgruber The dev now contains a fix to the IO bug #4385 (and possibly #4386). You can run your tests.

[bernhardmgruber]

@bernhardmgruber The dev now contains a fix to the IO bug #4385 (and possibly #4386). You can run your tests.

Thank you, I rebased my branch.

[PrometheusPi]

@bernhardmgruber I think we still have an issue #4392

[bernhardmgruber]

Can I prevent the CI from running if I push changes, but keep this PR open?