refactor: Vertex performance monitoring

[paulgessinger]

Previously, the vertex performance writer would assume that there's a 1:1 correspondence between fitted tracks and truth particles. This prevented to run it on track finding output, where we have many more tracks, esp. without ambiguity resolution.

This PR implements hit based truth matching between tracks and particles and uses that to define truth vertices, etc. I have to change the minimum track fraction to consider a vertex truth matched drastically, which I'm guessing is because of the aforementioned high duplicate count: if you have a ton of tacks, even correctly reconstructed tracks will likely have a bunch of fake tracks attached.

I also switched the physmon particle gun to 201 vertices with beamspot smearing (but the vertexing still runs without beamspot constraint). The IVF gives something like 50 vertices per event, while the AMVF only finds a handful.

Finally, I added a script to make histograms from the performance ntuple and now feed that into the physmon histogram comparison.

[paulgessinger]

While the CI histograms are still running, here are some histograms i made with HardQCD + SoftQCD in pythia:

Seems consistent to me with the observation that the efficiency with the default configuration at this time isn't great (#1362)

[codecov[bot]]

Codecov Report

Merging #1417 (4d51380) into main (5f4052a) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #1417   +/-   ##
=======================================
  Coverage   48.59%   48.59%           
=======================================
  Files         381      381           
  Lines       20631    20631           
  Branches     9463     9463           
=======================================
  Hits        10026    10026           
  Misses       4066     4066           
  Partials     6539     6539           

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

[paulgessinger]

Ah right, this currently also includes some extra stuff that I'll rip out and put into separate PRs.

[paulgessinger]

I slightly modified the ODD full chain example, to get some numbers as a function of pileup:

[paulgessinger]

Looking at the code a bit more, it seems like the AMVF in this scenario loses most of the vertex candidates because not tracks are compatible with the vertex after being added to the fit. This could be in fact a result of the high duplication rate that @andiwand was observing as a consequence of fakes influencing the vertex fit.

[paulgessinger]

Hm, I think the perfmon job takes a lot longer than before. I'm guessing this has to do with the increased pseudo-pileup count. I'll see if I can tune this to complete in a reasonable time while giving enough stats.

[paulgessinger]

Ok, I created #1460 which extracts some of the changes here. Once that's in, I'll update this and we can move ahead.

[paulgessinger]

I think this is finally good to merge, @asalzburger @andiwand.

[paulgessinger]

Ok references updates after rebase, can you approve again @andiwand ?

[acts-project-service]

The backport to develop/v19.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-develop/v19.x develop/v19.x
# Navigate to the new working tree
cd .worktrees/backport-develop/v19.x
# Create a new branch
git switch --create backport-1417-to-develop/v19.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 f5d312f98ad92f6b94d82bfac6108d38ce185799
# Push it to GitHub
git push --set-upstream origin backport-1417-to-develop/v19.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-develop/v19.x

Then, create a pull request where the base branch is develop/v19.x and the compare/head branch is backport-1417-to-develop/v19.x.