Joosep Pata

The ray test now crashes with the following (it fails despite being called a warning): ``` DeprecationWarning: The environment variable `RAY_AIR_LOCAL_CACHE_DIR` is deprecated. It is no longer used and will...

When running evaluation of a trained model: ``` export CUDA_VISIBLE_DEVICES=MIG-632c6fca-30c4-5fce-97be-8dab51b1f2f6 singularity exec -B /scratch/persistent --nv \ --env PYTHONPATH=hep_tfds \ --env KERAS_BACKEND=torch \ $IMG python3.10 mlpf/pyg_pipeline.py --dataset cms --gpus 1 \...

Taking the CMS MINIAOD, investigate outlier / badly reconstructed events or particles and understand what can be done better. ``` $ gfal-ls root://xrootd.hep.kbfi.ee///store/user/jpata/mlpf/results/CMSSW_14_1_0_pre3/QCD_PU_pf $ gfal-ls root://xrootd.hep.kbfi.ee///store/user/jpata/mlpf/results/CMSSW_14_1_0_pre3/QCD_PU_mlpf_acat2022 ```

Previously in #206 we got pytorch post-training static quantization to work, but the model was not faster in inference, probably due to some missing ops on CPU/GPU in the pytorch...

- truth jets from CMSSW [here](https://github.com/jpata/cmssw/blob/pfanalysis_caloparticle_CMSSW_14_1_0_pre3/Validation/RecoParticleFlow/plugins/PFAnalysisNtuplizer.cc#L315) are defined as `ak4GenJet`, not `ak4GenJetsNoNu`, therefore they include unreconstructable neutrinos and bias the final plots - therefore, redefine truth jets in postprocessing2.py to...

For the CMS model, we have a per-particle `is_pu` target that is varying between 0 for particles from the hard event and 1 for particles from PU, and inbetween for...

Log in comet / tensorboard, and also final plots from predictions. 

Cluster jets for events in the batch, log plots to comet and tensorboard.