AliceO2 icon indicating copy to clipboard operation
AliceO2 copied to clipboard

[EMCAL-788] EMCAL cell selection, on-the-fly calibration

Open mfasDa opened this issue 1 year ago • 14 comments

  • Selection of good EMCAL cells
    • Require FEE readout (HG and LG cells)
    • Time cut around trigger peak
    • Min. cell energy
  • On-the-fly recalibration
    • Optional
    • Only in synchronous workflow

mfasDa avatar Aug 31 '23 12:08 mfasDa

Needs https://github.com/AliceO2Group/AliceO2/pull/11844

mfasDa avatar Aug 31 '23 12:08 mfasDa

@jmyrcha Please review!

mfasDa avatar Aug 31 '23 12:08 mfasDa

On-the-fly calibration should run in the synchronous stage while the it must not run in the asynchronous stage: In the asynchronous stage we apply the calibrations as part of the asynchronous reconstruction, consequently the event display will get fully calibrated cells and does not need to calibrate internally. In the synchronous stage we don't want to run the recalibration workflow because we want to make sure that the calibrated cells don't get mixed with the uncalibrated cells by some wildcarded data request, since we want to store uncalibrated cells persistently (in the synchronous stage we work with an a-priori calibration).

mfasDa avatar Aug 31 '23 12:08 mfasDa

Error while checking build/O2/fullCI for 16ac3d62bc14d2f9f1e13df5d3ca7f5a6ecf6c33 at 2023-09-01 13:15:

## sw/BUILD/O2-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'


## sw/BUILD/O2Physics-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[0 more errors; see full log]

Full log here.

alibuild avatar Aug 31 '23 15:08 alibuild

Error while checking build/O2/fullCI for 9b0a4c569c8e29435349d1eebd6898d2d6f524af at 2023-10-04 01:20:

## sw/BUILD/O2-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'


## sw/BUILD/O2Physics-latest/log
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[ERROR] See the example of usage in Tutorials/src/usingPDGService.cxx.
[ERROR] Function RecoDecay::getMassPDG is deprecated and will be removed soon.
[ERROR] Please use the Mass function in the O2DatabasePDG service instead.
[0 more errors; see full log]

Full log here.

alibuild avatar Sep 01 '23 15:09 alibuild

I have seen that there is a way to do the rejection of low-E cells in the visualization part, so this would be optional for the producer and could be removed. Not handled however and still important is the time cut. For this I attached two plots from our QC from a run with single bunches that demonstrates the effect. Uncalibrated: CellTimeEnergy_uncalibrated ... and calibrated: CellTimeEnergy_calibrated One sees here that a substantial amount of hot cells fires at time far from the trigger peak however at energies sufficiently high so that they are not removed by the energy cut.

For what concerns the on-the-fly recalibration: The amount of operations is one subtraction, one multiplication and one check for status, in each cases the coefficients are taken from an array or bitfield - the CPU overhead should be moderate. Furthermore the way it is implemented allows to switch it off via a command line argument.

mfasDa avatar Sep 04 '23 11:09 mfasDa

Error while checking build/O2/fullCI for a0e4311a377193295bcffa41b539f6d7fec2026a at 2023-12-08 13:00:

## sw/BUILD/O2-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'


## sw/BUILD/QualityControl-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'


## sw/BUILD/O2-full-system-test-latest/log
Detected critical problem in logfile reco_NOGPU.log
reco_NOGPU.log:[FATAL] error while setting up workflow in o2-eve-export-workflow: missing option: emcal-max-celltime
[FATAL] error while setting up workflow in o2-eve-export-workflow: missing option: emcal-max-celltime


## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ GRERR=0
++ grep -v clang-diagnostic-error error-log.txt
++ grep ' error:'
++ GRERR=1
++ [[ 1 == 0 ]]
++ mkdir -p /sw/INSTALLROOT/9cdf24eaefed9073a479038783adabc3b5ba4b9a/slc8_x86-64/o2checkcode/1.0-local407/etc/modulefiles
++ cat
--

Full log here.

alibuild avatar Oct 31 '23 19:10 alibuild

@mfasDa : what is the status of this PR? Is it still needed? Apparently it makes the FST crash.

davidrohr avatar Dec 08 '23 08:12 davidrohr

@davidrohr As can be seen above it is pending in review from @jmyrcha. Currently the event display neither uses the bad channel map nor does it reject pileup via a time cut, would be advisable to do so. As I explained above the application of the online calibration should be done in the components itself in order to not have online-calibrated cells entering the CTF.

mfasDa avatar Dec 08 '23 08:12 mfasDa

I don't fully understand the error message, since the option names match. The only issue I could imagine is an implicit cast issue due to the missing f in the default value specification for the new float options, which I just fixed.

mfasDa avatar Dec 08 '23 17:12 mfasDa

@mfasDa : could you revert the CMake formatting changes, and only add your change? Or if the formatting change came from running the O2 CMaker formatter, it should go into a seperate commit.

@jmyrcha : Could you check if you have objections? Otherwise I'll merge it once formatting changes are fixed.

davidrohr avatar Dec 13 '23 13:12 davidrohr

Error while checking build/O2/fullCI for e9e9f3ec8cbd0ab510343415baa8e7bd506392df at 2023-12-16 12:28:

## sw/BUILD/O2-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'


## sw/BUILD/QualityControl-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'


## sw/BUILD/O2-full-system-test-latest/log
command /sw/slc8_x86-64/O2/11845-slc8_x86-64-local28/prodtests/full-system-test/dpl-workflow.sh had nonzero exit code 1


## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ GRERR=0
++ grep -v clang-diagnostic-error error-log.txt
++ grep ' error:'
++ GRERR=1
++ [[ 1 == 0 ]]
++ mkdir -p /sw/INSTALLROOT/c49f6c2ae9ebcfcd629ef3506bbd49ff84f34265/slc8_x86-64/o2checkcode/1.0-local514/etc/modulefiles
++ cat
--

Full log here.

alibuild avatar Dec 15 '23 04:12 alibuild

Hi @davidrohr, indeed my VSCode autoformat settings applied the cmake formatter when saving the file after changes. I manually unapplied now all the formatting changes in the CMakeLists.txt - now there are only the dependencies left over which I added.

mfasDa avatar Dec 18 '23 13:12 mfasDa

Error while checking build/O2/fullCI for 47427fdfa481b26a06bff073c7242cfe0cb22713 at 2024-04-10 17:43:

## sw/BUILD/O2-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'


## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ GRERR=0
++ grep -v clang-diagnostic-error error-log.txt
++ grep ' error:'
/sw/SOURCES/O2/11845-slc8_x86-64/0/Framework/Core/include/Framework/ASoA.h:3115:81: error: invalid use of incomplete type 'typename table_t::RowViewFiltered<Filtered<Join<TracksExtra_001Extension, Table<TPCInnerParam, Flags, ITSClusterSizes, TPCNClsFindable, TPCNClsFindableMinusFound, TPCNClsFindableMinusCrossedRows, TPCNClsShared, TRDPattern, ITSChi2NCl, TPCChi2NCl, TRDChi2, TOFChi2, TPCSignal, TRDSignal, Length, TOFExpMom, PIDForTracking<Flags>, IsPVContributor<Flags>, HasITS<DetectorMap>, HasTPC<DetectorMap>, HasTRD<DetectorMap>, HasTOF<DetectorMap>, TPCNClsFound<TPCNClsFindable, TPCNClsFindableMinusFound>, TPCNClsCrossedRows<TPCNClsFindable, TPCNClsFindableMinusCrossedRows>, ITSClusterMap<ITSClusterSizes>, ITSNCls<ITSClusterSizes>, ITSNClsInnerBarrel<ITSClusterSizes>, ITSClsSizeInLayer<ITSClusterSizes>, TPCCrossedRowsOverFindableCls<TPCNClsFindable, TPCNClsFindableMinusCrossedRows>, TPCFoundOverFindableCls<TPCNClsFindable, TPCNClsFindableMinusFound>, TPCFractionSharedCls<TPCNClsShared, TPCNClsFindable, TPCNClsFindableMinusFound>, TrackEtaEMCAL, TrackPhiEMCAL, TrackTime, TrackTimeRes>>>, TracksExtra_001Extension, Table<TPCInnerParam, Flags, ITSClusterSizes, TPCNClsFindable, TPCNClsFindableMinusFound, TPCNClsFindableMinusCrossedRows, TPCNClsShared, TRDPattern, ITSChi2NCl, TPCChi2NCl, TRDChi2, TOFChi2, TPCSignal, TRDSignal, Length, TOFExpMom, PIDForTracking<Flags>, IsPVContributor<Flags>, HasITS<DetectorMap>, HasTPC<DetectorMap>, HasTRD<DetectorMap>, HasTOF<DetectorMap>, TPCNClsFound<TPCNClsFindable, TPCNClsFindableMinusFound>, TPCNClsCrossedRows<TPCNClsFindable, TPCNClsFindableMinusCrossedRows>, ITSClusterMap<ITSClusterSizes>, ITSNCls<ITSClusterSizes>, ITSNClsInnerBarrel<ITSClusterSizes>, ITSClsSizeInLayer<ITSClusterSizes>, TPCCrossedRowsOverFindableCls<TPCNClsFindable, TPCNClsFindableMinusCrossedRows>, TPCFoundOverFindableCls<TPCNClsFindable, TPCNClsFindableMinusFound>, TPCFractionSharedCls<TPCNClsShared, TPCNClsFindable, TPCNClsFindableMinusFound>, TrackEtaEMCAL, TrackPhiEMCAL, TrackTime, TrackTimeRes>>' (aka 'RowViewBase<o2::soa::FilteredIndexPolicy, o2::soa::Filtered<o2::soa::Join<o2::aod::TracksExtra_001Extension, o2::soa::Table<o2::aod::track::TPCInnerParam, o2::aod::track::Flags, o2::aod::track::ITSClusterSizes, o2::aod::track::TPCNClsFindable, o2::aod::track::TPCNClsFindableMinusFound, o2::aod::track::TPCNClsFindableMinusCrossedRows, o2::aod::track::TPCNClsShared, o2::aod::track::TRDPattern, o2::aod::track::ITSChi2NCl, o2::aod::track::TPCChi2NCl, o2::aod::track::TRDChi2, o2::aod::track::TOFChi2, o2::aod::track::TPCSignal, o2::aod::track::TRDSignal, o2::aod::track::Length, o2::aod::track::TOFExpMom, o2::aod::track::PIDForTracking<o2::aod::track::Flags>, o2::aod::track::IsPVContributor<o2::aod::track::Flags>, o2::aod::track::HasITS<o2::aod::track::v001::DetectorMap>, o2::aod::track::HasTPC<o2::aod::track::v001::DetectorMap>, o2::aod::track::HasTRD<o2::aod::track::v001::DetectorMap>, o2::aod::track::HasTOF<o2::aod::track::v001::DetectorMap>, o2::aod::track::TPCNClsFound<o2::aod::track::TPCNClsFindable, o2::aod::track::TPCNClsFindableMinusFound>, o2::aod::track::TPCNClsCrossedRows<o2::aod::track::TPCNClsFindable, o2::aod::track::TPCNClsFindableMinusCrossedRows>, o2::aod::track::v001::ITSClusterMap<o2::aod::track::ITSClusterSizes>, o2::aod::track::v001::ITSNCls<o2::aod::track::ITSClusterSizes>, o2::aod::track::v001::ITSNClsInnerBarrel<o2::aod::track::ITSClusterSizes>, o2::aod::track::v001::ITSClsSizeInLayer<o2::aod::track::ITSClusterSizes>, o2::aod::track::TPCCrossedRowsOverFindableCls<o2::aod::track::TPCNClsFindable, o2::aod::track::TPCNClsFindableMinusCrossedRows>, o2::aod::track::TPCFoundOverFindableCls<o2::aod::track::TPCNClsFindable, o2::aod::track::TPCNClsFindableMinusFound>, o2::aod::track::TPCFractionSharedCls<o2::aod::track::TPCNClsShared, o2::aod::track::TPCNClsFindable, o2::aod::track::TPCNClsFindableMinusFound>, o2::aod::track::TrackEtaEMCAL, o2::aod::track::TrackPhiEMCAL, o2::aod::track::TrackTime, o2::aod::track::TrackTimeRes>>>, o2::aod::TracksExtra_001Extension, o2::soa::Table<o2::aod::track::TPCInne/sw/SOURCES/O2/11845-slc8_x86-64/0/Framework/Core/include/Framework/ASoA.h:1401:114: note: in instantiation of template class 'o2::soa::Filtered<o2::soa::Join<aod::Tracks, aod::TracksExtra>>' requested here
++ [[ 0 == 0 ]]
++ exit 1
--

Full log here.

alibuild avatar Jan 12 '24 09:01 alibuild

@mfasDa : Sorry, I failed to follow it up. Now it is green, and the formatting seems ok. Shall we merge it?

davidrohr avatar Apr 17 '24 08:04 davidrohr

@davidrohr , yes I think we should merge it. The requested review will probably never come :(

mfasDa avatar Apr 17 '24 08:04 mfasDa