AliceO2
AliceO2 copied to clipboard
AliceO2Group
AOD: Disable MC tables for data
Now MC tables (even empty ones) are only written to disk when MC is active. Marginally effects file size (reduction by a few KBs, e.g. tree headers).
Tested on local MC production generated with DOANALYSIS=1 ./O2DPG_pp_minbias.sh and with files from apass1_debug_build97 to reproduce data AO2D and run an analysis on this.
Hi, @jgrosseo, these changes would stop writing empty MC trees for data. I want to avoid breaking stuff downstream, do you have suggestion on what else I could test or anything that would immediately speak against doing this?
For me it looks good from just checking the code. @ktf @aalkin Any comments?
Other than "breaking" the current generalization, the code seems to do what is expected. I thinks this was tested long time ago and the effect on the AOD size was really marginal. Would it make more sense to actually provide the configuration to the AOD writer to store only certain subset of the tables created by AOD producer? It is more flexible.
What are the benefits of doing this apart from a few bytes? I fear this might just add extra complication in the code without having a major impact on users. The data format specification of AOD tells that these tables are present. Empty data is probably easier to handle than missing tables.
What are the benefits of doing this apart from a few bytes? I fear this might just add extra complication in the code without having a major impact on users. The data format specification of AOD tells that these tables are present. Empty data is probably easier to handle than missing tables.
Where does the specification tell you that for data you have MC tables? :)
What are the benefits of doing this apart from a few bytes? I fear this might just add extra complication in the code without having a major impact on users. The data format specification of AOD tells that these tables are present. Empty data is probably easier to handle than missing tables.
Where does the specification tell you that for data you have MC tables? :)
Well, I thought you had a well-defined list of AOD tables written in the specification according to https://aliceo2group.github.io/analysis-framework/docs/datamodel/ao2dTables.html. If some of them are optional it is just fine with me. Maybe the doc would need to be more specific about this.
What are the benefits of doing this apart from a few bytes? I fear this might just add extra complication in the code without having a major impact on users. The data format specification of AOD tells that these tables are present. Empty data is probably easier to handle than missing tables.
Where does the specification tell you that for data you have MC tables? :)
Well, I thought you had a well-defined list of AOD tables written in the specification. If some of them are optional it is just fine with me.
Yes, quite some are optional: E.g. Run 2 vs. Run 3. And for now MC tables have always been there but it was more a limitation that no one made the effort to make add the if statement in the aod producer.
Several times people got confused why our data has MC tables.
On the size-level it does not save much of course. An advantage of not having the tables is that the merger does not merge them (even if empty, it costs a few cycles in reco).
Error while checking build/O2/fullCI for 11c0f56d024565beb628ab6c69f1e20a6d5a6731 at 2024-05-21 00:36:
## sw/BUILD/O2-sim-challenge-test-latest/log
./sim-challenge.log[ERROR] Workflow crashed - PID 26301 (tof-matcher) did not exit correctly however it's not clear why. Exit code forced to 128.
./sim-challenge.log[ERROR] - Device tof-matcher: pid 26301 (exit 128)
./sim-challenge.log[ERROR] SEVERE: Device tof-matcher (26301) returned with 128
./midreco.log[17873:MIDRecoWriter]: [22:33:56][ERROR] Some Lifetime::Timeframe data got dropped starting at 1
./digi.log[ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/37}
./digi.log[ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/38}
./digi.log[ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/40}
./digi.log[ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/42}
./digi.log[ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/43}
./digi.log[ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/45}
./digi.log[ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/46}
./digi.log[ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/47}
./digi.log[ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/48}
./digi.log[ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/49}
./digi.log[7621:internal-dpl-clock]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/37}
./digi.log[7621:internal-dpl-clock]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/38}
./digi.log[7621:internal-dpl-clock]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/40}
./digi.log[7621:internal-dpl-clock]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/42}
./digi.log[7621:internal-dpl-clock]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/43}
./digi.log[7621:internal-dpl-clock]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/45}
./digi.log[7621:internal-dpl-clock]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/46}
./digi.log[7621:internal-dpl-clock]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/47}
./digi.log[7621:internal-dpl-clock]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/48}
./digi.log[7621:internal-dpl-clock]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/49}
./digi.log[7626:SimReader]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/37}
./digi.log[7626:SimReader]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/38}
./digi.log[7626:SimReader]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/40}
./digi.log[7626:SimReader]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/42}
./digi.log[7626:SimReader]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/43}
./digi.log[7626:SimReader]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/45}
./digi.log[7626:SimReader]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/46}
./digi.log[7626:SimReader]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/47}
./digi.log[7626:SimReader]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/48}
./digi.log[7626:SimReader]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/49}
./digi.log[7631:internal-dpl-ccdb-backend]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/37}
./digi.log[7631:internal-dpl-ccdb-backend]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/38}
./digi.log[7631:internal-dpl-ccdb-backend]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/40}
./digi.log[7631:internal-dpl-ccdb-backend]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/42}
./digi.log[7631:internal-dpl-ccdb-backend]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/43}
./digi.log[7631:internal-dpl-ccdb-backend]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/45}
./digi.log[7631:internal-dpl-ccdb-backend]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/46}
./digi.log[7631:internal-dpl-ccdb-backend]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/47}
./digi.log[7631:internal-dpl-ccdb-backend]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/48}
./digi.log[7631:internal-dpl-ccdb-backend]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/49}
./digi.log[7636:TRDDigitizer]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/37}
./digi.log[7636:TRDDigitizer]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/38}
./digi.log[7636:TRDDigitizer]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/40}
./digi.log[7636:TRDDigitizer]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/42}
[0 more errors; see full log]
Full log here.
