dev: adding counters to bookkeeping

[lietava]

This development is related to https://github.com/AliceO2Group/Bookkeeping/blob/mboulais/O2B-1283/create-cxx-client-for-trigger-counters/cxx-client/README.md The first problem I can not solve is how to link bookkeeping-api::BookkeepingApi to my O2 code

[github-actions[bot]]

REQUEST FOR PRODUCTION RELEASES: To request your PR to be included in production software, please add the corresponding labels called "async-" to your PR. Add the labels directly (if you have the permissions) or add a comment of the form (note that labels are separated by a ",")

+async-label <label1>, <label2>, !<label3> ...

This will add <label1> and <label2> and removes <label3>.

The following labels are available async-2023-pbpb-apass3 async-2023-pbpb-apass4 async-2022-pp-apass6-2023-PbPb-apass2 async-2022-pp-apass4 async-2022-pp-apass4-accepted async-2022-pp-apass6-2023-PbPb-apass2-accepted async-2023-pbpb-apass3-accepted async-2023-pbpb-apass4-accepted async-2023-pp-apass4 async-2023-pp-apass4-accepted async-2024-pp-apass1 async-2024-pp-apass1-accepted async-2022-pp-apass7 async-2022-pp-apass7-accepted async-2024-pp-cpass0 async-2024-pp-cpass0-accepted

[lietava]

@ktf , please, can you have a look. I do not know how to link bookeeping library to my O2 code. I am already discussing with @shahor02 and @martinboulais, but we are stuck.

[Barthelemy]

Hi,

You can see in QC how to link against the BK api.

1. In your CMakeLists.txt you should find the package : find_package(BookkeepingApi REQUIRED) # it must be before O2 for some reasons.
2. In your CMakeLists.txt you should point to the library name when building the binary : AliceO2::BookkeepingApi and not bookkeeping-api::BookkeepingApi
3. include in your code

#include "BookkeepingApi/BkpClientFactory.h"
#include "BookkeepingApi/BkpClient.h"

4. mClient = BkpClientFactory::create(url);
5. use mClient

[lietava]

@Barthelemy : thanks. I can compile now.

[lietava]

@Barthelemy , @ktf : how can I "you need to add a dependency on bookkeeping-api in alidist" ? I added bookkeeping-api to alidist/o2.sh and it seems it is accepted. Next question is how can I compile my own (martin's) version with API I need to test ?

[ktf]

@lietava just clone bookkeeping-api at the same level as O2 and alidist, it will be picked up automatically from your local checkout, rather than the one specified in the recipe. It's a feature which we call "development packages" and which is described at https://alisw.github.io/alibuild/user.html#developing-packages-locally

[lietava]

@ktf : thank you for previous response , it worked.

[alibuild]

Error while checking build/O2/fullCI for 12f1125e2bce88c05357ccce02f03007ffe2c814 at 2024-07-01 18:12:

## sw/BUILD/O2-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
CMake Error at DataFormats/Detectors/CTP/CMakeLists.txt:11 (find_package):

Full log here.

[lietava]

@ktf @shahor02 : I can compile locally but at check it says: CMake Error at DataFormats/Detectors/CTP/CMakeLists.txt:11 (find_package): By not providing "FindBookkeepingApi.cmake" in CMAKE_MODULE_PATH this project has asked CMake to find a package configuration file provided by "BookkeepingApi", but CMake did not find one. PLease, let me know how to solve.

[lietava]

adding also @Barthelemy

[lietava]

@martinboulais suggest to update https://github.com/alisw/alidist/blob/master/o2.sh ?

[lietava]

HI @ktf , please, let me know how to proceed here. As I understand the minor fix in alidist/o2.sh is needed or let me know if something to be done on my side.

[ktf]

https://github.com/alisw/alidist/pull/5536 should do the job.

[ktf]

As a side note, I am not sure having a DataFormat depending on the Bookkeeping API is a good idea. Can't you factorise it into a separate component?

[lietava]

@ktf thanks ! I can move dependency to Detectors/CTP - would it be better ?

[ktf]

Yeah, I would personally do that...

[lietava]

@ktf I moved dependency to Detectors/CTP. Now I think there will be conflict with QC which I shall solve. I still make it to compile to see here.

[alibuild]

Error while checking build/O2/fullCI for 49bdf39aae5938d6e1c87dc6e982d3f29774eedb at 2024-07-11 19:17:

## sw/BUILD/O2-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'


## sw/BUILD/O2-full-system-test-latest/log
Detected critical problem in logfile reco_NOGPU.log
reco_NOGPU.log:[34274:internal-dpl-ccdb-backend]: [17:17:03][ERROR] Exception while running: Fatal error. Rethrowing.
reco_NOGPU.log-[34274:internal-dpl-ccdb-backend]: [17:17:03][FATAL] Unhandled o2::framework::runtime_error reached the top of main of o2-itsmft-stf-decoder-workflow, device shutting down. Reason: Fatal error
[34274:internal-dpl-ccdb-backend]: [17:17:03][ERROR] CcdbDownloader finished transfer http://alice-ccdb.cern.ch/PHS/Config/RecoParams for 1550600800000 (agent_id: alimetal07.cern.ch-1720718202-2K28Ud) with http code: 0
[34274:internal-dpl-ccdb-backend]: [17:17:03][ERROR] File PHS/Config/RecoParams could not be retrieved. No more hosts to try.
[34274:internal-dpl-ccdb-backend]: [17:17:03][FATAL] Unable to find object PHS/Config/RecoParams/1550600800000
[34274:internal-dpl-ccdb-backend]: [17:17:03][ERROR] Exception while running: Fatal error. Rethrowing.
[34274:internal-dpl-ccdb-backend]: [17:17:03][FATAL] Unhandled o2::framework::runtime_error reached the top of main of o2-itsmft-stf-decoder-workflow, device shutting down. Reason: Fatal error
[ERROR] Workflow crashed - PID 34274 (internal-dpl-ccdb-backend) did not exit correctly however it's not clear why. Exit code forced to 128.


## sw/BUILD/o2checkcode-latest/log
--
========== List of errors found ==========
++ GRERR=0
++ grep -v clang-diagnostic-error error-log.txt
++ grep ' error:'
++ GRERR=1
++ [[ 1 == 0 ]]
++ mkdir -p /sw/INSTALLROOT/c9d4a6ff9f4498e2d78beae4035e4087f0871692/slc8_x86-64/o2checkcode/1.0-local3632/etc/modulefiles
++ cat
--

Full log here.

[lietava]

@ktf : I created QC PR to remove RunManager dependence there: https://github.com/AliceO2Group/QualityControl/pull/2367 If I understand correctly the following sequence of merges is to be done.

1. QC 2367,
2. alidist 5536,
3. O2 13224. Do I understand correctly ?

[ktf]

Your sequence will work. Notice there is no dependency between 1. and 2, so you can do either.

[lietava]

@ktf Thanks. Well, then please approve alidist, @Barthelemy or @knopers8 , please, approve QC. Anyway I wait whoever is to approve PRs.

[lietava]

Hi @Barthelemy , @shahor02 and @ktf compilation failed . O2/o2 O2-o2-cs8 failed on macros which I can fix. But mac failed in cmake
CMake Error at /opt/homebrew/opt/llvm@15/lib/cmake/llvm/FindgRPC.cmake:85 (find_library): Could not find GRPC_LIBRARY using the following names: grpc++ ? Any advise on that, please ?

[Barthelemy]

@lietava I think that I know what this issue is. Checking

[Barthelemy]

@lietava unfortunately I was wrong. I don't know what the issue is.

[lietava]

@ktf @shahor02 : now it is clean only mac error.

[alibuild]

Error while checking build/O2/fullCI for 91c6f5a9613853e8916fe97816c6d46a66dd9eff at 2024-07-22 06:41:

## sw/BUILD/O2-latest/log
c++: error: unrecognized command-line option '--rtlib=compiler-rt'
c++: error: unrecognized command-line option '--rtlib=compiler-rt'


## sw/BUILD/O2-sim-challenge-test-latest/log
./sim-challenge.log[ERROR] Workflow crashed - PID 17757 (tof-matcher) did not exit correctly however it's not clear why. Exit code forced to 128.
./sim-challenge.log[ERROR]  - Device tof-matcher: pid 17757 (exit 128)
./sim-challenge.log[ERROR] SEVERE: Device tof-matcher (17757) returned with 128
./midreco.log[13459:MIDRecoWriter]: [04:40:23][ERROR] Some Lifetime::Timeframe data got dropped starting at 1
./digi.log[ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/37}
./digi.log[ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/38}
./digi.log[ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/40}
./digi.log[ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/42}
./digi.log[ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/43}
./digi.log[ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/45}
./digi.log[ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/46}
./digi.log[ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/47}
./digi.log[ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/48}
./digi.log[ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/49}
./digi.log[7132:internal-dpl-ccdb-backend]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/37}
./digi.log[7132:internal-dpl-ccdb-backend]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/38}
./digi.log[7132:internal-dpl-ccdb-backend]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/40}
./digi.log[7132:internal-dpl-ccdb-backend]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/42}
./digi.log[7132:internal-dpl-ccdb-backend]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/43}
./digi.log[7132:internal-dpl-ccdb-backend]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/45}
./digi.log[7132:internal-dpl-ccdb-backend]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/46}
./digi.log[7132:internal-dpl-ccdb-backend]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/47}
./digi.log[7132:internal-dpl-ccdb-backend]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/48}
./digi.log[7132:internal-dpl-ccdb-backend]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/49}
./digi.log[7127:SimReader]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/37}
./digi.log[7127:SimReader]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/38}
./digi.log[7127:SimReader]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/40}
./digi.log[7127:SimReader]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/42}
./digi.log[7127:SimReader]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/43}
./digi.log[7127:SimReader]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/45}
./digi.log[7127:SimReader]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/46}
./digi.log[7127:SimReader]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/47}
./digi.log[7127:SimReader]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/48}
./digi.log[7127:SimReader]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/49}
./digi.log[7126:internal-dpl-clock]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/37}
./digi.log[7126:internal-dpl-clock]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/38}
./digi.log[7126:internal-dpl-clock]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/40}
./digi.log[7126:internal-dpl-clock]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/42}
./digi.log[7126:internal-dpl-clock]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/43}
./digi.log[7126:internal-dpl-clock]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/45}
./digi.log[7126:internal-dpl-clock]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/46}
./digi.log[7126:internal-dpl-clock]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/47}
./digi.log[7126:internal-dpl-clock]: [ERROR] Found duplicate input binding with different spec.:collisioncontext {SIM/COLLISIONCONTEXT/48}
[0 more errors; see full log]

Full log here.

[lietava]

@ktf can you, please, comment on this - it compiles on linuxes but failing for mac.

[ktf]

it looks like it's picking up grpc from the system rather than our own.

[lietava]

@ktf , thanks. So you or somebody will fix it ? (It is not superurgent but ~ week time would be nice).

[ktf]

I can have a look in a bit.