AliceO2 icon indicating copy to clipboard operation
AliceO2 copied to clipboard

Alpide simulation response for Vbb=0V and Vbb=-3V

Open AlexBigO opened this issue 1 year ago • 11 comments

Implementation of the Vbb=0V charge-collection tables (on top of the Vbb=-3V ones already there). Conversion of charged-collection tables from 810 .txt files to one .json file for each Vbb value.

AlexBigO avatar Aug 03 '22 12:08 AlexBigO

@ktf @sawenzel @shahor02 Hi Giulio, Sandro, Ruben. Could not you help us with the "whitespace" check, please ? There are two kinds of reported errors: "Indent code using spaces instead of tabs" and "Remove the trailing spaces at the end of the line". Yes, one may guess what it actually means. But is there a way to see the line number(s) where these errors happen ? Or, what kind of "grep" should be applied, in order to find the problematic lines ? Thank you very much, Yura.

iouribelikov avatar Aug 05 '22 09:08 iouribelikov

Hi. I understand the motivation behind this transformation (reading many files from CVMFS over the network or local file system is not ideal) and thanks for reacting to my initial comments !

However, I am personally not convinced that the new proposition is a step forward. Essentially I see that

a) we have 4 million of new lines of text to treat in git (maybe not critical but still maybe not necessary)

b) have text files of considerable size (22MB) that still need to be fetched from CVMFS ...and also parsed with a json parser during runtime. Note that the size of the new json file is also considerably larger than all the original 810 files taken together.

Wouldn't it be more useful to convert these textual data into actual compressed binary objects ? I think we can by all means have a hybrid approach here:

a) keep text files that you can modify in git

b) convert these text files to a small compressed binary file for instance containing ROOT objects (by calling a conversion step during build)

c) install binary files as part of installation, not text files (eventually one could also install them via CCDB)

d) use these ROOT objects directly in the code, without needing to parse large text files

I cannot judge how critical this PR is now ... but maybe this is something that could be considered.

sawenzel avatar Aug 16 '22 12:08 sawenzel

Hi, I agree with @sawenzel , I would consider converting it to CCDB object.

shahor02 avatar Aug 29 '22 13:08 shahor02

Error while checking build/O2/o2 for f6cf3d68d8dc5a0e01e10bd6b8cf7aa4d4617154 at 2022-10-01 22:09:

## sw/BUILD/O2-latest/log
100% tests passed, 0 tests failed out of 458
 86/106 Test #100: test_Framework_test_SimpleDataProcessingDevice01 ........***Failed    2.17 sec
[ERROR] pid 27272 (internal-dpl-injected-dummy-sink) crashed with 1
[ERROR] SEVERE: Device internal-dpl-injected-dummy-sink (27272) returned with 1
99% tests passed, 1 tests failed out of 102

Full log here.

alibuild avatar Aug 30 '22 13:08 alibuild

Error while checking build/O2/o2-cs8 for f6cf3d68d8dc5a0e01e10bd6b8cf7aa4d4617154 at 2022-10-01 15:54:

## sw/BUILD/O2-latest/log
100% tests passed, 0 tests failed out of 458
 76/106 Test  #90: test_Framework_test_CallbackService .....................***Failed    6.10 sec
[ERROR] pid 26579 (dest) crashed with 1
[ERROR] SEVERE: Device dest (26579) returned with 1
99% tests passed, 1 tests failed out of 102

Full log here.

alibuild avatar Sep 01 '22 20:09 alibuild

Error while checking build/O2/o2-dataflow-cs8 for f6cf3d68d8dc5a0e01e10bd6b8cf7aa4d4617154 at 2022-10-01 11:35:

## sw/BUILD/O2-latest/log
100% tests passed, 0 tests failed out of 435
 86/106 Test #100: test_Framework_test_SimpleDataProcessingDevice01 ........***Failed    2.88 sec
[ERROR] pid 25319 (internal-dpl-injected-dummy-sink) crashed with 1
[ERROR] SEVERE: Device internal-dpl-injected-dummy-sink (25319) returned with 1
99% tests passed, 1 tests failed out of 102

Full log here.

alibuild avatar Sep 04 '22 05:09 alibuild

Error while checking build/AliceO2/O2/o2/macOS-arm for f6cf3d68d8dc5a0e01e10bd6b8cf7aa4d4617154 at 2022-09-28 19:51:

No log files found

Full log here.

alibuild avatar Sep 04 '22 06:09 alibuild

Error while checking build/O2/fullCI for f6cf3d68d8dc5a0e01e10bd6b8cf7aa4d4617154 at 2022-10-01 07:58:

## sw/BUILD/o2checkcode-latest/log
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/Common/test/testGPUsortCUDA.cu:22:10: error: 'boost/test/unit_test.hpp' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUTracking/TRDTracking/GPUTRDTracker.cxx:37:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUTracking/Base/GPUReconstruction.cxx:37:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUTracking/Base/GPUReconstructionCPU.cxx:45:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUTracking/display/GPUDisplay.cxx:36:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUTracking/Base/cuda/GPUReconstructionCUDAGenRTC.cu:16:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/cuda/../Shared/Utils.h:26:10: error: 'boost/program_options.hpp' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/cuda/../Shared/Utils.h:26:10: error: 'boost/program_options.hpp' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/../Shared/Utils.h:146:34: error: use of undeclared identifier 'int4'; did you mean 'int'? [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/benchmark.hip.cxx:199:35: error: use of undeclared identifier 'int4'; did you mean 'int'? [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/../Shared/Utils.h:146:34: error: use of undeclared identifier 'int4'; did you mean 'int'? [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:64:8: error: unknown type name '__host__' [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:64:27: error: expected ';' after top level declarator [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:64:33: error: overloaded 'operator+=' must have at least one parameter of class or enumeration type [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:64:44: error: unknown type name 'int4'; did you mean 'int'? [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:64:53: error: unknown type name 'int4'; did you mean 'int'? [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:66:4: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:66:11: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:67:4: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:67:11: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:68:4: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:68:11: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:69:4: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:69:11: error: member reference base type 'int' is not a structure or union [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:85:1: error: unknown type name '__global__' [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:90:19: error: use of undeclared identifier 'blockIdx' [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:90:32: error: use of undeclared identifier 'blockDim' [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:90:45: error: use of undeclared identifier 'threadIdx' [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/GPU/GPUbenchmark/hip/Kernels.hip.cxx:90:78: error: use of undeclared identifier 'blockDim' [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/Framework/Logger/include/Framework/Logger.h:14:10: error: 'fairlogger/Logger.h' file not found [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:520:12: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:1059:69: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:1175:5: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:1852:16: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:2276:18: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:4336:16: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/sm_20_atomic_functions.h:89:39: error: redefinition of 'atomicAdd' [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/Framework/Logger/include/Framework/Logger.h:14:10: error: 'fairlogger/Logger.h' file not found [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:520:12: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:1059:69: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:1175:5: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:1852:16: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:2276:18: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/cuda/std/detail/libcxx/include/type_traits:4336:16: error: CUDA device code does not support variadic functions [clang-diagnostic-error]
/usr/local/cuda-11.7/include/sm_20_atomic_functions.h:89:39: error: redefinition of 'atomicAdd' [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/Detectors/EMCAL/calibration/include/EMCALCalibration/EMCALCalibExtractor.h:36:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/Detectors/EMCAL/calibration/include/EMCALCalibration/EMCALCalibExtractor.h:36:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/Detectors/EMCAL/calibration/include/EMCALCalibration/EMCALCalibExtractor.h:36:10: error: 'omp.h' file not found [clang-diagnostic-error]
/sw/SOURCES/O2/9590-slc8_x86-64/0/Detectors/TOF/calibration/src/TOFChannelCalibrator.cxx:23:10: error: 'omp.h' file not found [clang-diagnostic-error]

Full log here.

alibuild avatar Sep 06 '22 09:09 alibuild

@sawenzel I see there is a number of PRs which fail all with:

## sw/BUILD/O2-latest/log
448/448 Test #404: o2sim_hepmc ..........................................................................***Timeout 400.02 sec

I assume this is some timeout CCDB related? Or are we at the limit for that test?

ktf avatar Sep 07 '22 06:09 ktf

Error while checking build/O2/o2-dataflow for f6cf3d68d8dc5a0e01e10bd6b8cf7aa4d4617154 at 2022-10-01 06:32:

## sw/BUILD/O2-latest/log
100% tests passed, 0 tests failed out of 435
100% tests passed, 0 tests failed out of 102


## sw/BUILD/QualityControl-latest/log
27/38 Test #33: multinode_test ..........................***Failed   37.48 sec
[3856:qc-task-TST-MultiNodeLocalTest49749]: 2022-10-01 04:31:21.982958 !!! [1;31mError - Could not find the DPL InfoLogger[0m
[3845:qc-check-TST-MultiNodeLocalTest]: 2022-10-01 04:31:22.011351 !!! [1;31mError - Could not find the DPL InfoLogger.[0m
[3844:qc-task-TST-MultiNodeRemoteTest38816]: 2022-10-01 04:31:22.011862 !!! [1;31mError - Could not find the DPL InfoLogger[0m
[3846:qc-check-TST-MultiNodeRemoteTest]: 2022-10-01 04:31:22.012280 !!! [1;31mError - Could not find the DPL InfoLogger.[0m
[ERROR] websocket_server_callback: Error while reading from websocketconnection reset by peer
[ERROR] SEVERE: Device internal-dpl-clock (3840) returned with 128
97% tests passed, 1 tests failed out of 38

Full log here.

alibuild avatar Sep 08 '22 17:09 alibuild

Error while checking build/AliceO2/O2/o2/macOS for f6cf3d68d8dc5a0e01e10bd6b8cf7aa4d4617154 at 2022-09-28 21:16:

No log files found

Full log here.

alibuild avatar Sep 28 '22 19:09 alibuild

This PR did not have any update in the last 30 days. Is it still needed? Unless further action in will be closed in 5 days.

github-actions[bot] avatar Nov 01 '22 01:11 github-actions[bot]