lingua-franca icon indicating copy to clipboard operation
lingua-franca copied to clipboard

CPP Memory Management Problem

Open Elocien opened this issue 4 years ago • 7 comments

Cigarette Smoker Benchmark

Console Output
`[INFO]  Starting the execution
Benchmark: CigaretteSmokerReactorLFCppBenchmark
Arguments:
            numIterations = 12
                numRounds = 1000
               numSmokers = 200
System information:
                 O/S Name = Mac OSX
Iteration 1 - 6.486 ms
Iteration 2 - 6.037 ms
Iteration 3 - 5.347 ms
Iteration 4 - 5.02 ms
Iteration 5 - 4.127 ms
Iteration 6 - 3.897 ms
Iteration 7 - 4.068 ms
Iteration 8 - 4.235 ms
Iteration 9 - 3.8 ms
Iteration 10 - 3.617 ms
Iteration 11 - 3.961 ms
Iteration 12 - 3.646 ms

Execution - Summary:
           Best Time:     0.000 ms
          Worst Time:     6.486 ms
              Median:     4.098 ms
[INFO]  Terminating the execution
CigaretteSmoker(3851,0x7000076be000) malloc: Incorrect checksum for freed object 0x7fbcf9f04110: probably modified after being freed.
Corrupt value: 0x37a230
CigaretteSmoker(3851,0x7000076be000) malloc: *** set a breakpoint in malloc_error_break to debug
[1]    3851 illegal hardware instruction`
Filter Bank:
[INFO]  Starting the execution
Benchmark: FilterBankBenchmark
Arguments:
            numIterations = 12
           numSimulations = 34816
               numColumns = 16384
              numChannels = 8
System information:
                 O/S Name = Mac OSX
Iteration 1 - 1544.33 ms
Iteration 2 - 1468.65 ms
Iteration 3 - 1350.61 ms
Iteration 4 - 1382.67 ms
Iteration 5 - 1305.86 ms
Iteration 6 - 1326.42 ms
Iteration 7 - 1259.69 ms
Iteration 8 - 1364.56 ms
Iteration 9 - 1271.04 ms
Iteration 10 - 1271.02 ms
Iteration 11 - 1259.41 ms
Iteration 12 - 1266.07 ms
FilterBank(3915,0x7000071a0000) malloc: Incorrect checksum for freed object 0x7fd8c61040f0: probably modified after being freed.
Corrupt value: 0x4b76ae08
FilterBank(3915,0x7000071a0000) malloc: *** set a breakpoint in malloc_error_break to debug
[1]    3915 abort

Elocien avatar Nov 12 '21 10:11 Elocien

Thanks! This is strange indeed. Since you are running this on a Mac Book: Do you have an x86 or an ARM-based processor (M1)? The C++ runtime isn't tested on ARM and there could be issues with incorrect usage of atomic variables (this is a lot trickier on ARM).

cmnrd avatar Nov 12 '21 10:11 cmnrd

I'm running this on an x86 processor

Elocien avatar Nov 12 '21 10:11 Elocien

Just did a fresh installation of lfc and did multiple reruns of the benchmarks CigraretteSmoker and FilterBank.

On the first run of FilterBank I got a Segmentation Error: [INFO] Starting the execution Benchmark: FilterBankBenchmark Arguments: numIterations = 12 numSimulations = 34816 numColumns = 16384 numChannels = 8 System information: O/S Name = Mac OSX Iteration 1 - 1561.76 ms Iteration 2 - 1376.66 ms Iteration 3 - 1278.81 ms Iteration 4 - 1267.52 ms Iteration 5 - 1274.12 ms Iteration 6 - 1255.24 ms Iteration 7 - 1261.75 ms Iteration 8 - 1252.3 ms Iteration 9 - 1260.98 ms Iteration 10 - 1271.17 ms Iteration 11 - 1264.25 ms Iteration 12 - 1265.94 ms

Execution - Summary: [1] 13283 segmentation fault benchmark/Cpp/Savina/src/bin/FilterBank


The second and third run resulted in the same errors as above.

CigaretteSmoker, on the other hand, ran without problem multiple times.

Elocien avatar Nov 17 '21 18:11 Elocien

There must be some concurrency bug in the handling of shutdown. So far I haven't seen segmentation faults, but some programs occasionally deadlock. I will have another close look at this and see if I can spot the bug.

cmnrd avatar Nov 18 '21 07:11 cmnrd

I didn't experience any problems running lingua-franca / reactor-cpp on arm. I tested my little project on armv6, armv6 and aarch64.

@cmnrd I think a good test would be to apply the strict memory orders (sequentially consistent ordering) and let him re-run the tests and see if the issue persists.

tanneberger avatar Feb 23 '22 20:02 tanneberger

@Elocien You really encountered this problem only with FilterBank and the CigraretteSmokers Tests right ?

tanneberger avatar Mar 10 '22 16:03 tanneberger

There must be some concurrency bug in the handling of shutdown. So far I haven't seen segmentation faults, but some programs occasionally deadlock.

I reproduced this twice with FilterBank and once with Throughput today (e.g., here). In my experience, the bug is common enough to make it challenging to get complete benchmark results.

petervdonovan avatar Jul 16 '22 00:07 petervdonovan