hyrise
hyrise copied to clipboard
Published
20 hours ago •
hyrise
hyrise
Update CI and clang-tidy adaptions
Changing a few things to get the CI and our Docker image up to date:
- use Ubuntu 22.04 (for the Docker CI image)
- with that we now use the following compilers in the CI
- clang11 and clang 14 (clang11 only a single debug build)
- GCC 9 and GCC 11 (GCC 9 only a single debug build)
- with that we now use the following compilers in the CI
- many small adaptions to make clang-tidy14 happy (+ exclude several new tidy rules)
- we use clang11 for the coverage calculation as there is an unresolved problem with clang14 (https://github.com/llvm/llvm-project/issues/54907)
Todos:
- [x] Reevaluated necessity for
.load()in some std::atomic cases (see #2439 andchunk.cpp/abstract_task.cpp) - [x] Pull out format changes and create upfront PR (#2470)
- [x] Determine where failing ThreadSanitizer test comes from (~the sanitizer does not complain, but we run into timeouts~ probably a sanitizer issues with the timeouts, significantly larger tests in release build work)
- [x] ~Test GCC 12 (should be available in Jammy)~ (see #2473)
- [x] Fix Valgrind issues (we need to use GCC for Valgrind runs, as there is an open bug with Valgrind and Clang 14)
Review parts:
- [x] @dey4ss
./src/lib/import_export 5 files changed, 40 insertions(+), 35 deletions(-)
./src/lib/utils 13 files changed, 99 insertions(+), 84 deletions(-)
./src/lib/operators 38 files changed, 204 insertions(+), 176 deletions(-)
- [x] @dey4ss
./src/lib/sql 4 files changed, 80 insertions(+), 61 deletions(-)
./src/lib/memory 1 file changed, 14 insertions(+), 6 deletions(-)
./src/lib/storage 30 files changed, 292 insertions(+), 260 deletions(-)
- [ ]
./src/lib/expression 7 files changed, 142 insertions(+), 127 deletions(-)
./src/lib/scheduler 6 files changed, 15 insertions(+), 15 deletions(-)
./src/lib/logical_query_plan 7 files changed, 61 insertions(+), 38 deletions(-)
./src/lib/statistics 7 files changed, 67 insertions(+), 58 deletions(-)
- [ ]
./src/lib/server 7 files changed, 39 insertions(+), 35 deletions(-)
./src/lib/visualization 4 files changed, 46 insertions(+), 35 deletions(-)
./src/lib/cost_estimation 1 file changed, 9 insertions(+), 9 deletions(-)
./src/lib/optimizer 8 files changed, 115 insertions(+), 104 deletions(-)
- [ ]
./src/benchmarklib 18 files changed, 325 insertions(+), 275 deletions(-)
- [ ]
./scripts 2 files changed, 6 insertions(+), 4 deletions(-)
./resources 1 file changed, 11 insertions(+), 2 deletions(-)
./src/bin 2 files changed, 17 insertions(+), 17 deletions(-)
./src/benchmark 2 files changed, 14 insertions(+), 12 deletions(-)
./src/plugins 6 files changed, 25 insertions(+), 21 deletions(-)
./src/test 31 files changed, 93 insertions(+), 78 deletions(-)
Can locally build with gcc9 but the CI fails. There is a very similar bug mentioned in https://github.com/google/googletest/issues/3384.
Closing for now, might reopen in a few weeks, when Gtest has resolved this issue.
Apparently, we cannot update google test until we have updated our CI GCC version. I don't see a reason for that just now, but we should consider enforcing a newer GCC version once other issues arise with GCC 9.
Pinging @Bensk1 and @mweisgut: shall I remove the update of google test and reopen this PR, so we can at least build "most" targets with clang 13?
I haven't investigated the errors raised with llvm 13 in detail, but I thought that even without the google test update of this PR, i.e., also with the google test version we currently use in Hyrise, the compilation of google test fails.
I tried to build the PR with llvm: stable 13.0.0 (bottled) on macOS Big Sur 11.4 and clang-13 also reports reserved-identifier warnings for __dtrace:
hyrise/build-release-clang-13/provider.hpp:155:13: error: identifier '__dtrace_probe$hyrise$create_pipeline$v1$75696e747074725f74' is reserved because it starts with '__' [-Werror,-Wreserved-identifier]
extern void __dtrace_probe$hyrise$create_pipeline$v1$75696e747074725f74(uintptr_t);
[...]
As GCC stated here, the issue will not be fixed. I am currently trying if GCC 11.2 works fine. That would mean we need to use Ubuntu 21.10, which is fine for me and actually also what I would prefer. Then, having a recent GCC means that some old clang versions (here 9) are no longer working as they have issues with a too recent stdlib.
I am currently testing it out, but I might vote for "let's update it all as much as we can". That means latest Ubuntu 21.10 with GCC 11.2 and whatever LLVM comes with it (I think 12). LLVM 14 works fine by the way. So this setup hopefully works then for quite some time.
One alternative is to keep GoogleTest at a pretty outdated version (which does not bother us), but I would still favor the "let's update it all" path.
So apparently we can avoid the problem with unity builds are not used with GCC. I added a comment.
That means we can continue to use Ubuntu 20.04. While I usually would prefer a newer version, I just realized that 21.10 will have its end of support already in four months. Since we don't plan to update the Docker image every three months, using the latest LTS might be better. Plus having a base image that's available pretty much everywhere (e.g., GitHub Actions etc.).
Hence, we can now use GCC 10, Clang 10, and Clang 12.
So just as an update and written notes in case I get distracted and forget this PR for a couple of weeks.
Still working on some other smaller issues.
Waiting for https://github.com/google/googletest/pull/3762 ... it's an incredible pain.
We're getting close.
Right now it looks like Ubuntu 22.04 and updated compilers work so far. There is one issue left with clang-tidy for which we might need to open an issue. This might block this PR for quite a while. We'll see.
We are currently using clang 14, which is the first release candidate. Release and debug builds looks okay, so using clang 14 might be okay since the semester is starting in two months and we should almost see the final release by then. By the way, using LLVM 13 does not change the tidy fail. Thus downgrading won't help here. But we might be in time to get this thing fixed for the release of LLVM 14.
Just to give everybody a short update:
- still waiting for an reaction on the LLVM issue
- trying to find out, where the sanitizer fails come from (fails often, not always)
- I can reproduce them with the current master and the master on a 20.04 image works fine (so it's not this branch here)
- it fails with clang 11 and 14 (so it's not clang either)
- will look into TBB and Boost
- ... no really good ideas so far
Short update: one of the things blocking this PR is the failing thread sanitizer test StressTest.TestTransactionConflicts . To my best knowledge, nothing has been touched that should effect this test, but it still fails.
To check if the thread sanitizer and its overhead might be the cause, I ran a release build of this test and used a thread count of 10.000 (that's 100x the threads of the test and clearly overusing the server) and used a timeout of 18 s (that a 10x shorter timeout).
I ran the test the test for hours and it hasn't failed. As of now, I think we have an issue with the new thread sanitizer. I'd vote for excluding this test in the thread sanitizer run. It's still executed in other test runs.
FYI: struggling with the dwarf version of clang 14. GCC 11 and clang 14 generate version 5 dwarf formats, which is not supported by Valgrind.
I am struggling to get jemalloc compiled with an enforced dwarf4 version.
If anybody ever had to deal with such issues: please help me. :)
FYI: struggling with the dwarf version of clang 14. GCC 11 and clang 14 generate version 5 dwarf formats, which is not supported by Valgrind.
I am struggling to get jemalloc compiled with an enforced dwarf4 version.
If anybody ever had to deal with such issues: please help me. :)
It's a little bit trickier than I hoped. Analyzing libjemalloc.so with readelf showed that using -gdwarf-4 what not sufficient as there have still been translation units using version 5.
But in the end it does not matter, as there is still an open issue with valgrind and clang 14: https://bugs.kde.org/show_bug.cgi?id=452758
For that reason, Jenkins now uses the GCC release to check for memory issues. Looking at it now.
@mweisgut, @tjjordan, @dey4ss : I think I have resolved (or answered, please have a look) all review issues brought up.
Ran again into the LZ4 problems with valgrind. I do not consider this to be an issue we should solve within the LZ4 segment, as many people consider valgrind to be outdated. And here, I am pretty sure that it is a false positive.
Since this issue came up multiple times lately, I added a small patch here, which flattens the use_caching variable and stacks if conditionals. I should have the same logic. And valgrind does not complain.
https://gist.github.com/Bouncner/bf348c4c23f824d42494d0a3a77a3883
I have seen some performance degradations on the AMD server, so I ran the entire benchmark twice. Unfortunately, unstable AMD results are not new to us. But anyhow, there does not appear to be a problem here.
System
node-19 - click to expand
| property | value |
|---|---|
| Hostname | node-19 |
| CPU | AMD EPYC 7742 64-Core Processor |
| Memory | 499GB |
| numactl | nodebind: 0 1 |
| numactl | membind: 0 1 |
Commit Info and Build Time
| commit | date | message | build time |
|---|---|---|---|
| 3228b77c0 | 18.08.2022 12:56 | Link profiling libraries for coverage builds (#2499) | real 325.51 user 2794.13 sys 139.46 |
| 833281d1e | 26.08.2022 19:51 | Move comment | real 320.09 user 2783.98 sys 133.81 |
hyriseBenchmarkTPCH - single-threaded, SF 10.0
Sum of avg. item runtimes: -2% || Geometric mean of throughput changes: +0%
Configuration Overview - click to expand
+Configuration Overview----+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_3228b77c02306e4bd15c207f87717af8acd30522_st.json | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_833281d1ee76a14a730e02fff4b4a0784c44baf7_st.json |
+--------------------------+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 3228b77c02306e4bd15c207f87717af8acd30522-dirty | 833281d1ee76a14a730e02fff4b4a0784c44baf7-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| clustering | None | None |
| compiler | clang 14.0.0 | clang 14.0.0 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2022-08-27 09:50:05 | 2022-08-27 13:12:09 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 100 | 100 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+--------------------------+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
+----------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+----------+--------++----------+----------+--------+---------+
+| TPC-H 01 || 5804.044 | 5512.343 | -5% || 0.17 | 0.18 | +5% | 0.0933 |
| TPC-H 02 || 43.440 | 43.011 | -1%˄ || 23.02 | 23.25 | +1%˄ | 0.3733 |
| TPC-H 03 || 2262.572 | 2276.604 | +1% || 0.44 | 0.44 | -1% | 0.4811 |
| TPC-H 04 || 1357.241 | 1370.676 | +1% || 0.74 | 0.73 | -1% | 0.2319 |
| TPC-H 05 || 2951.951 | 2975.914 | +1% || 0.34 | 0.34 | -1% | 0.3664 |
| TPC-H 06 || 150.522 | 151.686 | +1%˄ || 6.64 | 6.59 | -1%˄ | 0.2530 |
| TPC-H 07 || 885.077 | 886.185 | +0% || 1.13 | 1.13 | -0% | 0.6856 |
+| TPC-H 08 || 708.674 | 677.857 | -4% || 1.41 | 1.48 | +5% | 0.0000 |
+| TPC-H 09 || 6307.780 | 5851.991 | -7% || 0.16 | 0.17 | +8% | 0.1289 |
+| TPC-H 10 || 3164.181 | 2995.792 | -5% || 0.32 | 0.33 | +6% | 0.0005 |
| TPC-H 11 || 72.592 | 72.016 | -1%˄ || 13.78 | 13.89 | +1%˄ | 0.2240 |
-| TPC-H 12 || 954.963 | 1001.911 | +5% || 1.05 | 1.00 | -5% | 0.0000 |
| TPC-H 13 || 4528.206 | 4542.415 | +0% || 0.22 | 0.22 | -0% | 0.5889 |
| TPC-H 14 || 423.501 | 422.701 | -0%˄ || 2.36 | 2.37 | +0%˄ | 0.5707 |
| TPC-H 15 || 189.335 | 183.177 | -3%˄ || 5.28 | 5.46 | +3%˄ | 0.0000 |
| TPC-H 16 || 645.375 | 659.899 | +2% || 1.55 | 1.52 | -2% | 0.0000 |
| TPC-H 17 || 205.919 | 204.282 | -1%˄ || 4.86 | 4.90 | +1%˄ | 0.0003 |
| TPC-H 18 || 1427.266 | 1462.262 | +2% || 0.70 | 0.68 | -2% | 0.0035 |
| TPC-H 19 || 256.646 | 255.061 | -1%˄ || 3.90 | 3.92 | +1%˄ | 0.0056 |
| TPC-H 20 || 384.509 | 378.353 | -2%˄ || 2.60 | 2.64 | +2%˄ | 0.0094 |
| TPC-H 21 || 4483.365 | 4515.491 | +1% || 0.22 | 0.22 | -1% | 0.3272 |
-| TPC-H 22 || 409.265 | 446.985 | +9%˄ || 2.44 | 2.24 | -8%˄ | 0.0000 |
+----------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 37616.4 | 36886.6 | -2% || | | | |
| Geomean || | | || | | +0% | |
+----------++----------+----------+--------++----------+----------+--------+---------+
| Notes || ˄ Execution stopped due to max runs reached |
+----------++----------+----------+--------++----------+----------+--------+---------+
hyriseBenchmarkTPCH - single-threaded, SF 0.01
Sum of avg. item runtimes: +3% || Geometric mean of throughput changes: -2%
Configuration Overview - click to expand
+Configuration Overview----+------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_3228b77c02306e4bd15c207f87717af8acd30522_st_s01.json | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_833281d1ee76a14a730e02fff4b4a0784c44baf7_st_s01.json |
+--------------------------+------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 3228b77c02306e4bd15c207f87717af8acd30522-dirty | 833281d1ee76a14a730e02fff4b4a0784c44baf7-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| clustering | None | None |
| compiler | clang 14.0.0 | clang 14.0.0 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2022-08-27 10:09:46 | 2022-08-27 13:31:49 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 100 | 100 |
| scale_factor | 0.009999999776482582 | 0.009999999776482582 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+--------------------------+------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------+
+----------++----------+---------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+---------+--------++----------+----------+--------+---------+
| TPC-H 01 || 4.778 | 4.665 | -2%˄ || 209.20 | 214.28 | +2%˄ | 0.0000 |
| TPC-H 02 || 3.634 | 3.622 | -0%˄ || 275.05 | 275.96 | +0%˄ | 0.9574 |
| TPC-H 03 || 0.618 | 0.625 | +1%˄ || 1613.08 | 1596.14 | -1%˄ | 0.3219 |
| TPC-H 04 || 0.412 | 0.416 | +1%˄ || 2419.27 | 2398.31 | -1%˄ | 0.0189 |
| TPC-H 05 || 0.983 | 0.964 | -2%˄ || 1016.15 | 1036.13 | +2%˄ | 0.1183 |
| TPC-H 06 || 0.166 | 0.169 | +2%˄ || 5993.82 | 5862.38 | -2%˄ | 0.0000 |
-| TPC-H 07 || 9.237 | 10.760 | +16%˄ || 108.24 | 92.93 | -14%˄ | 0.0414 |
| TPC-H 08 || 15.205 | 15.052 | -1%˄ || 65.76 | 66.43 | +1%˄ | 0.6474 |
| TPC-H 09 || 3.381 | 3.479 | +3%˄ || 295.58 | 287.28 | -3%˄ | 0.8501 |
| TPC-H 10 || 0.731 | 0.739 | +1%˄ || 1364.38 | 1350.54 | -1%˄ | 0.0037 |
| TPC-H 11 || 0.193 | 0.196 | +2%˄ || 5143.79 | 5058.14 | -2%˄ | 0.1961 |
| TPC-H 12 || 0.592 | 0.583 | -2%˄ || 1684.37 | 1711.64 | +2%˄ | 0.0000 |
| TPC-H 13 || 1.935 | 1.955 | +1%˄ || 516.28 | 511.06 | -1%˄ | 0.0027 |
| TPC-H 14 || 0.318 | 0.314 | -1%˄ || 3125.34 | 3169.48 | +1%˄ | 0.1230 |
| TPC-H 15 || 1.107 | 1.096 | -1%˄ || 900.21 | 909.51 | +1%˄ | 0.0001 |
| TPC-H 16 || 1.881 | 1.901 | +1%˄ || 531.03 | 525.51 | -1%˄ | 0.0001 |
-| TPC-H 17 || 0.589 | 0.700 | +19%˄ || 1694.17 | 1424.69 | -16%˄ | 0.1480 |
| TPC-H 18 || 1.076 | 1.121 | +4%˄ || 928.19 | 890.86 | -4%˄ | 0.0000 |
| TPC-H 19 || 4.738 | 4.698 | -1%˄ || 210.99 | 212.75 | +1%˄ | 0.1419 |
| TPC-H 20 || 2.051 | 2.037 | -1%˄ || 487.16 | 490.22 | +1%˄ | 0.7558 |
| TPC-H 21 || 1.346 | 1.381 | +3%˄ || 741.10 | 723.32 | -2%˄ | 0.5508 |
| TPC-H 22 || 0.971 | 0.976 | +1%˄ || 1028.16 | 1022.37 | -1%˄ | 0.0008 |
+----------++----------+---------+--------++----------+----------+--------+---------+
| Sum || 55.9 | 57.4 | +3% || | | | |
| Geomean || | | || | | -2% | |
+----------++----------+---------+--------++----------+----------+--------+---------+
| Notes || ˄ Execution stopped due to max runs reached |
+----------++----------+---------+--------++----------+----------+--------+---------+
hyriseBenchmarkTPCH - multi-threaded, ordered, 1 client, 64 cores, SF 10.0
Sum of avg. item runtimes: +1% || Geometric mean of throughput changes: -1%
Configuration Overview - click to expand
+Configuration Overview---------+----------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_3228b77c02306e4bd15c207f87717af8acd30522_mt_ordered.json | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_833281d1ee76a14a730e02fff4b4a0784c44baf7_mt_ordered.json |
+-------------------------------+----------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 3228b77c02306e4bd15c207f87717af8acd30522-dirty | 833281d1ee76a14a730e02fff4b4a0784c44baf7-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| clustering | None | None |
| compiler | clang 14.0.0 | clang 14.0.0 |
| cores | 9 | 9 |
| data_preparation_cores | 0 | 0 |
| date | 2022-08-27 10:10:14 | 2022-08-27 13:32:17 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 60000000000 | 60000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [9] | [9] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+----------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------+
+----------++----------+----------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
| TPC-H 01 || 5060.072 | 5016.495 | -1% || 0.18 | 0.18 | -0% | (run time too short) |
| TPC-H 02 || 31.253 | 31.086 | -1% || 27.55 | 27.62 | +0% | 0.4970 |
| TPC-H 03 || 972.959 | 974.505 | +0% || 1.02 | 1.02 | +0% | 0.9109 |
| TPC-H 04 || 494.022 | 493.911 | -0% || 2.00 | 2.00 | -0% | 0.9825 |
| TPC-H 05 || 847.294 | 849.567 | +0% || 1.17 | 1.17 | -0% | 0.8128 |
| TPC-H 06 || 44.351 | 44.118 | -1% || 19.63 | 19.70 | +0% | 0.0449 |
| TPC-H 07 || 305.298 | 306.733 | +0% || 3.22 | 3.20 | -1% | 0.2800 |
| TPC-H 08 || 251.425 | 248.549 | -1% || 3.88 | 3.93 | +1% | 0.0054 |
| TPC-H 09 || 2197.256 | 2212.945 | +1% || 0.45 | 0.45 | -0% | 0.3405 |
| TPC-H 10 || 1733.360 | 1752.674 | +1% || 0.57 | 0.57 | -0% | (run time too short) |
| TPC-H 11 || 62.447 | 62.023 | -1% || 14.68 | 14.88 | +1% | 0.0077 |
-| TPC-H 12 || 396.028 | 416.860 | +5% || 2.48 | 2.37 | -5% | 0.0000 |
| TPC-H 13 || 3136.727 | 3124.190 | -0% || 0.32 | 0.32 | -0% | 0.8032 |
| TPC-H 14 || 115.966 | 115.690 | -0% || 8.25 | 8.27 | +0% | 0.2763 |
| TPC-H 15 || 127.687 | 127.975 | +0% || 7.50 | 7.50 | +0% | 0.5755 |
| TPC-H 16 || 597.541 | 609.471 | +2% || 1.65 | 1.62 | -2% | 0.0001 |
| TPC-H 17 || 62.124 | 61.526 | -1% || 14.77 | 14.80 | +0% | 0.0009 |
-| TPC-H 18 || 2145.528 | 2258.271 | +5% || 0.45 | 0.43 | -4% | (run time too short) |
| TPC-H 19 || 96.525 | 96.133 | -0% || 9.72 | 9.82 | +1% | 0.1579 |
| TPC-H 20 || 157.567 | 155.377 | -1% || 6.17 | 6.25 | +1% | 0.0003 |
| TPC-H 21 || 1002.838 | 1002.254 | -0% || 0.98 | 0.98 | -0% | 0.9227 |
-| TPC-H 22 || 128.716 | 135.775 | +5% || 7.53 | 7.08 | -6% | 0.0000 |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
| Sum || 19967.0 | 20096.1 | +1% || | | | |
| Geomean || | | || | | -1% | |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
hyriseBenchmarkTPCH - multi-threaded, shuffled, 64 clients, 64 cores, SF 10.0
Sum of avg. item runtimes: +0% || Geometric mean of throughput changes: -0%
Configuration Overview - click to expand
+Configuration Overview---------+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_3228b77c02306e4bd15c207f87717af8acd30522_mt.json | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_833281d1ee76a14a730e02fff4b4a0784c44baf7_mt.json |
+-------------------------------+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 3228b77c02306e4bd15c207f87717af8acd30522-dirty | 833281d1ee76a14a730e02fff4b4a0784c44baf7-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 9 | 9 |
| clustering | None | None |
| compiler | clang 14.0.0 | clang 14.0.0 |
| cores | 9 | 9 |
| data_preparation_cores | 0 | 0 |
| date | 2022-08-27 10:34:18 | 2022-08-27 13:56:26 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [9] | [9] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
+----------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+----------+--------++----------+----------+--------+---------+
| TPC-H 01 || 5945.329 | 6037.168 | +2% || 0.15 | 0.15 | -0% | 0.6075 |
-| TPC-H 02 || 259.378 | 387.057 | +49% || 0.15 | 0.15 | +0% | 0.3145 |
-| TPC-H 03 || 3311.461 | 3507.495 | +6% || 0.15 | 0.15 | +0% | 0.5558 |
-| TPC-H 04 || 2011.605 | 2419.812 | +20% || 0.15 | 0.15 | -0% | 0.1379 |
+| TPC-H 05 || 4687.323 | 4054.401 | -14% || 0.15 | 0.15 | +0% | 0.1134 |
+| TPC-H 06 || 805.134 | 495.653 | -38% || 0.15 | 0.15 | -0% | 0.0622 |
+| TPC-H 07 || 2672.994 | 2067.638 | -23% || 0.15 | 0.15 | +1% | 0.0850 |
| TPC-H 08 || 2090.211 | 2099.885 | +0% || 0.15 | 0.15 | -0% | 0.9764 |
-| TPC-H 09 || 5696.675 | 6408.858 | +13% || 0.15 | 0.15 | +0% | 0.1005 |
-| TPC-H 10 || 4702.466 | 5275.958 | +12% || 0.15 | 0.15 | +0% | 0.2399 |
+| TPC-H 11 || 372.765 | 340.427 | -9% || 0.15 | 0.15 | -0% | 0.7986 |
-| TPC-H 12 || 2647.415 | 2803.487 | +6% || 0.15 | 0.15 | -0% | 0.6605 |
+| TPC-H 13 || 6334.802 | 5589.628 | -12% || 0.15 | 0.15 | -0% | 0.0460 |
| TPC-H 14 || 1048.955 | 1073.249 | +2% || 0.15 | 0.15 | +0% | 0.9059 |
-| TPC-H 15 || 526.750 | 570.532 | +8% || 0.15 | 0.15 | +0% | 0.6908 |
+| TPC-H 16 || 1989.428 | 1879.174 | -6% || 0.15 | 0.15 | +0% | 0.7326 |
+| TPC-H 17 || 1077.596 | 892.783 | -17% || 0.15 | 0.15 | -1% | 0.4911 |
| TPC-H 18 || 3302.013 | 3199.233 | -3% || 0.15 | 0.15 | +0% | 0.6285 |
-| TPC-H 19 || 654.744 | 767.008 | +17% || 0.15 | 0.15 | -0% | 0.5523 |
-| TPC-H 20 || 1515.146 | 1791.241 | +18% || 0.15 | 0.15 | -0% | 0.3955 |
-| TPC-H 21 || 6176.968 | 6567.009 | +6% || 0.15 | 0.15 | -1% | 0.5642 |
+| TPC-H 22 || 1135.553 | 892.938 | -21% || 0.15 | 0.15 | +0% | 0.2738 |
+----------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 58964.7 | 59120.6 | +0% || | | | |
| Geomean || | | || | | -0% | |
+----------++----------+----------+--------++----------+----------+--------+---------+
hyriseBenchmarkTPCDS - single-threaded
Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +1%
Configuration Overview - click to expand
+Configuration Overview---+---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCDS_3228b77c02306e4bd15c207f87717af8acd30522_st.json | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCDS_833281d1ee76a14a730e02fff4b4a0784c44baf7_st.json |
+-------------------------+---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 3228b77c02306e4bd15c207f87717af8acd30522-dirty | 833281d1ee76a14a730e02fff4b4a0784c44baf7-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 14.0.0 | clang 14.0.0 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2022-08-27 10:56:18 | 2022-08-27 14:18:24 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 100 | 100 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------+---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------+
+---------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+---------+
| 01 || 269.734 | 259.800 | -4%˄ || 3.71 | 3.85 | +4%˄ | 0.0000 |
| 03 || 73.541 | 73.655 | +0%˄ || 13.60 | 13.58 | -0%˄ | 0.8621 |
| 06 || 158.150 | 156.216 | -1%˄ || 6.32 | 6.40 | +1%˄ | 0.0005 |
| 07 || 290.187 | 287.393 | -1%˄ || 3.45 | 3.48 | +1%˄ | 0.1119 |
| 09 || 644.724 | 660.614 | +2% || 1.55 | 1.51 | -2% | 0.0040 |
| 10 || 129.756 | 126.776 | -2%˄ || 7.71 | 7.89 | +2%˄ | 0.0000 |
| 13 || 418.203 | 404.143 | -3%˄ || 2.39 | 2.47 | +3%˄ | 0.0000 |
| 15 || 107.978 | 104.354 | -3%˄ || 9.26 | 9.58 | +3%˄ | 0.0000 |
| 16 || 157.669 | 152.139 | -4%˄ || 6.34 | 6.57 | +4%˄ | 0.0000 |
| 17 || 310.189 | 299.412 | -3%˄ || 3.22 | 3.34 | +4%˄ | 0.0000 |
| 19 || 104.893 | 101.541 | -3%˄ || 9.53 | 9.85 | +3%˄ | 0.0000 |
| 25 || 169.381 | 164.637 | -3%˄ || 5.90 | 6.07 | +3%˄ | 0.0000 |
+| 26 || 137.142 | 127.863 | -7%˄ || 7.29 | 7.82 | +7%˄ | 0.0000 |
| 28 || 618.739 | 595.611 | -4%˄ || 1.62 | 1.68 | +4%˄ | 0.0000 |
| 29 || 471.613 | 457.276 | -3%˄ || 2.12 | 2.19 | +3%˄ | 0.0000 |
+| 31 || 1442.504 | 1216.246 | -16% || 0.69 | 0.82 | +19% | 0.0000 |
| 32 || 45.130 | 44.054 | -2%˄ || 22.16 | 22.70 | +2%˄ | 0.0000 |
| 34 || 161.410 | 155.628 | -4%˄ || 6.20 | 6.43 | +4%˄ | 0.0000 |
| 35 || 626.396 | 616.777 | -2% || 1.60 | 1.62 | +2% | 0.0000 |
| 37 || 265.105 | 261.573 | -1%˄ || 3.77 | 3.82 | +1%˄ | 0.0484 |
| 39a || 1710.581 | 1749.872 | +2% || 0.58 | 0.57 | -2% | 0.0000 |
| 39b || 1706.808 | 1732.971 | +2% || 0.59 | 0.58 | -2% | 0.0016 |
| 41 || 280.894 | 281.462 | +0%˄ || 3.56 | 3.55 | -0%˄ | 0.2653 |
| 42 || 82.194 | 80.470 | -2%˄ || 12.17 | 12.43 | +2%˄ | 0.0000 |
| 43 || 914.124 | 917.814 | +0% || 1.09 | 1.09 | -0% | 0.1623 |
| 45 || 115.658 | 117.517 | +2%˄ || 8.65 | 8.51 | -2%˄ | 0.0000 |
| 48 || 1013.678 | 1016.602 | +0% || 0.99 | 0.98 | -0% | 0.4436 |
| 50 || 110.790 | 111.105 | +0%˄ || 9.03 | 9.00 | -0%˄ | 0.0893 |
| 52 || 79.770 | 79.977 | +0%˄ || 12.54 | 12.50 | -0%˄ | 0.2977 |
| 55 || 75.661 | 76.293 | +1%˄ || 13.22 | 13.11 | -1%˄ | 0.0001 |
| 62 || 524.404 | 526.587 | +0%˄ || 1.91 | 1.90 | -0%˄ | 0.0003 |
| 65 || 1866.594 | 1866.322 | -0% || 0.54 | 0.54 | +0% | 0.9745 |
| 69 || 119.389 | 118.982 | -0%˄ || 8.38 | 8.40 | +0%˄ | 0.3045 |
| 73 || 79.198 | 77.856 | -2%˄ || 12.63 | 12.84 | +2%˄ | 0.0000 |
| 79 || 457.961 | 460.123 | +0%˄ || 2.18 | 2.17 | -0%˄ | 0.0059 |
| 81 || 205.033 | 207.034 | +1%˄ || 4.88 | 4.83 | -1%˄ | 0.0000 |
| 82 || 338.972 | 336.298 | -1%˄ || 2.95 | 2.97 | +1%˄ | 0.1477 |
| 83 || 49.553 | 49.600 | +0%˄ || 20.18 | 20.16 | -0%˄ | 0.9257 |
| 85 || 131.564 | 132.142 | +0%˄ || 7.60 | 7.57 | -0%˄ | 0.0659 |
| 88 || 614.537 | 613.341 | -0% || 1.63 | 1.63 | +0% | 0.1386 |
| 91 || 18.425 | 18.393 | -0%˄ || 54.26 | 54.36 | +0%˄ | 0.0666 |
| 92 || 47.114 | 46.371 | -2%˄ || 21.22 | 21.56 | +2%˄ | 0.0000 |
| 93 || 3661.661 | 3717.472 | +2% || 0.27 | 0.27 | -2% | 0.0002 |
| 94 || 96.418 | 98.258 | +2%˄ || 10.37 | 10.18 | -2%˄ | 0.0000 |
| 95 || 8830.358 | 8683.116 | -2% || 0.11 | 0.12 | +2% | ˅ |
| 96 || 58.406 | 58.367 | -0%˄ || 17.12 | 17.13 | +0%˄ | 0.9102 |
| 97 || 3211.494 | 3230.009 | +1% || 0.31 | 0.31 | -1% | 0.2748 |
| 99 || 999.044 | 999.367 | +0% || 1.00 | 1.00 | -0% | 0.8128 |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 34002.7 | 33669.4 | -1% || | | | |
| Geomean || | | || | | +1% | |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Notes || ˄ Execution stopped due to max runs reached |
| || ˅ Insufficient number of runs for p-value calculation |
+---------++----------+----------+--------++----------+----------+--------+---------+
hyriseBenchmarkTPCDS - multi-threaded, shuffled, 64 clients, 64 cores
Sum of avg. item runtimes: -0% || Geometric mean of throughput changes: +0%
Configuration Overview - click to expand
+Configuration Overview---------+---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCDS_3228b77c02306e4bd15c207f87717af8acd30522_mt.json | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCDS_833281d1ee76a14a730e02fff4b4a0784c44baf7_mt.json |
+-------------------------------+---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 3228b77c02306e4bd15c207f87717af8acd30522-dirty | 833281d1ee76a14a730e02fff4b4a0784c44baf7-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 9 | 9 |
| compiler | clang 14.0.0 | clang 14.0.0 |
| cores | 9 | 9 |
| data_preparation_cores | 0 | 0 |
| date | 2022-08-27 11:22:24 | 2022-08-27 14:44:20 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [9] | [9] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------+
+---------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+---------+
| 01 || 651.388 | 633.094 | -3% || 0.16 | 0.16 | +1% | 0.8738 |
-| 03 || 190.734 | 249.490 | +31% || 0.16 | 0.16 | +1% | 0.3138 |
| 06 || 643.766 | 644.354 | +0% || 0.16 | 0.16 | +1% | 0.9971 |
+| 07 || 856.150 | 719.151 | -16% || 0.16 | 0.16 | +1% | 0.3080 |
| 09 || 1462.744 | 1414.732 | -3% || 0.16 | 0.16 | +1% | 0.8152 |
-| 10 || 514.353 | 573.135 | +11% || 0.16 | 0.16 | +1% | 0.6877 |
+| 13 || 1337.640 | 1044.952 | -22% || 0.16 | 0.16 | +1% | 0.0851 |
+| 15 || 431.090 | 285.049 | -34% || 0.16 | 0.16 | +1% | 0.0838 |
+| 16 || 412.548 | 373.628 | -9% || 0.16 | 0.16 | +1% | 0.6131 |
| 17 || 1018.938 | 986.775 | -3% || 0.16 | 0.16 | +0% | 0.8899 |
+| 19 || 484.252 | 375.127 | -23% || 0.16 | 0.16 | +0% | 0.4499 |
-| 25 || 696.380 | 743.030 | +7% || 0.16 | 0.16 | +1% | 0.7779 |
| 26 || 446.261 | 442.172 | -1% || 0.16 | 0.16 | +0% | 0.9732 |
+| 28 || 1339.691 | 1053.410 | -21% || 0.16 | 0.16 | +1% | 0.1633 |
+| 29 || 1199.895 | 1076.690 | -10% || 0.16 | 0.16 | +1% | 0.4544 |
+| 31 || 2244.589 | 1804.544 | -20% || 0.16 | 0.16 | +0% | 0.1086 |
+| 32 || 278.247 | 167.163 | -40% || 0.16 | 0.16 | +1% | 0.1407 |
+| 34 || 580.663 | 521.646 | -10% || 0.16 | 0.16 | +1% | 0.6099 |
+| 35 || 1613.168 | 1455.854 | -10% || 0.16 | 0.16 | +1% | 0.3966 |
| 37 || 624.350 | 635.145 | +2% || 0.16 | 0.16 | +1% | 0.9302 |
-| 39a || 2015.926 | 2136.236 | +6% || 0.16 | 0.16 | +1% | 0.4794 |
| 39b || 2173.998 | 2096.131 | -4% || 0.16 | 0.16 | +0% | 0.7063 |
-| 41 || 2164.153 | 2840.557 | +31% || 0.16 | 0.16 | +1% | 0.0205 |
+| 42 || 341.515 | 219.181 | -36% || 0.16 | 0.16 | +0% | 0.1202 |
-| 43 || 1416.569 | 1807.128 | +28% || 0.16 | 0.16 | +1% | 0.0268 |
-| 45 || 383.112 | 574.326 | +50% || 0.16 | 0.16 | +1% | 0.2092 |
-| 48 || 1689.639 | 2102.169 | +24% || 0.16 | 0.16 | +1% | 0.0797 |
+| 50 || 681.221 | 433.218 | -36% || 0.16 | 0.16 | +1% | 0.1276 |
+| 52 || 371.260 | 279.675 | -25% || 0.16 | 0.16 | +1% | 0.2777 |
-| 55 || 248.803 | 374.379 | +50% || 0.16 | 0.16 | +1% | 0.1855 |
-| 62 || 895.066 | 936.902 | +5% || 0.16 | 0.16 | +1% | 0.7099 |
-| 65 || 3195.421 | 3372.873 | +6% || 0.16 | 0.16 | +0% | 0.2996 |
+| 69 || 680.296 | 615.229 | -10% || 0.16 | 0.16 | +1% | 0.6868 |
+| 73 || 281.993 | 243.648 | -14% || 0.16 | 0.16 | +0% | 0.4261 |
+| 79 || 1209.827 | 1033.887 | -15% || 0.16 | 0.16 | +1% | 0.2281 |
-| 81 || 524.451 | 588.739 | +12% || 0.16 | 0.16 | +0% | 0.4727 |
-| 82 || 842.667 | 1027.171 | +22% || 0.16 | 0.16 | +1% | 0.3186 |
+| 83 || 280.161 | 243.805 | -13% || 0.16 | 0.16 | +1% | 0.7303 |
+| 85 || 682.030 | 539.288 | -21% || 0.16 | 0.16 | +1% | 0.2822 |
-| 88 || 1394.429 | 1551.799 | +11% || 0.16 | 0.16 | +1% | 0.5371 |
-| 91 || 176.168 | 246.914 | +40% || 0.16 | 0.16 | +1% | 0.4825 |
-| 92 || 238.703 | 258.704 | +8% || 0.16 | 0.16 | +1% | 0.7796 |
| 93 || 2516.612 | 2612.020 | +4% || 0.16 | 0.16 | +1% | 0.5731 |
-| 94 || 335.911 | 549.745 | +64% || 0.16 | 0.16 | +1% | 0.0906 |
| 95 || 8456.544 | 8415.918 | -0% || 0.16 | 0.16 | +1% | 0.9064 |
+| 96 || 242.198 | 205.150 | -15% || 0.16 | 0.16 | +0% | 0.5217 |
| 97 || 4037.367 | 3954.251 | -2% || 0.16 | 0.16 | +0% | 0.7031 |
+| 99 || 1557.170 | 1373.885 | -12% || 0.16 | 0.16 | +1% | 0.2769 |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 56060.1 | 55832.1 | -0% || | | | |
| Geomean || | | || | | +0% | |
+---------++----------+----------+--------++----------+----------+--------+---------+
hyriseBenchmarkTPCC - single-threaded
Sum of avg. item runtimes: -0% || Geometric mean of throughput changes: +0%
Configuration Overview - click to expand
+Configuration Overview---+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCC_3228b77c02306e4bd15c207f87717af8acd30522_st.json | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCC_833281d1ee76a14a730e02fff4b4a0784c44baf7_st.json |
+-------------------------+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 3228b77c02306e4bd15c207f87717af8acd30522-dirty | 833281d1ee76a14a730e02fff4b4a0784c44baf7-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 14.0.0 | clang 14.0.0 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2022-08-27 11:42:53 | 2022-08-27 15:04:50 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 60000000000 | 60000000000 |
| max_runs | -1 | -1 |
| scale_factor | 1 | 1 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Delivery || 23.070 | 22.914 | -1% || 4.25 | 4.27 | +0% | 0.0680 |
| New-Order || 16.376 | 16.359 | -0% || 47.98 | 48.08 | +0% | 0.8967 |
| Order-Status || 1.089 | 1.082 | -1% || 4.25 | 4.27 | +0% | 0.4567 |
| Payment || 2.060 | 2.048 | -1% || 45.86 | 45.91 | +0% | 0.0512 |
| Stock-Level || 3.551 | 3.543 | -0% || 4.27 | 4.27 | -0% | 0.7768 |
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Sum || 46.1 | 45.9 | -0% || | | | |
| Geomean || | | || | | +0% | |
+--------------++----------+---------+--------++----------+----------+--------+---------+
hyriseBenchmarkTPCC - multi-threaded, shuffled, 64 clients, 64 cores
Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +0%
Configuration Overview - click to expand
+Configuration Overview---------+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCC_3228b77c02306e4bd15c207f87717af8acd30522_mt.json | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCC_833281d1ee76a14a730e02fff4b4a0784c44baf7_mt.json |
+-------------------------------+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 3228b77c02306e4bd15c207f87717af8acd30522-dirty | 833281d1ee76a14a730e02fff4b4a0784c44baf7-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 9 | 9 |
| compiler | clang 14.0.0 | clang 14.0.0 |
| cores | 9 | 9 |
| data_preparation_cores | 0 | 0 |
| date | 2022-08-27 11:43:53 | 2022-08-27 15:05:51 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| scale_factor | 1 | 1 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [9] | [9] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Delivery || 132.250 | 130.275 | -1% || 5.74 | 5.79 | +1% | 0.0716 |
| unsucc.: || 2.0 | 2.0 | -2% || 11.38 | 11.39 | +0% | |
| New-Order || 36.769 | 36.713 | -0% || 114.66 | 114.24 | -0% | 0.4390 |
| unsucc.: || 3.3 | 3.5 | +4% || 77.92 | 79.01 | +1% | |
| Order-Status || 5.069 | 4.971 | -2% || 17.12 | 17.18 | +0% | 0.0053 |
| Payment || 5.425 | 5.414 | -0% || 46.72 | 47.05 | +1% | 0.3589 |
| unsucc.: || 2.1 | 2.1 | -0% || 137.31 | 137.61 | +0% | |
| Stock-Level || 11.626 | 11.502 | -1% || 17.12 | 17.18 | +0% | 0.2546 |
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Sum || 191.1 | 188.9 | -1% || | | | |
| Geomean || | | || | | +0% | |
+--------------++----------+---------+--------++----------+----------+--------+---------+
hyriseBenchmarkJoinOrder - single-threaded
Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +0%
Configuration Overview - click to expand
+Configuration Overview---+-------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkJoinOrder_3228b77c02306e4bd15c207f87717af8acd30522_st.json | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkJoinOrder_833281d1ee76a14a730e02fff4b4a0784c44baf7_st.json |
+-------------------------+-------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 3228b77c02306e4bd15c207f87717af8acd30522-dirty | 833281d1ee76a14a730e02fff4b4a0784c44baf7-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 14.0.0 | clang 14.0.0 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2022-08-27 12:03:57 | 2022-08-27 15:25:54 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 100 | 100 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------+-------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+
+---------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+---------+
+| 10a || 183.613 | 174.250 | -5%˄ || 5.45 | 5.74 | +5%˄ | 0.0000 |
-| 10b || 111.638 | 124.133 | +11%˄ || 8.96 | 8.06 | -10%˄ | 0.0000 |
| 10c || 320.593 | 321.829 | +0%˄ || 3.12 | 3.11 | -0%˄ | 0.3592 |
| 11a || 16.427 | 16.623 | +1%˄ || 60.86 | 60.15 | -1%˄ | 0.1090 |
| 11b || 16.262 | 16.283 | +0%˄ || 61.48 | 61.40 | -0%˄ | 0.5742 |
| 11c || 22.105 | 21.849 | -1%˄ || 45.23 | 45.76 | +1%˄ | 0.0004 |
| 11d || 24.738 | 24.989 | +1%˄ || 40.42 | 40.01 | -1%˄ | 0.2542 |
| 12a || 51.497 | 50.858 | -1%˄ || 19.42 | 19.66 | +1%˄ | 0.0000 |
| 12b || 29.831 | 28.709 | -4%˄ || 33.52 | 34.83 | +4%˄ | 0.0000 |
| 12c || 88.659 | 88.364 | -0%˄ || 11.28 | 11.32 | +0%˄ | 0.2915 |
| 13a || 285.775 | 277.667 | -3%˄ || 3.50 | 3.60 | +3%˄ | 0.0000 |
+| 13b || 178.365 | 156.415 | -12%˄ || 5.61 | 6.39 | +14%˄ | 0.0000 |
+| 13c || 135.201 | 119.942 | -11%˄ || 7.40 | 8.34 | +13%˄ | 0.0000 |
+| 13d || 622.639 | 590.308 | -5%˄ || 1.61 | 1.69 | +5%˄ | 0.0000 |
| 14a || 420.039 | 413.331 | -2%˄ || 2.38 | 2.42 | +2%˄ | 0.0000 |
| 14b || 237.885 | 231.705 | -3%˄ || 4.20 | 4.32 | +3%˄ | 0.0000 |
+| 14c || 513.357 | 489.895 | -5%˄ || 1.95 | 2.04 | +5%˄ | 0.0000 |
| 15a || 82.431 | 84.123 | +2%˄ || 12.13 | 11.89 | -2%˄ | 0.0000 |
| 15b || 81.979 | 80.816 | -1%˄ || 12.20 | 12.37 | +1%˄ | 0.0000 |
| 15c || 88.473 | 87.470 | -1%˄ || 11.30 | 11.43 | +1%˄ | 0.0000 |
| 15d || 82.436 | 81.956 | -1%˄ || 12.13 | 12.20 | +1%˄ | 0.0032 |
| 16a || 2179.182 | 2143.911 | -2% || 0.46 | 0.47 | +2% | 0.0003 |
| 16b || 3152.478 | 3158.711 | +0% || 0.32 | 0.32 | -0% | 0.5861 |
| 16c || 2230.260 | 2240.602 | +0% || 0.45 | 0.45 | -0% | 0.3060 |
| 16d || 2205.176 | 2199.923 | -0% || 0.45 | 0.45 | +0% | 0.4117 |
| 17a || 565.333 | 564.388 | -0%˄ || 1.77 | 1.77 | +0%˄ | 0.0621 |
| 17b || 443.005 | 440.416 | -1%˄ || 2.26 | 2.27 | +1%˄ | 0.0000 |
| 17c || 419.906 | 418.188 | -0%˄ || 2.38 | 2.39 | +0%˄ | 0.0000 |
| 17d || 496.043 | 493.633 | -0%˄ || 2.02 | 2.03 | +0%˄ | 0.0000 |
| 17e || 1556.262 | 1536.289 | -1% || 0.64 | 0.65 | +1% | 0.0000 |
| 17f || 914.248 | 913.894 | -0% || 1.09 | 1.09 | +0% | 0.5712 |
| 18a || 596.663 | 595.795 | -0%˄ || 1.68 | 1.68 | +0%˄ | 0.0388 |
| 18b || 109.478 | 109.696 | +0%˄ || 9.13 | 9.12 | -0%˄ | 0.2483 |
| 18c || 552.104 | 555.718 | +1%˄ || 1.81 | 1.80 | -1%˄ | 0.0000 |
| 19a || 178.086 | 176.026 | -1%˄ || 5.62 | 5.68 | +1%˄ | 0.0000 |
| 19b || 125.111 | 124.248 | -1%˄ || 7.99 | 8.05 | +1%˄ | 0.0000 |
| 19c || 189.653 | 189.316 | -0%˄ || 5.27 | 5.28 | +0%˄ | 0.0540 |
| 19d || 716.273 | 714.596 | -0% || 1.40 | 1.40 | +0% | 0.0026 |
| 1a || 10.569 | 10.727 | +1%˄ || 94.60 | 93.20 | -1%˄ | 0.0000 |
| 1b || 9.933 | 10.081 | +1%˄ || 100.65 | 99.18 | -1%˄ | 0.0000 |
| 1c || 11.586 | 11.435 | -1%˄ || 86.29 | 87.43 | +1%˄ | 0.0000 |
| 1d || 9.993 | 10.100 | +1%˄ || 100.05 | 98.99 | -1%˄ | 0.0000 |
| 20a || 390.295 | 375.645 | -4%˄ || 2.56 | 2.66 | +4%˄ | 0.0000 |
| 20b || 233.765 | 229.849 | -2%˄ || 4.28 | 4.35 | +2%˄ | 0.0000 |
| 20c || 226.824 | 226.902 | +0%˄ || 4.41 | 4.41 | -0%˄ | 0.5146 |
| 21a || 34.895 | 34.628 | -1%˄ || 28.65 | 28.88 | +1%˄ | 0.0000 |
| 21b || 20.477 | 20.411 | -0%˄ || 48.82 | 48.98 | +0%˄ | 0.0064 |
| 21c || 34.902 | 34.846 | -0%˄ || 28.65 | 28.70 | +0%˄ | 0.0962 |
| 22a || 197.688 | 197.506 | -0%˄ || 5.06 | 5.06 | +0%˄ | 0.4113 |
| 22b || 144.023 | 144.372 | +0%˄ || 6.94 | 6.93 | -0%˄ | 0.0011 |
| 22c || 488.080 | 489.352 | +0%˄ || 2.05 | 2.04 | -0%˄ | 0.0178 |
| 22d || 891.582 | 883.835 | -1% || 1.12 | 1.13 | +1% | 0.0000 |
| 23a || 43.863 | 43.683 | -0%˄ || 22.80 | 22.89 | +0%˄ | 0.0061 |
| 23b || 50.616 | 48.839 | -4%˄ || 19.76 | 20.47 | +4%˄ | 0.0000 |
| 23c || 85.265 | 85.482 | +0%˄ || 11.73 | 11.70 | -0%˄ | 0.1093 |
| 24a || 140.366 | 139.592 | -1%˄ || 7.12 | 7.16 | +1%˄ | 0.0040 |
| 24b || 117.790 | 117.751 | -0%˄ || 8.49 | 8.49 | +0%˄ | 0.8284 |
| 25a || 271.334 | 271.507 | +0%˄ || 3.69 | 3.68 | -0%˄ | 0.4121 |
| 25b || 108.470 | 108.650 | +0%˄ || 9.22 | 9.20 | -0%˄ | 0.4028 |
| 25c || 925.166 | 923.490 | -0% || 1.08 | 1.08 | +0% | 0.1642 |
| 26a || 169.842 | 169.807 | -0%˄ || 5.89 | 5.89 | +0%˄ | 0.7737 |
| 26b || 133.655 | 132.818 | -1%˄ || 7.48 | 7.53 | +1%˄ | 0.0014 |
| 26c || 244.166 | 244.322 | +0%˄ || 4.10 | 4.09 | -0%˄ | 0.5239 |
| 27a || 24.597 | 24.631 | +0%˄ || 40.65 | 40.59 | -0%˄ | 0.4207 |
| 27b || 24.131 | 24.173 | +0%˄ || 41.43 | 41.36 | -0%˄ | 0.1354 |
| 27c || 34.876 | 34.862 | -0%˄ || 28.67 | 28.68 | +0%˄ | 0.7088 |
| 28a || 293.147 | 293.171 | +0%˄ || 3.41 | 3.41 | -0%˄ | 0.9305 |
| 28b || 39.882 | 39.984 | +0%˄ || 25.07 | 25.01 | -0%˄ | 0.0573 |
| 28c || 329.406 | 328.162 | -0%˄ || 3.04 | 3.05 | +0%˄ | 0.0000 |
| 29a || 102.129 | 101.680 | -0%˄ || 9.79 | 9.83 | +0%˄ | 0.0000 |
| 29b || 74.932 | 74.333 | -1%˄ || 13.34 | 13.45 | +1%˄ | 0.0001 |
| 29c || 144.523 | 144.485 | -0%˄ || 6.92 | 6.92 | +0%˄ | 0.8575 |
| 2a || 51.294 | 51.285 | -0%˄ || 19.49 | 19.50 | +0%˄ | 0.8509 |
| 2b || 41.875 | 41.985 | +0%˄ || 23.88 | 23.82 | -0%˄ | 0.0407 |
| 2c || 22.860 | 22.963 | +0%˄ || 43.74 | 43.54 | -0%˄ | 0.0000 |
| 2d || 72.708 | 73.102 | +1%˄ || 13.75 | 13.68 | -1%˄ | 0.0001 |
| 30a || 146.613 | 146.620 | +0%˄ || 6.82 | 6.82 | -0%˄ | 0.9371 |
| 30b || 129.759 | 130.038 | +0%˄ || 7.71 | 7.69 | -0%˄ | 0.0209 |
| 30c || 373.215 | 372.248 | -0%˄ || 2.68 | 2.69 | +0%˄ | 0.0254 |
| 31a || 142.365 | 142.374 | +0%˄ || 7.02 | 7.02 | -0%˄ | 0.9496 |
| 31b || 119.315 | 119.848 | +0%˄ || 8.38 | 8.34 | -0%˄ | 0.0136 |
| 31c || 168.270 | 166.067 | -1%˄ || 5.94 | 6.02 | +1%˄ | 0.0000 |
| 32a || 15.882 | 15.639 | -2%˄ || 62.95 | 63.92 | +2%˄ | 0.1702 |
| 32b || 37.587 | 37.755 | +0%˄ || 26.60 | 26.48 | -0%˄ | 0.0363 |
| 33a || 22.827 | 23.231 | +2%˄ || 43.80 | 43.04 | -2%˄ | 0.0000 |
| 33b || 22.239 | 22.655 | +2%˄ || 44.96 | 44.13 | -2%˄ | 0.0000 |
| 33c || 25.859 | 26.293 | +2%˄ || 38.66 | 38.03 | -2%˄ | 0.0000 |
| 3a || 124.438 | 124.964 | +0%˄ || 8.04 | 8.00 | -0%˄ | 0.0342 |
| 3b || 14.108 | 14.208 | +1%˄ || 70.87 | 70.37 | -1%˄ | 0.0000 |
| 3c || 449.906 | 441.562 | -2%˄ || 2.22 | 2.26 | +2%˄ | 0.0000 |
| 4a || 114.967 | 114.695 | -0%˄ || 8.70 | 8.72 | +0%˄ | 0.3968 |
| 4b || 12.891 | 13.140 | +2%˄ || 77.56 | 76.09 | -2%˄ | 0.0000 |
| 4c || 127.010 | 127.286 | +0%˄ || 7.87 | 7.86 | -0%˄ | 0.0518 |
| 5a || 58.807 | 58.716 | -0%˄ || 17.00 | 17.03 | +0%˄ | 0.2796 |
| 5b || 139.639 | 138.409 | -1%˄ || 7.16 | 7.22 | +1%˄ | 0.0000 |
| 5c || 193.384 | 193.290 | -0%˄ || 5.17 | 5.17 | +0%˄ | 0.7307 |
| 6a || 98.820 | 99.197 | +0%˄ || 10.12 | 10.08 | -0%˄ | 0.0395 |
| 6b || 117.711 | 117.226 | -0%˄ || 8.50 | 8.53 | +0%˄ | 0.0000 |
| 6c || 93.813 | 94.384 | +1%˄ || 10.66 | 10.59 | -1%˄ | 0.0000 |
| 6d || 509.882 | 510.144 | +0%˄ || 1.96 | 1.96 | -0%˄ | 0.4468 |
| 6e || 98.591 | 99.191 | +1%˄ || 10.14 | 10.08 | -1%˄ | 0.0000 |
| 6f || 681.298 | 680.914 | -0% || 1.47 | 1.47 | +0% | 0.4217 |
| 7a || 53.245 | 53.369 | +0%˄ || 18.78 | 18.74 | -0%˄ | 0.2600 |
| 7b || 48.725 | 48.706 | -0%˄ || 20.52 | 20.53 | +0%˄ | 0.8713 |
| 7c || 572.342 | 571.674 | -0%˄ || 1.75 | 1.75 | +0%˄ | 0.5618 |
| 8a || 38.721 | 38.509 | -1%˄ || 25.82 | 25.97 | +1%˄ | 0.0274 |
| 8b || 36.112 | 35.966 | -0%˄ || 27.69 | 27.80 | +0%˄ | 0.0565 |
| 8c || 1839.838 | 1835.074 | -0% || 0.54 | 0.54 | +0% | 0.0696 |
| 8d || 305.047 | 304.544 | -0%˄ || 3.28 | 3.28 | +0%˄ | 0.1247 |
| 9a || 186.154 | 184.862 | -1%˄ || 5.37 | 5.41 | +1%˄ | 0.0000 |
| 9b || 108.537 | 108.221 | -0%˄ || 9.21 | 9.24 | +0%˄ | 0.0709 |
| 9c || 272.787 | 270.256 | -1%˄ || 3.67 | 3.70 | +1%˄ | 0.0000 |
| 9d || 477.024 | 474.937 | -0%˄ || 2.10 | 2.11 | +0%˄ | 0.0000 |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 34452.4 | 34233.5 | -1% || | | | |
| Geomean || | | || | | +0% | |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+----------+--------++----------+----------+--------+---------+
hyriseBenchmarkJoinOrder - multi-threaded, shuffled, 64 clients, 64 cores
Sum of avg. item runtimes: -0% || Geometric mean of throughput changes: +0%
Configuration Overview - click to expand
+Configuration Overview---------+-------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkJoinOrder_3228b77c02306e4bd15c207f87717af8acd30522_mt.json | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkJoinOrder_833281d1ee76a14a730e02fff4b4a0784c44baf7_mt.json |
+-------------------------------+-------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 3228b77c02306e4bd15c207f87717af8acd30522-dirty | 833281d1ee76a14a730e02fff4b4a0784c44baf7-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 9 | 9 |
| compiler | clang 14.0.0 | clang 14.0.0 |
| cores | 9 | 9 |
| data_preparation_cores | 0 | 0 |
| date | 2022-08-27 12:46:18 | 2022-08-27 16:07:57 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [9] | [9] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+-------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+
+---------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+---------+
+| 10a || 247.449 | 227.357 | -8% || 0.17 | 0.17 | -0% | 0.6711 |
+| 10b || 179.341 | 158.737 | -11% || 0.17 | 0.17 | -0% | 0.5649 |
-| 10c || 375.961 | 414.434 | +10% || 0.17 | 0.17 | -0% | 0.4467 |
| 11a || 46.706 | 48.312 | +3% || 0.17 | 0.17 | -1% | 0.8761 |
-| 11b || 58.024 | 80.797 | +39% || 0.17 | 0.17 | -0% | 0.4204 |
-| 11c || 63.104 | 99.185 | +57% || 0.17 | 0.17 | -0% | 0.1620 |
+| 11d || 99.609 | 86.244 | -13% || 0.17 | 0.17 | -0% | 0.6477 |
+| 12a || 174.544 | 108.876 | -38% || 0.17 | 0.17 | -0% | 0.0810 |
-| 12b || 96.808 | 138.543 | +43% || 0.17 | 0.17 | -0% | 0.4008 |
-| 12c || 252.986 | 301.283 | +19% || 0.17 | 0.17 | -0% | 0.3071 |
| 13a || 458.088 | 469.664 | +3% || 0.17 | 0.17 | -0% | 0.8189 |
-| 13b || 236.554 | 268.951 | +14% || 0.17 | 0.17 | -0% | 0.5524 |
-| 13c || 188.173 | 199.747 | +6% || 0.17 | 0.17 | -0% | 0.7323 |
+| 13d || 946.469 | 837.342 | -12% || 0.17 | 0.17 | -0% | 0.2011 |
| 14a || 705.466 | 728.938 | +3% || 0.17 | 0.17 | -0% | 0.7707 |
-| 14b || 404.006 | 537.359 | +33% || 0.17 | 0.17 | -0% | 0.0410 |
| 14c || 736.335 | 725.666 | -1% || 0.17 | 0.17 | -0% | 0.8550 |
+| 15a || 175.579 | 137.678 | -22% || 0.17 | 0.17 | -0% | 0.2956 |
+| 15b || 169.005 | 132.432 | -22% || 0.17 | 0.17 | -0% | 0.1948 |
-| 15c || 140.304 | 182.421 | +30% || 0.17 | 0.17 | +0% | 0.2109 |
-| 15d || 152.463 | 170.792 | +12% || 0.17 | 0.17 | -0% | 0.5578 |
| 16a || 2591.281 | 2563.573 | -1% || 0.17 | 0.16 | -1% | 0.8284 |
| 16b || 3712.850 | 3827.197 | +3% || 0.17 | 0.17 | -0% | 0.3367 |
| 16c || 2621.026 | 2588.688 | -1% || 0.17 | 0.17 | -0% | 0.7612 |
-| 16d || 2566.421 | 2777.787 | +8% || 0.16 | 0.17 | +0% | 0.0904 |
| 17a || 959.350 | 955.507 | -0% || 0.17 | 0.17 | -0% | 0.9707 |
| 17b || 693.856 | 724.423 | +4% || 0.17 | 0.17 | -0% | 0.7320 |
| 17c || 664.481 | 652.229 | -2% || 0.17 | 0.17 | -0% | 0.8784 |
| 17d || 761.351 | 786.703 | +3% || 0.17 | 0.16 | -1% | 0.7835 |
| 17e || 1803.804 | 1764.366 | -2% || 0.17 | 0.17 | -0% | 0.7466 |
| 17f || 1428.526 | 1373.086 | -4% || 0.17 | 0.17 | -0% | 0.6701 |
+| 18a || 858.675 | 790.696 | -8% || 0.17 | 0.17 | +0% | 0.5180 |
+| 18b || 190.865 | 174.616 | -9% || 0.17 | 0.17 | +0% | 0.7122 |
+| 18c || 910.690 | 795.818 | -13% || 0.17 | 0.17 | -0% | 0.2617 |
+| 19a || 549.959 | 454.503 | -17% || 0.17 | 0.17 | -0% | 0.2985 |
+| 19b || 446.807 | 326.602 | -27% || 0.17 | 0.17 | -0% | 0.1127 |
-| 19c || 441.916 | 480.061 | +9% || 0.17 | 0.17 | -0% | 0.5483 |
+| 19d || 1295.755 | 1206.027 | -7% || 0.17 | 0.17 | -0% | 0.3979 |
| 1a || 37.368 | 35.758 | -4% || 0.17 | 0.17 | +0% | 0.9303 |
| 1b || 32.108 | 33.307 | +4% || 0.17 | 0.17 | -0% | 0.9137 |
+| 1c || 48.407 | 25.059 | -48% || 0.17 | 0.17 | -0% | 0.2451 |
+| 1d || 36.722 | 27.130 | -26% || 0.17 | 0.17 | -0% | 0.6157 |
-| 20a || 613.766 | 648.387 | +6% || 0.17 | 0.17 | -0% | 0.6472 |
-| 20b || 380.845 | 582.873 | +53% || 0.17 | 0.17 | -0% | 0.0406 |
-| 20c || 385.869 | 425.418 | +10% || 0.17 | 0.17 | -0% | 0.5271 |
+| 21a || 133.926 | 82.821 | -38% || 0.17 | 0.17 | -0% | 0.2185 |
-| 21b || 64.729 | 100.601 | +55% || 0.17 | 0.17 | -0% | 0.2904 |
+| 21c || 99.254 | 87.564 | -12% || 0.17 | 0.17 | +0% | 0.6990 |
| 22a || 510.122 | 508.666 | -0% || 0.17 | 0.17 | -0% | 0.9860 |
-| 22b || 247.380 | 313.146 | +27% || 0.17 | 0.17 | -0% | 0.2174 |
-| 22c || 754.476 | 800.653 | +6% || 0.17 | 0.17 | -0% | 0.5612 |
+| 22d || 1135.754 | 1027.904 | -9% || 0.16 | 0.17 | +0% | 0.1649 |
+| 23a || 257.018 | 147.100 | -43% || 0.17 | 0.17 | -0% | 0.0620 |
+| 23b || 146.015 | 137.376 | -6% || 0.17 | 0.17 | -0% | 0.8508 |
-| 23c || 200.873 | 217.630 | +8% || 0.17 | 0.17 | -0% | 0.7870 |
+| 24a || 305.916 | 267.985 | -12% || 0.17 | 0.17 | +0% | 0.4861 |
+| 24b || 282.885 | 211.132 | -25% || 0.17 | 0.17 | -0% | 0.1865 |
-| 25a || 425.694 | 485.442 | +14% || 0.17 | 0.17 | -0% | 0.3945 |
-| 25b || 175.199 | 183.186 | +5% || 0.17 | 0.17 | -0% | 0.8255 |
| 25c || 1038.432 | 1063.283 | +2% || 0.17 | 0.17 | +0% | 0.8006 |
| 26a || 336.672 | 325.369 | -3% || 0.17 | 0.17 | -0% | 0.8317 |
-| 26b || 178.047 | 250.959 | +41% || 0.17 | 0.17 | +0% | 0.0556 |
-| 26c || 400.893 | 461.451 | +15% || 0.17 | 0.17 | +0% | 0.2779 |
-| 27a || 90.706 | 98.305 | +8% || 0.17 | 0.17 | +0% | 0.8249 |
+| 27b || 101.952 | 94.093 | -8% || 0.17 | 0.17 | -1% | 0.7981 |
+| 27c || 156.208 | 99.240 | -36% || 0.16 | 0.17 | +0% | 0.2835 |
-| 28a || 543.324 | 579.425 | +7% || 0.17 | 0.17 | -0% | 0.5783 |
+| 28b || 211.222 | 185.883 | -12% || 0.17 | 0.17 | -0% | 0.6055 |
+| 28c || 711.396 | 662.232 | -7% || 0.17 | 0.17 | -0% | 0.6030 |
+| 29a || 212.918 | 192.346 | -10% || 0.17 | 0.17 | -0% | 0.5458 |
+| 29b || 217.157 | 165.612 | -24% || 0.17 | 0.17 | -0% | 0.2936 |
-| 29c || 253.227 | 296.255 | +17% || 0.17 | 0.17 | +0% | 0.3508 |
+| 2a || 169.558 | 138.947 | -18% || 0.17 | 0.17 | -0% | 0.4390 |
+| 2b || 190.633 | 102.530 | -46% || 0.17 | 0.17 | +0% | 0.0337 |
-| 2c || 58.036 | 78.227 | +35% || 0.17 | 0.17 | +0% | 0.3765 |
-| 2d || 215.913 | 271.680 | +26% || 0.17 | 0.17 | +0% | 0.4539 |
+| 30a || 392.222 | 347.349 | -11% || 0.17 | 0.17 | -0% | 0.5565 |
-| 30b || 270.400 | 349.155 | +29% || 0.17 | 0.17 | -0% | 0.2791 |
-| 30c || 698.103 | 731.048 | +5% || 0.17 | 0.17 | +0% | 0.7165 |
+| 31a || 303.320 | 272.816 | -10% || 0.17 | 0.17 | +0% | 0.5940 |
-| 31b || 160.633 | 271.936 | +69% || 0.17 | 0.17 | -0% | 0.0496 |
+| 31c || 377.190 | 324.952 | -14% || 0.17 | 0.17 | -0% | 0.4230 |
-| 32a || 49.108 | 98.731 | +101% || 0.17 | 0.17 | -0% | 0.2153 |
-| 32b || 121.631 | 156.804 | +29% || 0.17 | 0.17 | +0% | 0.3600 |
+| 33a || 103.693 | 67.782 | -35% || 0.17 | 0.17 | -0% | 0.2560 |
-| 33b || 78.573 | 86.735 | +10% || 0.17 | 0.17 | -0% | 0.7845 |
-| 33c || 84.691 | 90.638 | +7% || 0.17 | 0.17 | +0% | 0.8695 |
+| 3a || 294.651 | 244.847 | -17% || 0.17 | 0.17 | +0% | 0.2703 |
-| 3b || 37.241 | 60.718 | +63% || 0.17 | 0.17 | +0% | 0.1390 |
| 3c || 576.487 | 561.384 | -3% || 0.17 | 0.17 | -0% | 0.8378 |
-| 4a || 179.592 | 206.397 | +15% || 0.17 | 0.17 | +0% | 0.5502 |
-| 4b || 51.635 | 67.280 | +30% || 0.17 | 0.17 | -0% | 0.7267 |
-| 4c || 171.103 | 199.969 | +17% || 0.17 | 0.17 | +0% | 0.3815 |
-| 5a || 135.994 | 190.228 | +40% || 0.17 | 0.17 | +0% | 0.1402 |
+| 5b || 205.250 | 164.951 | -20% || 0.17 | 0.17 | -0% | 0.2303 |
+| 5c || 371.825 | 322.832 | -13% || 0.17 | 0.17 | -0% | 0.5298 |
| 6a || 159.576 | 162.552 | +2% || 0.17 | 0.17 | -0% | 0.9390 |
-| 6b || 251.509 | 279.470 | +11% || 0.17 | 0.17 | +0% | 0.6674 |
-| 6c || 136.132 | 146.636 | +8% || 0.17 | 0.17 | -0% | 0.7604 |
+| 6d || 889.355 | 647.002 | -27% || 0.17 | 0.17 | -0% | 0.0134 |
+| 6e || 169.174 | 99.614 | -41% || 0.17 | 0.17 | -0% | 0.0565 |
| 6f || 1006.107 | 976.882 | -3% || 0.17 | 0.17 | -0% | 0.6570 |
-| 7a || 105.174 | 114.955 | +9% || 0.17 | 0.17 | -0% | 0.7121 |
-| 7b || 127.736 | 136.666 | +7% || 0.17 | 0.17 | -0% | 0.8305 |
| 7c || 980.388 | 1020.097 | +4% || 0.17 | 0.17 | -0% | 0.6524 |
| 8a || 137.189 | 142.334 | +4% || 0.17 | 0.17 | +0% | 0.8958 |
-| 8b || 86.441 | 138.433 | +60% || 0.17 | 0.17 | +0% | 0.0752 |
| 8c || 2160.848 | 2204.234 | +2% || 0.17 | 0.17 | -0% | 0.6991 |
+| 8d || 584.476 | 449.275 | -23% || 0.17 | 0.17 | +0% | 0.0283 |
+| 9a || 617.087 | 487.714 | -21% || 0.17 | 0.17 | -0% | 0.1505 |
-| 9b || 238.783 | 319.376 | +34% || 0.17 | 0.17 | +0% | 0.1516 |
-| 9c || 590.724 | 627.333 | +6% || 0.17 | 0.17 | -1% | 0.6009 |
-| 9d || 914.318 | 1057.809 | +16% || 0.17 | 0.17 | -0% | 0.1043 |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 53583.6 | 53538.5 | -0% || | | | |
| Geomean || | | || | | +0% | |
+---------++----------+----------+--------++----------+----------+--------+---------+
System
node-19 - click to expand
| property | value |
|---|---|
| Hostname | node-19 |
| CPU | AMD EPYC 7742 64-Core Processor |
| Memory | 499GB |
| numactl | nodebind: 0 1 |
| numactl | membind: 0 1 |
Commit Info and Build Time
| commit | date | message | build time |
|---|---|---|---|
| 3228b77c0 | 18.08.2022 12:56 | Link profiling libraries for coverage builds (#2499) | real 326.73 user 2779.18 sys 160.79 |
| 833281d1e | 26.08.2022 19:51 | Move comment | real 324.22 user 2783.54 sys 133.40 |
hyriseBenchmarkTPCH - single-threaded, SF 10.0
Sum of avg. item runtimes: +2% || Geometric mean of throughput changes: -2%
Configuration Overview - click to expand
+Configuration Overview----+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_3228b77c02306e4bd15c207f87717af8acd30522_st.json | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_833281d1ee76a14a730e02fff4b4a0784c44baf7_st.json |
+--------------------------+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 3228b77c02306e4bd15c207f87717af8acd30522-dirty | 833281d1ee76a14a730e02fff4b4a0784c44baf7-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| clustering | None | None |
| compiler | clang 14.0.0 | clang 14.0.0 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2022-08-26 23:40:10 | 2022-08-27 03:01:51 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 100 | 100 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+--------------------------+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
+----------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+----------+--------++----------+----------+--------+---------+
| TPC-H 01 || 5553.779 | 5662.471 | +2% || 0.18 | 0.18 | -2% | 0.4096 |
| TPC-H 02 || 43.543 | 43.078 | -1%˄ || 22.96 | 23.21 | +1%˄ | 0.3464 |
| TPC-H 03 || 2262.254 | 2308.342 | +2% || 0.44 | 0.43 | -2% | 0.0109 |
| TPC-H 04 || 1383.920 | 1402.205 | +1% || 0.72 | 0.71 | -1% | 0.1109 |
| TPC-H 05 || 2931.056 | 2975.042 | +2% || 0.34 | 0.34 | -1% | 0.0181 |
| TPC-H 06 || 148.762 | 153.896 | +3%˄ || 6.72 | 6.50 | -3%˄ | 0.0000 |
| TPC-H 07 || 890.076 | 900.219 | +1% || 1.12 | 1.11 | -1% | 0.0004 |
| TPC-H 08 || 678.754 | 686.261 | +1% || 1.47 | 1.46 | -1% | 0.0000 |
| TPC-H 09 || 5031.267 | 5030.910 | -0% || 0.20 | 0.20 | +0% | 0.9861 |
| TPC-H 10 || 2951.163 | 3004.597 | +2% || 0.34 | 0.33 | -2% | 0.0577 |
| TPC-H 11 || 73.123 | 72.560 | -1%˄ || 13.67 | 13.78 | +1%˄ | 0.1290 |
-| TPC-H 12 || 956.758 | 1005.633 | +5% || 1.05 | 0.99 | -5% | 0.0000 |
| TPC-H 13 || 4576.347 | 4572.521 | -0% || 0.22 | 0.22 | +0% | 0.8810 |
| TPC-H 14 || 423.234 | 426.587 | +1%˄ || 2.36 | 2.34 | -1%˄ | 0.0160 |
| TPC-H 15 || 184.733 | 183.054 | -1%˄ || 5.41 | 5.46 | +1%˄ | 0.0000 |
| TPC-H 16 || 646.219 | 664.707 | +3% || 1.55 | 1.50 | -3% | 0.0000 |
| TPC-H 17 || 205.478 | 207.017 | +1%˄ || 4.87 | 4.83 | -1%˄ | 0.0120 |
-| TPC-H 18 || 1421.950 | 1503.532 | +6% || 0.70 | 0.67 | -5% | 0.0000 |
| TPC-H 19 || 256.540 | 258.262 | +1%˄ || 3.90 | 3.87 | -1%˄ | 0.1171 |
| TPC-H 20 || 377.305 | 381.625 | +1%˄ || 2.65 | 2.62 | -1%˄ | 0.0147 |
| TPC-H 21 || 4453.258 | 4557.220 | +2% || 0.22 | 0.22 | -2% | 0.0002 |
-| TPC-H 22 || 407.053 | 451.824 | +11%˄ || 2.46 | 2.21 | -10%˄ | 0.0000 |
+----------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 35856.6 | 36451.6 | +2% || | | | |
| Geomean || | | || | | -2% | |
+----------++----------+----------+--------++----------+----------+--------+---------+
| Notes || ˄ Execution stopped due to max runs reached |
+----------++----------+----------+--------++----------+----------+--------+---------+
hyriseBenchmarkTPCH - single-threaded, SF 0.01
Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +2%
Configuration Overview - click to expand
+Configuration Overview----+------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_3228b77c02306e4bd15c207f87717af8acd30522_st_s01.json | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_833281d1ee76a14a730e02fff4b4a0784c44baf7_st_s01.json |
+--------------------------+------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 3228b77c02306e4bd15c207f87717af8acd30522-dirty | 833281d1ee76a14a730e02fff4b4a0784c44baf7-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| clustering | None | None |
| compiler | clang 14.0.0 | clang 14.0.0 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2022-08-26 23:59:36 | 2022-08-27 03:21:34 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 100 | 100 |
| scale_factor | 0.009999999776482582 | 0.009999999776482582 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+--------------------------+------------------------------------------------------------------------------------------------------------------+------------------------------------------------------------------------------------------------------------------+
+----------++----------+---------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+---------+--------++----------+----------+--------+---------+
| TPC-H 01 || 4.886 | 4.741 | -3%˄ || 204.58 | 210.86 | +3%˄ | 0.0000 |
| TPC-H 02 || 3.609 | 3.611 | +0%˄ || 276.96 | 276.82 | -0%˄ | 0.9931 |
| TPC-H 03 || 0.644 | 0.627 | -3%˄ || 1547.62 | 1592.02 | +3%˄ | 0.0075 |
| TPC-H 04 || 0.430 | 0.414 | -4%˄ || 2316.78 | 2407.14 | +4%˄ | 0.0000 |
| TPC-H 05 || 0.988 | 0.975 | -1%˄ || 1010.26 | 1024.13 | +1%˄ | 0.3208 |
| TPC-H 06 || 0.178 | 0.179 | +1%˄ || 5583.10 | 5554.34 | -1%˄ | 0.4420 |
| TPC-H 07 || 10.762 | 10.705 | -1%˄ || 92.90 | 93.39 | +1%˄ | 0.9344 |
| TPC-H 08 || 14.961 | 15.073 | +1%˄ || 66.83 | 66.34 | -1%˄ | 0.7768 |
| TPC-H 09 || 3.482 | 3.476 | -0%˄ || 287.00 | 287.57 | +0%˄ | 0.9897 |
| TPC-H 10 || 0.765 | 0.746 | -2%˄ || 1304.36 | 1338.11 | +3%˄ | 0.0000 |
+| TPC-H 11 || 0.209 | 0.197 | -6%˄ || 4739.51 | 5046.89 | +6%˄ | 0.0000 |
| TPC-H 12 || 0.602 | 0.587 | -2%˄ || 1656.77 | 1699.45 | +3%˄ | 0.0000 |
| TPC-H 13 || 1.978 | 1.956 | -1%˄ || 505.20 | 510.77 | +1%˄ | 0.0025 |
+| TPC-H 14 || 0.343 | 0.322 | -6%˄ || 2900.30 | 3093.42 | +7%˄ | 0.0000 |
| TPC-H 15 || 1.141 | 1.117 | -2%˄ || 873.37 | 892.41 | +2%˄ | 0.0000 |
| TPC-H 16 || 1.934 | 1.897 | -2%˄ || 516.38 | 526.70 | +2%˄ | 0.0000 |
-| TPC-H 17 || 0.618 | 0.655 | +6%˄ || 1612.11 | 1522.62 | -6%˄ | 0.6418 |
| TPC-H 18 || 1.109 | 1.121 | +1%˄ || 900.63 | 890.65 | -1%˄ | 0.0000 |
| TPC-H 19 || 4.828 | 4.712 | -2%˄ || 207.04 | 212.16 | +2%˄ | 0.0000 |
| TPC-H 20 || 2.071 | 2.035 | -2%˄ || 482.24 | 490.96 | +2%˄ | 0.4765 |
+| TPC-H 21 || 1.456 | 1.371 | -6%˄ || 685.00 | 728.44 | +6%˄ | 0.1770 |
| TPC-H 22 || 1.003 | 1.003 | +0%˄ || 995.06 | 995.15 | +0%˄ | 0.9177 |
+----------++----------+---------+--------++----------+----------+--------+---------+
| Sum || 58.0 | 57.5 | -1% || | | | |
| Geomean || | | || | | +2% | |
+----------++----------+---------+--------++----------+----------+--------+---------+
| Notes || ˄ Execution stopped due to max runs reached |
+----------++----------+---------+--------++----------+----------+--------+---------+
hyriseBenchmarkTPCH - multi-threaded, ordered, 1 client, 64 cores, SF 10.0
Sum of avg. item runtimes: -0% || Geometric mean of throughput changes: +0%
Configuration Overview - click to expand
+Configuration Overview---------+----------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_3228b77c02306e4bd15c207f87717af8acd30522_mt_ordered.json | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_833281d1ee76a14a730e02fff4b4a0784c44baf7_mt_ordered.json |
+-------------------------------+----------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 3228b77c02306e4bd15c207f87717af8acd30522-dirty | 833281d1ee76a14a730e02fff4b4a0784c44baf7-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| clustering | None | None |
| compiler | clang 14.0.0 | clang 14.0.0 |
| cores | 9 | 9 |
| data_preparation_cores | 0 | 0 |
| date | 2022-08-27 00:00:04 | 2022-08-27 03:22:02 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 60000000000 | 60000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [9] | [9] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+----------------------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------+
+----------++----------+----------+--------++----------+----------+--------+----------------------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
+| TPC-H 01 || 5036.547 | 4924.653 | -2% || 0.18 | 0.20 | +9% | (run time too short) |
| TPC-H 02 || 31.199 | 30.870 | -1% || 27.55 | 27.78 | +1% | 0.1802 |
| TPC-H 03 || 969.508 | 969.270 | -0% || 1.02 | 1.02 | +0% | 0.9796 |
| TPC-H 04 || 487.948 | 490.485 | +1% || 2.02 | 2.02 | +0% | 0.6249 |
| TPC-H 05 || 837.275 | 859.643 | +3% || 1.18 | 1.15 | -3% | 0.0221 |
| TPC-H 06 || 44.183 | 43.939 | -1% || 19.70 | 19.72 | +0% | 0.0371 |
| TPC-H 07 || 305.874 | 304.296 | -1% || 3.22 | 3.23 | +1% | 0.2455 |
| TPC-H 08 || 250.479 | 249.819 | -0% || 3.90 | 3.92 | +0% | 0.5217 |
| TPC-H 09 || 2224.010 | 2229.923 | +0% || 0.43 | 0.43 | -0% | (run time too short) |
| TPC-H 10 || 1717.113 | 1737.141 | +1% || 0.57 | 0.57 | -0% | (run time too short) |
| TPC-H 11 || 62.644 | 62.440 | -0% || 14.61 | 14.71 | +1% | 0.2175 |
| TPC-H 12 || 395.029 | 402.880 | +2% || 2.50 | 2.43 | -3% | 0.0000 |
| TPC-H 13 || 3132.648 | 3102.455 | -1% || 0.32 | 0.32 | +0% | (run time too short) |
| TPC-H 14 || 115.711 | 115.748 | +0% || 8.27 | 8.25 | -0% | 0.9113 |
| TPC-H 15 || 127.639 | 127.225 | -0% || 7.52 | 7.52 | -0% | 0.3531 |
| TPC-H 16 || 599.518 | 611.600 | +2% || 1.65 | 1.62 | -2% | 0.0000 |
| TPC-H 17 || 61.522 | 60.766 | -1% || 14.83 | 15.02 | +1% | 0.0000 |
| TPC-H 18 || 2187.966 | 2215.725 | +1% || 0.45 | 0.45 | -0% | 0.6179 |
| TPC-H 19 || 95.731 | 94.687 | -1% || 9.82 | 9.88 | +1% | 0.0151 |
| TPC-H 20 || 155.669 | 154.704 | -1% || 6.25 | 6.27 | +0% | 0.1068 |
| TPC-H 21 || 1012.951 | 1009.798 | -0% || 0.97 | 0.98 | +2% | (run time too short) |
-| TPC-H 22 || 128.121 | 136.626 | +7% || 7.55 | 7.07 | -6% | 0.0000 |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
| Sum || 19979.3 | 19934.7 | -0% || | | | |
| Geomean || | | || | | +0% | |
+----------++----------+----------+--------++----------+----------+--------+----------------------+
hyriseBenchmarkTPCH - multi-threaded, shuffled, 64 clients, 64 cores, SF 10.0
Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +1%
Configuration Overview - click to expand
+Configuration Overview---------+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_3228b77c02306e4bd15c207f87717af8acd30522_mt.json | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCH_833281d1ee76a14a730e02fff4b4a0784c44baf7_mt.json |
+-------------------------------+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 3228b77c02306e4bd15c207f87717af8acd30522-dirty | 833281d1ee76a14a730e02fff4b4a0784c44baf7-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 9 | 9 |
| clustering | None | None |
| compiler | clang 14.0.0 | clang 14.0.0 |
| cores | 9 | 9 |
| data_preparation_cores | 0 | 0 |
| date | 2022-08-27 00:24:07 | 2022-08-27 03:46:18 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| scale_factor | 10.0 | 10.0 |
| time_unit | ns | ns |
| use_prepared_statements | False | False |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [9] | [9] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
+----------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+----------++----------+----------+--------++----------+----------+--------+---------+
+| TPC-H 01 || 6237.808 | 5808.355 | -7% || 0.15 | 0.15 | +1% | 0.0665 |
-| TPC-H 02 || 244.395 | 391.673 | +60% || 0.15 | 0.15 | +1% | 0.3077 |
+| TPC-H 03 || 4115.757 | 3477.773 | -16% || 0.15 | 0.15 | +1% | 0.0993 |
+| TPC-H 04 || 2493.807 | 2298.300 | -8% || 0.15 | 0.15 | +1% | 0.4894 |
+| TPC-H 05 || 4314.641 | 4079.599 | -5% || 0.15 | 0.15 | +1% | 0.5399 |
+| TPC-H 06 || 913.367 | 765.786 | -16% || 0.15 | 0.15 | +1% | 0.5139 |
+| TPC-H 07 || 2570.019 | 2453.855 | -5% || 0.15 | 0.15 | +2% | 0.7027 |
-| TPC-H 08 || 1915.527 | 2094.724 | +9% || 0.15 | 0.15 | +1% | 0.5477 |
| TPC-H 09 || 6320.537 | 6243.775 | -1% || 0.15 | 0.15 | +1% | 0.8673 |
-| TPC-H 10 || 4263.006 | 4462.626 | +5% || 0.15 | 0.15 | +1% | 0.5422 |
+| TPC-H 11 || 506.481 | 314.442 | -38% || 0.15 | 0.15 | +1% | 0.2958 |
| TPC-H 12 || 2709.106 | 2718.571 | +0% || 0.15 | 0.15 | +1% | 0.9816 |
-| TPC-H 13 || 5567.597 | 5992.832 | +8% || 0.15 | 0.15 | +1% | 0.1828 |
-| TPC-H 14 || 1094.390 | 1195.341 | +9% || 0.15 | 0.15 | +1% | 0.6870 |
+| TPC-H 15 || 901.992 | 511.767 | -43% || 0.15 | 0.15 | +1% | 0.0794 |
-| TPC-H 16 || 1757.803 | 1881.267 | +7% || 0.15 | 0.15 | +2% | 0.6053 |
-| TPC-H 17 || 474.802 | 633.116 | +33% || 0.15 | 0.15 | +1% | 0.1856 |
-| TPC-H 18 || 3438.613 | 3614.108 | +5% || 0.15 | 0.15 | +1% | 0.5027 |
| TPC-H 19 || 798.340 | 795.220 | -0% || 0.15 | 0.15 | +1% | 0.9849 |
+| TPC-H 20 || 1988.167 | 1607.740 | -19% || 0.15 | 0.15 | +1% | 0.2886 |
| TPC-H 21 || 6097.908 | 6072.518 | -0% || 0.15 | 0.15 | +1% | 0.9576 |
-| TPC-H 22 || 844.370 | 1633.294 | +93% || 0.15 | 0.15 | +1% | 0.0085 |
+----------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 59568.4 | 59046.7 | -1% || | | | |
| Geomean || | | || | | +1% | |
+----------++----------+----------+--------++----------+----------+--------+---------+
hyriseBenchmarkTPCDS - single-threaded
Sum of avg. item runtimes: +0% || Geometric mean of throughput changes: -0%
Configuration Overview - click to expand
+Configuration Overview---+---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCDS_3228b77c02306e4bd15c207f87717af8acd30522_st.json | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCDS_833281d1ee76a14a730e02fff4b4a0784c44baf7_st.json |
+-------------------------+---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 3228b77c02306e4bd15c207f87717af8acd30522-dirty | 833281d1ee76a14a730e02fff4b4a0784c44baf7-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 14.0.0 | clang 14.0.0 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2022-08-27 00:46:09 | 2022-08-27 04:08:22 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 100 | 100 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------+---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------+
+---------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+---------+
| 01 || 263.544 | 264.319 | +0%˄ || 3.79 | 3.78 | -0%˄ | 0.4321 |
| 03 || 71.049 | 72.718 | +2%˄ || 14.07 | 13.75 | -2%˄ | 0.0673 |
| 06 || 154.916 | 157.862 | +2%˄ || 6.45 | 6.33 | -2%˄ | 0.0000 |
| 07 || 282.012 | 293.208 | +4%˄ || 3.55 | 3.41 | -4%˄ | 0.0000 |
| 09 || 630.494 | 626.255 | -1% || 1.59 | 1.60 | +1% | 0.4044 |
-| 10 || 121.368 | 127.638 | +5%˄ || 8.24 | 7.83 | -5%˄ | 0.0000 |
| 13 || 409.957 | 419.434 | +2%˄ || 2.44 | 2.38 | -2%˄ | 0.0000 |
| 15 || 105.995 | 107.157 | +1%˄ || 9.43 | 9.33 | -1%˄ | 0.0002 |
| 16 || 154.050 | 155.990 | +1%˄ || 6.49 | 6.41 | -1%˄ | 0.0001 |
| 17 || 299.023 | 304.104 | +2%˄ || 3.34 | 3.29 | -2%˄ | 0.0000 |
| 19 || 100.845 | 103.085 | +2%˄ || 9.92 | 9.70 | -2%˄ | 0.0000 |
| 25 || 160.739 | 164.438 | +2%˄ || 6.22 | 6.08 | -2%˄ | 0.0000 |
| 26 || 134.714 | 133.916 | -1%˄ || 7.42 | 7.47 | +1%˄ | 0.1348 |
| 28 || 612.526 | 600.289 | -2%˄ || 1.63 | 1.67 | +2%˄ | 0.0000 |
| 29 || 460.697 | 460.108 | -0%˄ || 2.17 | 2.17 | +0%˄ | 0.4662 |
| 31 || 1246.051 | 1224.768 | -2% || 0.80 | 0.82 | +2% | 0.0000 |
| 32 || 44.953 | 43.642 | -3%˄ || 22.24 | 22.91 | +3%˄ | 0.0000 |
| 34 || 155.892 | 152.751 | -2%˄ || 6.41 | 6.55 | +2%˄ | 0.0000 |
| 35 || 623.441 | 610.815 | -2% || 1.60 | 1.64 | +2% | 0.0000 |
| 37 || 263.225 | 257.831 | -2%˄ || 3.80 | 3.88 | +2%˄ | 0.0007 |
| 39a || 1701.587 | 1741.163 | +2% || 0.59 | 0.57 | -2% | 0.0000 |
| 39b || 1686.786 | 1707.492 | +1% || 0.59 | 0.59 | -1% | 0.0003 |
| 41 || 281.448 | 279.868 | -1%˄ || 3.55 | 3.57 | +1%˄ | 0.0000 |
| 42 || 81.680 | 79.565 | -3%˄ || 12.24 | 12.57 | +3%˄ | 0.0000 |
| 43 || 901.354 | 913.461 | +1% || 1.11 | 1.09 | -1% | 0.0000 |
| 45 || 115.626 | 116.971 | +1%˄ || 8.65 | 8.55 | -1%˄ | 0.0000 |
| 48 || 1010.705 | 1011.202 | +0% || 0.99 | 0.99 | -0% | 0.8621 |
| 50 || 112.297 | 111.097 | -1%˄ || 8.90 | 9.00 | +1%˄ | 0.0000 |
| 52 || 81.562 | 79.790 | -2%˄ || 12.26 | 12.53 | +2%˄ | 0.0000 |
| 55 || 77.741 | 76.075 | -2%˄ || 12.86 | 13.14 | +2%˄ | 0.0000 |
| 62 || 526.210 | 529.409 | +1%˄ || 1.90 | 1.89 | -1%˄ | 0.0000 |
| 65 || 1847.866 | 1853.362 | +0% || 0.54 | 0.54 | -0% | 0.4484 |
| 69 || 118.671 | 118.085 | -0%˄ || 8.43 | 8.47 | +0%˄ | 0.0251 |
| 73 || 77.901 | 77.549 | -0%˄ || 12.84 | 12.89 | +0%˄ | 0.0150 |
| 79 || 455.055 | 457.540 | +1%˄ || 2.20 | 2.19 | -1%˄ | 0.0002 |
| 81 || 206.283 | 208.246 | +1%˄ || 4.85 | 4.80 | -1%˄ | 0.0002 |
| 82 || 336.296 | 336.715 | +0%˄ || 2.97 | 2.97 | -0%˄ | 0.7934 |
| 83 || 49.451 | 49.556 | +0%˄ || 20.22 | 20.18 | -0%˄ | 0.8272 |
| 85 || 131.377 | 130.209 | -1%˄ || 7.61 | 7.68 | +1%˄ | 0.0000 |
| 88 || 627.804 | 612.256 | -2% || 1.59 | 1.63 | +3% | 0.0000 |
| 91 || 18.460 | 18.545 | +0%˄ || 54.16 | 53.91 | -0%˄ | 0.4844 |
| 92 || 46.789 | 45.830 | -2%˄ || 21.37 | 21.82 | +2%˄ | 0.0000 |
| 93 || 3705.601 | 3711.126 | +0% || 0.27 | 0.27 | -0% | 0.5387 |
| 94 || 98.158 | 97.924 | -0%˄ || 10.19 | 10.21 | +0%˄ | 0.4223 |
| 95 || 8719.147 | 8729.429 | +0% || 0.11 | 0.11 | -0% | ˅ |
| 96 || 57.952 | 58.925 | +2%˄ || 17.25 | 16.97 | -2%˄ | 0.0163 |
| 97 || 3207.606 | 3259.587 | +2% || 0.31 | 0.31 | -2% | 0.0004 |
| 99 || 998.818 | 1003.463 | +0% || 1.00 | 1.00 | -0% | 0.0032 |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 33575.7 | 33694.8 | +0% || | | | |
| Geomean || | | || | | -0% | |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Notes || ˄ Execution stopped due to max runs reached |
| || ˅ Insufficient number of runs for p-value calculation |
+---------++----------+----------+--------++----------+----------+--------+---------+
hyriseBenchmarkTPCDS - multi-threaded, shuffled, 64 clients, 64 cores
Sum of avg. item runtimes: +1% || Geometric mean of throughput changes: -1%
Configuration Overview - click to expand
+Configuration Overview---------+---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCDS_3228b77c02306e4bd15c207f87717af8acd30522_mt.json | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCDS_833281d1ee76a14a730e02fff4b4a0784c44baf7_mt.json |
+-------------------------------+---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 3228b77c02306e4bd15c207f87717af8acd30522-dirty | 833281d1ee76a14a730e02fff4b4a0784c44baf7-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 9 | 9 |
| compiler | clang 14.0.0 | clang 14.0.0 |
| cores | 9 | 9 |
| data_preparation_cores | 0 | 0 |
| date | 2022-08-27 01:12:03 | 2022-08-27 04:34:22 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [9] | [9] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+---------------------------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------------------+
+---------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+---------+
+| 01 || 575.881 | 541.951 | -6% || 0.16 | 0.16 | -1% | 0.7206 |
-| 03 || 218.858 | 361.974 | +65% || 0.16 | 0.16 | -1% | 0.1075 |
-| 06 || 594.250 | 718.879 | +21% || 0.16 | 0.16 | -1% | 0.4196 |
+| 07 || 862.375 | 741.476 | -14% || 0.16 | 0.16 | -1% | 0.3440 |
-| 09 || 1204.957 | 1453.772 | +21% || 0.16 | 0.16 | -1% | 0.0847 |
| 10 || 417.233 | 412.073 | -1% || 0.16 | 0.16 | -0% | 0.9593 |
| 13 || 1214.072 | 1174.546 | -3% || 0.16 | 0.16 | -1% | 0.8542 |
+| 15 || 421.517 | 361.741 | -14% || 0.16 | 0.16 | -1% | 0.5926 |
-| 16 || 556.870 | 619.112 | +11% || 0.16 | 0.16 | -1% | 0.6340 |
-| 17 || 798.699 | 931.354 | +17% || 0.16 | 0.16 | -1% | 0.4717 |
-| 19 || 438.213 | 486.006 | +11% || 0.16 | 0.16 | -1% | 0.6845 |
+| 25 || 688.298 | 579.069 | -16% || 0.16 | 0.16 | -1% | 0.4320 |
-| 26 || 461.701 | 558.491 | +21% || 0.16 | 0.16 | -1% | 0.4283 |
+| 28 || 1058.642 | 961.230 | -9% || 0.16 | 0.16 | -0% | 0.5169 |
| 29 || 1133.339 | 1170.164 | +3% || 0.16 | 0.16 | -1% | 0.8447 |
+| 31 || 2195.204 | 1954.095 | -11% || 0.16 | 0.16 | -1% | 0.3444 |
+| 32 || 245.538 | 163.088 | -34% || 0.16 | 0.16 | -1% | 0.3625 |
+| 34 || 558.667 | 523.666 | -6% || 0.16 | 0.16 | -1% | 0.8023 |
+| 35 || 1673.527 | 1567.418 | -6% || 0.16 | 0.16 | -0% | 0.5961 |
| 37 || 756.521 | 724.532 | -4% || 0.16 | 0.16 | -1% | 0.8421 |
+| 39a || 2226.810 | 2059.879 | -7% || 0.16 | 0.16 | -1% | 0.3909 |
-| 39b || 2078.109 | 2171.946 | +5% || 0.16 | 0.16 | -1% | 0.6071 |
+| 41 || 2502.733 | 2229.097 | -11% || 0.16 | 0.16 | -1% | 0.2978 |
-| 42 || 301.528 | 407.779 | +35% || 0.16 | 0.16 | -1% | 0.3462 |
+| 43 || 1803.192 | 1628.806 | -10% || 0.16 | 0.16 | -1% | 0.4521 |
| 45 || 484.110 | 464.138 | -4% || 0.16 | 0.16 | -1% | 0.8735 |
+| 48 || 2098.256 | 1936.250 | -8% || 0.16 | 0.16 | -1% | 0.4092 |
-| 50 || 408.130 | 596.839 | +46% || 0.16 | 0.16 | -1% | 0.1149 |
| 52 || 375.092 | 387.737 | +3% || 0.16 | 0.16 | -1% | 0.9032 |
+| 55 || 388.046 | 281.343 | -27% || 0.16 | 0.16 | -1% | 0.2172 |
+| 62 || 995.267 | 909.181 | -9% || 0.16 | 0.16 | -1% | 0.4232 |
| 65 || 3119.725 | 3240.148 | +4% || 0.16 | 0.16 | -1% | 0.2948 |
+| 69 || 619.610 | 576.018 | -7% || 0.16 | 0.16 | -1% | 0.7599 |
+| 73 || 413.743 | 259.185 | -37% || 0.16 | 0.16 | -1% | 0.1652 |
| 79 || 1349.720 | 1317.562 | -2% || 0.16 | 0.16 | -1% | 0.8885 |
-| 81 || 436.183 | 525.311 | +20% || 0.16 | 0.16 | -1% | 0.2783 |
+| 82 || 902.805 | 835.389 | -7% || 0.16 | 0.16 | -1% | 0.6449 |
-| 83 || 176.518 | 267.346 | +51% || 0.16 | 0.16 | -1% | 0.1375 |
| 85 || 665.710 | 647.102 | -3% || 0.16 | 0.16 | -1% | 0.9085 |
-| 88 || 1275.312 | 1657.163 | +30% || 0.16 | 0.16 | -1% | 0.1291 |
-| 91 || 92.754 | 101.738 | +10% || 0.16 | 0.16 | -1% | 0.8012 |
+| 92 || 231.231 | 213.128 | -8% || 0.16 | 0.16 | -1% | 0.7821 |
-| 93 || 2440.757 | 2803.122 | +15% || 0.16 | 0.16 | -1% | 0.1142 |
| 94 || 337.495 | 344.477 | +2% || 0.16 | 0.16 | -1% | 0.9296 |
| 95 || 8100.067 | 8123.325 | +0% || 0.16 | 0.16 | -1% | 0.9325 |
+| 96 || 300.059 | 277.061 | -8% || 0.16 | 0.16 | -1% | 0.7952 |
| 97 || 3874.138 | 3997.197 | +3% || 0.16 | 0.16 | -0% | 0.5520 |
-| 99 || 1446.683 | 1543.815 | +7% || 0.16 | 0.16 | -1% | 0.5515 |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 55518.1 | 55806.7 | +1% || | | | |
| Geomean || | | || | | -1% | |
+---------++----------+----------+--------++----------+----------+--------+---------+
hyriseBenchmarkTPCC - single-threaded
Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +2%
Configuration Overview - click to expand
+Configuration Overview---+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCC_3228b77c02306e4bd15c207f87717af8acd30522_st.json | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCC_833281d1ee76a14a730e02fff4b4a0784c44baf7_st.json |
+-------------------------+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 3228b77c02306e4bd15c207f87717af8acd30522-dirty | 833281d1ee76a14a730e02fff4b4a0784c44baf7-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 14.0.0 | clang 14.0.0 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2022-08-27 01:32:30 | 2022-08-27 04:54:53 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 60000000000 | 60000000000 |
| max_runs | -1 | -1 |
| scale_factor | 1 | 1 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Delivery || 22.865 | 22.839 | -0% || 4.27 | 4.33 | +2% | 0.7498 |
| New-Order || 16.417 | 16.110 | -2% || 47.93 | 48.66 | +2% | 0.0166 |
| Order-Status || 1.087 | 1.077 | -1% || 4.27 | 4.33 | +2% | 0.3297 |
| Payment || 2.056 | 2.050 | -0% || 45.76 | 46.49 | +2% | 0.3374 |
| Stock-Level || 3.512 | 3.527 | +0% || 4.25 | 4.33 | +2% | 0.6110 |
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Sum || 45.9 | 45.6 | -1% || | | | |
| Geomean || | | || | | +2% | |
+--------------++----------+---------+--------++----------+----------+--------+---------+
hyriseBenchmarkTPCC - multi-threaded, shuffled, 64 clients, 64 cores
Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +1%
Configuration Overview - click to expand
+Configuration Overview---------+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCC_3228b77c02306e4bd15c207f87717af8acd30522_mt.json | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkTPCC_833281d1ee76a14a730e02fff4b4a0784c44baf7_mt.json |
+-------------------------------+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 3228b77c02306e4bd15c207f87717af8acd30522-dirty | 833281d1ee76a14a730e02fff4b4a0784c44baf7-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 9 | 9 |
| compiler | clang 14.0.0 | clang 14.0.0 |
| cores | 9 | 9 |
| data_preparation_cores | 0 | 0 |
| date | 2022-08-27 01:33:31 | 2022-08-27 04:55:54 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| scale_factor | 1 | 1 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [9] | [9] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+--------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------+
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Delivery || 131.905 | 131.081 | -1% || 5.72 | 5.76 | +1% | 0.4564 |
| unsucc.: || 2.1 | 2.1 | -3% || 11.46 | 11.39 | -1% | |
| New-Order || 36.804 | 36.713 | -0% || 112.63 | 115.32 | +2% | 0.2126 |
| unsucc.: || 4.0 | 3.1 | -22% || 80.63 | 77.65 | -4% | |
| Order-Status || 4.976 | 4.990 | +0% || 17.18 | 17.15 | -0% | 0.6606 |
| Payment || 5.421 | 5.408 | -0% || 46.69 | 46.68 | -0% | 0.2934 |
| unsucc.: || 2.1 | 2.1 | -2% || 137.99 | 137.71 | -0% | |
| Stock-Level || 11.649 | 11.539 | -1% || 17.18 | 17.15 | -0% | 0.3207 |
+--------------++----------+---------+--------++----------+----------+--------+---------+
| Sum || 190.8 | 189.7 | -1% || | | | |
| Geomean || | | || | | +1% | |
+--------------++----------+---------+--------++----------+----------+--------+---------+
hyriseBenchmarkJoinOrder - single-threaded
Sum of avg. item runtimes: -1% || Geometric mean of throughput changes: +0%
Configuration Overview - click to expand
+Configuration Overview---+-------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkJoinOrder_3228b77c02306e4bd15c207f87717af8acd30522_st.json | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkJoinOrder_833281d1ee76a14a730e02fff4b4a0784c44baf7_st.json |
+-------------------------+-------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 3228b77c02306e4bd15c207f87717af8acd30522-dirty | 833281d1ee76a14a730e02fff4b4a0784c44baf7-dirty |
| benchmark_mode | Ordered | Ordered |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 1 | 1 |
| compiler | clang 14.0.0 | clang 14.0.0 |
| cores | 0 | 0 |
| data_preparation_cores | 0 | 0 |
| date | 2022-08-27 01:53:35 | 2022-08-27 05:15:57 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 60000000000 | 60000000000 |
| max_runs | 100 | 100 |
| time_unit | ns | ns |
| using_scheduler | False | False |
| verify | False | False |
| warmup_duration | 1000000000 | 1000000000 |
+-------------------------+-------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+
+---------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+---------+
| 10a || 162.844 | 162.115 | -0%˄ || 6.14 | 6.17 | +0%˄ | 0.1792 |
| 10b || 112.348 | 112.854 | +0%˄ || 8.90 | 8.86 | -0%˄ | 0.1655 |
| 10c || 324.471 | 319.476 | -2%˄ || 3.08 | 3.13 | +2%˄ | 0.0001 |
| 11a || 16.459 | 16.540 | +0%˄ || 60.75 | 60.45 | -0%˄ | 0.4220 |
| 11b || 16.228 | 16.197 | -0%˄ || 61.61 | 61.73 | +0%˄ | 0.3802 |
| 11c || 21.926 | 21.767 | -1%˄ || 45.60 | 45.93 | +1%˄ | 0.2183 |
| 11d || 24.551 | 24.741 | +1%˄ || 40.73 | 40.41 | -1%˄ | 0.2207 |
| 12a || 50.558 | 51.344 | +2%˄ || 19.78 | 19.48 | -2%˄ | 0.0000 |
-| 12b || 29.470 | 31.249 | +6%˄ || 33.93 | 32.00 | -6%˄ | 0.0000 |
-| 12c || 86.923 | 96.754 | +11%˄ || 11.50 | 10.33 | -10%˄ | 0.0000 |
| 13a || 276.066 | 277.295 | +0%˄ || 3.62 | 3.61 | -0%˄ | 0.0011 |
| 13b || 155.890 | 156.453 | +0%˄ || 6.41 | 6.39 | -0%˄ | 0.0052 |
| 13c || 119.915 | 119.855 | -0%˄ || 8.34 | 8.34 | +0%˄ | 0.8073 |
| 13d || 583.619 | 583.386 | -0%˄ || 1.71 | 1.71 | +0%˄ | 0.7967 |
| 14a || 412.883 | 413.148 | +0%˄ || 2.42 | 2.42 | -0%˄ | 0.7581 |
| 14b || 231.700 | 231.447 | -0%˄ || 4.32 | 4.32 | +0%˄ | 0.2329 |
| 14c || 486.796 | 487.174 | +0%˄ || 2.05 | 2.05 | -0%˄ | 0.5761 |
| 15a || 82.617 | 81.354 | -2%˄ || 12.10 | 12.29 | +2%˄ | 0.0000 |
| 15b || 80.784 | 80.623 | -0%˄ || 12.38 | 12.40 | +0%˄ | 0.2417 |
| 15c || 87.338 | 87.628 | +0%˄ || 11.45 | 11.41 | -0%˄ | 0.1226 |
| 15d || 81.942 | 81.688 | -0%˄ || 12.20 | 12.24 | +0%˄ | 0.0263 |
| 16a || 2150.177 | 2143.847 | -0% || 0.47 | 0.47 | +0% | 0.5330 |
| 16b || 3143.204 | 3139.094 | -0% || 0.32 | 0.32 | +0% | 0.6987 |
| 16c || 2235.935 | 2218.349 | -1% || 0.45 | 0.45 | +1% | 0.0108 |
| 16d || 2214.424 | 2194.807 | -1% || 0.45 | 0.46 | +1% | 0.0177 |
| 17a || 567.624 | 565.751 | -0%˄ || 1.76 | 1.77 | +0%˄ | 0.0001 |
| 17b || 444.702 | 443.505 | -0%˄ || 2.25 | 2.25 | +0%˄ | 0.0003 |
| 17c || 421.997 | 420.872 | -0%˄ || 2.37 | 2.38 | +0%˄ | 0.0033 |
| 17d || 498.749 | 496.196 | -1%˄ || 2.01 | 2.02 | +1%˄ | 0.0000 |
| 17e || 1541.790 | 1539.004 | -0% || 0.65 | 0.65 | +0% | 0.1872 |
| 17f || 915.571 | 913.871 | -0% || 1.09 | 1.09 | +0% | 0.0021 |
| 18a || 597.485 | 598.679 | +0%˄ || 1.67 | 1.67 | -0%˄ | 0.0925 |
| 18b || 109.902 | 110.002 | +0%˄ || 9.10 | 9.09 | -0%˄ | 0.5880 |
| 18c || 550.228 | 553.786 | +1%˄ || 1.82 | 1.81 | -1%˄ | 0.0000 |
| 19a || 176.928 | 176.761 | -0%˄ || 5.65 | 5.66 | +0%˄ | 0.3204 |
| 19b || 125.043 | 124.587 | -0%˄ || 8.00 | 8.03 | +0%˄ | 0.0009 |
| 19c || 189.242 | 188.843 | -0%˄ || 5.28 | 5.30 | +0%˄ | 0.0020 |
| 19d || 714.412 | 718.575 | +1% || 1.40 | 1.39 | -1% | 0.0000 |
| 1a || 10.745 | 10.811 | +1%˄ || 93.05 | 92.47 | -1%˄ | 0.6079 |
| 1b || 10.053 | 10.191 | +1%˄ || 99.45 | 98.10 | -1%˄ | 0.0000 |
| 1c || 11.060 | 11.113 | +0%˄ || 90.40 | 89.96 | -0%˄ | 0.0056 |
| 1d || 10.045 | 10.109 | +1%˄ || 99.53 | 98.89 | -1%˄ | 0.0001 |
| 20a || 389.939 | 378.484 | -3%˄ || 2.56 | 2.64 | +3%˄ | 0.0000 |
| 20b || 234.657 | 233.722 | -0%˄ || 4.26 | 4.28 | +0%˄ | 0.0004 |
| 20c || 227.729 | 227.125 | -0%˄ || 4.39 | 4.40 | +0%˄ | 0.0000 |
| 21a || 34.594 | 34.924 | +1%˄ || 28.91 | 28.63 | -1%˄ | 0.0297 |
| 21b || 20.448 | 20.711 | +1%˄ || 48.90 | 48.27 | -1%˄ | 0.0000 |
| 21c || 34.815 | 34.885 | +0%˄ || 28.72 | 28.66 | -0%˄ | 0.0164 |
| 22a || 197.440 | 196.694 | -0%˄ || 5.06 | 5.08 | +0%˄ | 0.0055 |
| 22b || 143.875 | 143.704 | -0%˄ || 6.95 | 6.96 | +0%˄ | 0.0696 |
| 22c || 486.946 | 486.134 | -0%˄ || 2.05 | 2.06 | +0%˄ | 0.0934 |
| 22d || 881.697 | 877.893 | -0% || 1.13 | 1.14 | +0% | 0.0040 |
| 23a || 43.574 | 43.954 | +1%˄ || 22.95 | 22.75 | -1%˄ | 0.0000 |
-| 23b || 48.805 | 51.009 | +5%˄ || 20.49 | 19.60 | -4%˄ | 0.0000 |
| 23c || 85.172 | 85.311 | +0%˄ || 11.74 | 11.72 | -0%˄ | 0.1356 |
| 24a || 139.761 | 139.949 | +0%˄ || 7.15 | 7.15 | -0%˄ | 0.0276 |
| 24b || 118.243 | 117.982 | -0%˄ || 8.46 | 8.48 | +0%˄ | 0.0363 |
| 25a || 272.974 | 272.244 | -0%˄ || 3.66 | 3.67 | +0%˄ | 0.0133 |
| 25b || 109.215 | 108.736 | -0%˄ || 9.16 | 9.20 | +0%˄ | 0.0000 |
| 25c || 922.589 | 922.983 | +0% || 1.08 | 1.08 | -0% | 0.7587 |
| 26a || 170.587 | 170.269 | -0%˄ || 5.86 | 5.87 | +0%˄ | 0.2165 |
| 26b || 132.636 | 132.765 | +0%˄ || 7.54 | 7.53 | -0%˄ | 0.3838 |
| 26c || 244.902 | 243.372 | -1%˄ || 4.08 | 4.11 | +1%˄ | 0.0000 |
| 27a || 24.532 | 24.918 | +2%˄ || 40.76 | 40.12 | -2%˄ | 0.0000 |
| 27b || 24.033 | 24.431 | +2%˄ || 41.60 | 40.92 | -2%˄ | 0.0000 |
| 27c || 34.829 | 34.899 | +0%˄ || 28.71 | 28.65 | -0%˄ | 0.4001 |
| 28a || 292.691 | 291.376 | -0%˄ || 3.42 | 3.43 | +0%˄ | 0.0000 |
| 28b || 39.748 | 39.964 | +1%˄ || 25.16 | 25.02 | -1%˄ | 0.0000 |
| 28c || 328.693 | 326.598 | -1%˄ || 3.04 | 3.06 | +1%˄ | 0.0000 |
| 29a || 101.639 | 101.502 | -0%˄ || 9.84 | 9.85 | +0%˄ | 0.0793 |
| 29b || 74.481 | 74.256 | -0%˄ || 13.43 | 13.47 | +0%˄ | 0.0433 |
| 29c || 144.994 | 144.773 | -0%˄ || 6.90 | 6.91 | +0%˄ | 0.0534 |
| 2a || 51.139 | 51.267 | +0%˄ || 19.55 | 19.50 | -0%˄ | 0.0933 |
| 2b || 42.112 | 42.025 | -0%˄ || 23.74 | 23.79 | +0%˄ | 0.4416 |
| 2c || 22.952 | 23.079 | +1%˄ || 43.56 | 43.32 | -1%˄ | 0.0002 |
| 2d || 72.611 | 72.632 | +0%˄ || 13.77 | 13.77 | -0%˄ | 0.7321 |
| 30a || 146.950 | 146.726 | -0%˄ || 6.80 | 6.82 | +0%˄ | 0.0036 |
| 30b || 130.362 | 130.159 | -0%˄ || 7.67 | 7.68 | +0%˄ | 0.1279 |
+| 30c || 423.823 | 371.822 | -12%˄ || 2.36 | 2.69 | +14%˄ | 0.0000 |
| 31a || 144.170 | 142.289 | -1%˄ || 6.94 | 7.03 | +1%˄ | 0.0000 |
| 31b || 121.487 | 120.130 | -1%˄ || 8.23 | 8.32 | +1%˄ | 0.0000 |
| 31c || 169.134 | 166.289 | -2%˄ || 5.91 | 6.01 | +2%˄ | 0.0000 |
| 32a || 15.709 | 15.664 | -0%˄ || 63.65 | 63.83 | +0%˄ | 0.2367 |
| 32b || 37.496 | 37.399 | -0%˄ || 26.67 | 26.74 | +0%˄ | 0.4331 |
| 33a || 23.240 | 22.933 | -1%˄ || 43.02 | 43.60 | +1%˄ | 0.0000 |
| 33b || 22.730 | 22.423 | -1%˄ || 43.99 | 44.59 | +1%˄ | 0.0000 |
| 33c || 26.346 | 26.030 | -1%˄ || 37.95 | 38.41 | +1%˄ | 0.0000 |
| 3a || 124.495 | 124.418 | -0%˄ || 8.03 | 8.04 | +0%˄ | 0.5335 |
| 3b || 14.259 | 14.382 | +1%˄ || 70.12 | 69.51 | -1%˄ | 0.0182 |
| 3c || 452.697 | 450.641 | -0%˄ || 2.21 | 2.22 | +0%˄ | 0.0146 |
| 4a || 114.896 | 116.568 | +1%˄ || 8.70 | 8.58 | -1%˄ | 0.0000 |
| 4b || 13.102 | 13.038 | -0%˄ || 76.31 | 76.68 | +0%˄ | 0.0012 |
| 4c || 127.258 | 127.142 | -0%˄ || 7.86 | 7.86 | +0%˄ | 0.6605 |
| 5a || 58.135 | 58.037 | -0%˄ || 17.20 | 17.23 | +0%˄ | 0.1484 |
| 5b || 137.763 | 137.553 | -0%˄ || 7.26 | 7.27 | +0%˄ | 0.4020 |
| 5c || 192.621 | 190.599 | -1%˄ || 5.19 | 5.25 | +1%˄ | 0.0000 |
| 6a || 100.499 | 99.314 | -1%˄ || 9.95 | 10.07 | +1%˄ | 0.0000 |
| 6b || 119.286 | 117.339 | -2%˄ || 8.38 | 8.52 | +2%˄ | 0.0000 |
| 6c || 96.009 | 94.480 | -2%˄ || 10.42 | 10.58 | +2%˄ | 0.0000 |
| 6d || 516.012 | 508.912 | -1%˄ || 1.94 | 1.96 | +1%˄ | 0.0000 |
| 6e || 100.656 | 99.179 | -1%˄ || 9.93 | 10.08 | +1%˄ | 0.0000 |
| 6f || 686.964 | 677.515 | -1% || 1.46 | 1.48 | +1% | 0.0000 |
| 7a || 53.160 | 53.229 | +0%˄ || 18.81 | 18.79 | -0%˄ | 0.1661 |
| 7b || 48.910 | 48.727 | -0%˄ || 20.44 | 20.52 | +0%˄ | 0.0821 |
| 7c || 572.559 | 560.593 | -2%˄ || 1.75 | 1.78 | +2%˄ | 0.0000 |
| 8a || 38.742 | 38.687 | -0%˄ || 25.81 | 25.85 | +0%˄ | 0.2063 |
| 8b || 36.072 | 36.187 | +0%˄ || 27.72 | 27.63 | -0%˄ | 0.2524 |
| 8c || 1828.956 | 1823.972 | -0% || 0.55 | 0.55 | +0% | 0.0345 |
| 8d || 304.303 | 304.172 | -0%˄ || 3.29 | 3.29 | +0%˄ | 0.5669 |
| 9a || 186.145 | 184.781 | -1%˄ || 5.37 | 5.41 | +1%˄ | 0.0000 |
| 9b || 108.687 | 108.283 | -0%˄ || 9.20 | 9.23 | +0%˄ | 0.0000 |
| 9c || 273.558 | 270.350 | -1%˄ || 3.66 | 3.70 | +1%˄ | 0.0000 |
| 9d || 481.210 | 475.811 | -1%˄ || 2.08 | 2.10 | +1%˄ | 0.0000 |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 34328.1 | 34152.2 | -1% || | | | |
| Geomean || | | || | | +0% | |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Notes || ˄ Execution stopped due to max runs reached |
+---------++----------+----------+--------++----------+----------+--------+---------+
hyriseBenchmarkJoinOrder - multi-threaded, shuffled, 64 clients, 64 cores
Sum of avg. item runtimes: +0% || Geometric mean of throughput changes: -0%
Configuration Overview - click to expand
+Configuration Overview---------+-------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| Parameter | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkJoinOrder_3228b77c02306e4bd15c207f87717af8acd30522_mt.json | /hyrise/rel_clang/benchmark_all_results/hyriseBenchmarkJoinOrder_833281d1ee76a14a730e02fff4b4a0784c44baf7_mt.json |
+-------------------------------+-------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+
| GIT-HASH | 3228b77c02306e4bd15c207f87717af8acd30522-dirty | 833281d1ee76a14a730e02fff4b4a0784c44baf7-dirty |
| benchmark_mode | Shuffled | Shuffled |
| build_type | release | release |
| chunk_size | 65535 | 65535 |
| clients | 9 | 9 |
| compiler | clang 14.0.0 | clang 14.0.0 |
| cores | 9 | 9 |
| data_preparation_cores | 0 | 0 |
| date | 2022-08-27 02:35:51 | 2022-08-27 05:58:02 |
| encoding | {'default': {'encoding': 'Dictionary'}} | {'default': {'encoding': 'Dictionary'}} |
| indexes | False | False |
| max_duration | 1200000000000 | 1200000000000 |
| max_runs | -1 | -1 |
| time_unit | ns | ns |
| using_scheduler | True | True |
| utilized_cores_per_numa_node | [9] | [9] |
| verify | False | False |
| warmup_duration | 0 | 0 |
+-------------------------------+-------------------------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------------------+
+---------++----------+----------+--------++----------+----------+--------+---------+
| Item || Latency (ms/iter) | Change || Throughput (iter/s) | Change | p-value |
| || old | new | || old | new | | |
+---------++----------+----------+--------++----------+----------+--------+---------+
+| 10a || 303.285 | 218.742 | -28% || 0.17 | 0.17 | -1% | 0.1528 |
-| 10b || 162.135 | 192.073 | +18% || 0.17 | 0.17 | +0% | 0.5448 |
-| 10c || 428.813 | 450.883 | +5% || 0.17 | 0.17 | -0% | 0.7233 |
| 11a || 60.821 | 59.886 | -2% || 0.17 | 0.17 | -1% | 0.9690 |
-| 11b || 54.987 | 59.700 | +9% || 0.17 | 0.17 | -0% | 0.8267 |
+| 11c || 78.877 | 67.867 | -14% || 0.17 | 0.17 | -0% | 0.6640 |
+| 11d || 70.373 | 65.665 | -7% || 0.17 | 0.17 | -0% | 0.7095 |
-| 12a || 135.788 | 151.558 | +12% || 0.17 | 0.17 | -0% | 0.6611 |
-| 12b || 60.436 | 101.715 | +68% || 0.17 | 0.17 | -1% | 0.2644 |
-| 12c || 331.863 | 385.921 | +16% || 0.17 | 0.17 | -0% | 0.4802 |
+| 13a || 506.213 | 481.183 | -5% || 0.17 | 0.17 | -0% | 0.6632 |
+| 13b || 266.077 | 209.867 | -21% || 0.17 | 0.17 | +0% | 0.2308 |
-| 13c || 208.093 | 233.104 | +12% || 0.17 | 0.17 | -0% | 0.5910 |
| 13d || 867.918 | 898.265 | +3% || 0.17 | 0.17 | -1% | 0.7164 |
| 14a || 731.276 | 741.873 | +1% || 0.17 | 0.17 | -0% | 0.8831 |
+| 14b || 487.857 | 461.100 | -5% || 0.17 | 0.17 | -0% | 0.7589 |
| 14c || 829.936 | 802.690 | -3% || 0.17 | 0.17 | -0% | 0.7256 |
-| 15a || 166.233 | 174.015 | +5% || 0.17 | 0.17 | -0% | 0.8414 |
+| 15b || 164.840 | 135.740 | -18% || 0.17 | 0.17 | +0% | 0.3986 |
+| 15c || 162.783 | 127.244 | -22% || 0.17 | 0.17 | -0% | 0.2104 |
-| 15d || 189.542 | 206.051 | +9% || 0.17 | 0.17 | -0% | 0.7928 |
| 16a || 2579.322 | 2598.471 | +1% || 0.17 | 0.17 | -0% | 0.8823 |
| 16b || 3783.275 | 3712.734 | -2% || 0.17 | 0.17 | -0% | 0.5519 |
| 16c || 2598.521 | 2601.443 | +0% || 0.17 | 0.17 | -0% | 0.9775 |
| 16d || 2607.449 | 2617.440 | +0% || 0.17 | 0.17 | -0% | 0.9335 |
| 17a || 878.778 | 875.102 | -0% || 0.17 | 0.17 | -0% | 0.9662 |
+| 17b || 717.176 | 678.579 | -5% || 0.17 | 0.17 | -0% | 0.6898 |
-| 17c || 611.539 | 719.588 | +18% || 0.16 | 0.17 | +0% | 0.2371 |
+| 17d || 759.674 | 694.596 | -9% || 0.17 | 0.17 | -0% | 0.4806 |
+| 17e || 1835.448 | 1650.622 | -10% || 0.17 | 0.17 | -0% | 0.0847 |
-| 17f || 1227.826 | 1343.096 | +9% || 0.17 | 0.17 | -1% | 0.2991 |
| 18a || 808.628 | 838.114 | +4% || 0.17 | 0.17 | +0% | 0.7477 |
+| 18b || 188.877 | 156.890 | -17% || 0.17 | 0.17 | -1% | 0.3626 |
-| 18c || 773.667 | 815.152 | +5% || 0.17 | 0.17 | -0% | 0.6200 |
| 19a || 543.244 | 541.295 | -0% || 0.17 | 0.17 | -0% | 0.9842 |
| 19b || 314.228 | 313.753 | -0% || 0.17 | 0.17 | +0% | 0.9931 |
+| 19c || 503.697 | 461.220 | -8% || 0.17 | 0.17 | -1% | 0.5898 |
| 19d || 1349.821 | 1358.925 | +1% || 0.17 | 0.17 | -1% | 0.9397 |
-| 1a || 36.367 | 38.590 | +6% || 0.17 | 0.17 | +0% | 0.8760 |
+| 1b || 67.926 | 47.436 | -30% || 0.17 | 0.17 | -0% | 0.5167 |
-| 1c || 25.088 | 43.549 | +74% || 0.17 | 0.17 | -0% | 0.3316 |
+| 1d || 33.407 | 25.693 | -23% || 0.17 | 0.17 | -1% | 0.5454 |
+| 20a || 779.205 | 702.301 | -10% || 0.16 | 0.17 | +0% | 0.4442 |
| 20b || 495.514 | 480.198 | -3% || 0.17 | 0.17 | -0% | 0.8382 |
| 20c || 457.274 | 453.741 | -1% || 0.17 | 0.17 | -1% | 0.9572 |
-| 21a || 95.941 | 134.046 | +40% || 0.17 | 0.17 | -0% | 0.3944 |
-| 21b || 72.695 | 112.354 | +55% || 0.17 | 0.17 | +0% | 0.2262 |
+| 21c || 100.295 | 80.284 | -20% || 0.17 | 0.17 | +0% | 0.5662 |
-| 22a || 472.165 | 547.671 | +16% || 0.17 | 0.17 | -1% | 0.3247 |
-| 22b || 261.125 | 339.064 | +30% || 0.17 | 0.17 | +0% | 0.1251 |
-| 22c || 803.895 | 913.241 | +14% || 0.17 | 0.17 | -1% | 0.2355 |
+| 22d || 1227.277 | 1155.061 | -6% || 0.17 | 0.17 | -0% | 0.4602 |
-| 23a || 104.153 | 134.472 | +29% || 0.17 | 0.17 | -0% | 0.2285 |
+| 23b || 127.357 | 111.311 | -13% || 0.17 | 0.17 | -0% | 0.6174 |
-| 23c || 151.136 | 200.566 | +33% || 0.17 | 0.17 | -1% | 0.2571 |
+| 24a || 264.924 | 242.330 | -9% || 0.17 | 0.17 | -1% | 0.6123 |
| 24b || 227.737 | 231.550 | +2% || 0.17 | 0.17 | -0% | 0.9423 |
-| 25a || 408.294 | 521.450 | +28% || 0.16 | 0.17 | +0% | 0.1244 |
+| 25b || 168.558 | 160.361 | -5% || 0.17 | 0.17 | +0% | 0.8100 |
+| 25c || 1060.067 | 928.101 | -12% || 0.17 | 0.17 | -0% | 0.1495 |
+| 26a || 311.368 | 275.671 | -11% || 0.17 | 0.17 | -1% | 0.4518 |
-| 26b || 244.535 | 266.228 | +9% || 0.17 | 0.17 | -0% | 0.7238 |
+| 26c || 504.560 | 473.673 | -6% || 0.17 | 0.17 | -0% | 0.6830 |
-| 27a || 76.430 | 113.109 | +48% || 0.17 | 0.17 | -0% | 0.3011 |
+| 27b || 113.078 | 89.687 | -21% || 0.17 | 0.17 | -0% | 0.3450 |
-| 27c || 86.484 | 112.501 | +30% || 0.17 | 0.17 | -0% | 0.4381 |
+| 28a || 616.784 | 485.559 | -21% || 0.17 | 0.17 | -0% | 0.0475 |
| 28b || 174.755 | 168.149 | -4% || 0.17 | 0.17 | -1% | 0.8954 |
-| 28c || 557.835 | 612.488 | +10% || 0.17 | 0.17 | -1% | 0.3664 |
-| 29a || 215.066 | 263.999 | +23% || 0.17 | 0.17 | -0% | 0.3797 |
+| 29b || 211.592 | 154.223 | -27% || 0.17 | 0.17 | -0% | 0.1923 |
-| 29c || 274.651 | 313.799 | +14% || 0.17 | 0.17 | -1% | 0.5443 |
+| 2a || 156.735 | 114.859 | -27% || 0.17 | 0.17 | -0% | 0.2458 |
-| 2b || 112.940 | 175.558 | +55% || 0.17 | 0.17 | -0% | 0.2019 |
+| 2c || 85.749 | 68.882 | -20% || 0.17 | 0.17 | +0% | 0.5102 |
+| 2d || 241.128 | 191.269 | -21% || 0.17 | 0.17 | -0% | 0.3740 |
+| 30a || 393.238 | 338.831 | -14% || 0.17 | 0.17 | +0% | 0.4052 |
| 30b || 223.953 | 222.310 | -1% || 0.17 | 0.17 | -1% | 0.9712 |
+| 30c || 750.160 | 632.230 | -16% || 0.17 | 0.17 | -1% | 0.1993 |
-| 31a || 316.079 | 351.322 | +11% || 0.17 | 0.17 | -0% | 0.6030 |
-| 31b || 206.109 | 281.571 | +37% || 0.17 | 0.17 | -1% | 0.1609 |
| 31c || 365.014 | 372.758 | +2% || 0.17 | 0.17 | -0% | 0.9162 |
-| 32a || 56.115 | 61.557 | +10% || 0.17 | 0.17 | -0% | 0.8345 |
+| 32b || 135.398 | 98.361 | -27% || 0.17 | 0.17 | -0% | 0.3170 |
+| 33a || 66.753 | 55.111 | -17% || 0.17 | 0.17 | -0% | 0.4875 |
+| 33b || 51.360 | 41.234 | -20% || 0.17 | 0.17 | -0% | 0.2520 |
+| 33c || 78.428 | 73.012 | -7% || 0.17 | 0.17 | -1% | 0.8133 |
-| 3a || 221.363 | 298.338 | +35% || 0.17 | 0.17 | -0% | 0.1456 |
-| 3b || 66.499 | 72.317 | +9% || 0.17 | 0.17 | -0% | 0.8479 |
+| 3c || 588.363 | 508.323 | -14% || 0.17 | 0.17 | -1% | 0.2292 |
-| 4a || 137.524 | 201.373 | +46% || 0.17 | 0.17 | -0% | 0.0539 |
+| 4b || 54.257 | 37.189 | -31% || 0.17 | 0.17 | -1% | 0.2170 |
+| 4c || 214.957 | 203.184 | -5% || 0.17 | 0.17 | -1% | 0.7956 |
-| 5a || 152.521 | 189.118 | +24% || 0.17 | 0.17 | -0% | 0.4419 |
+| 5b || 254.931 | 188.888 | -26% || 0.17 | 0.17 | -1% | 0.2524 |
-| 5c || 269.499 | 310.173 | +15% || 0.17 | 0.17 | -0% | 0.5143 |
+| 6a || 142.480 | 115.280 | -19% || 0.17 | 0.17 | -0% | 0.3475 |
-| 6b || 182.445 | 224.457 | +23% || 0.17 | 0.17 | -0% | 0.4118 |
| 6c || 159.381 | 165.145 | +4% || 0.17 | 0.17 | -0% | 0.9210 |
-| 6d || 652.971 | 758.287 | +16% || 0.17 | 0.17 | -0% | 0.2478 |
-| 6e || 89.260 | 186.856 | +109% || 0.17 | 0.17 | -1% | 0.0145 |
+| 6f || 1115.381 | 1047.399 | -6% || 0.17 | 0.17 | -1% | 0.4271 |
| 7a || 125.859 | 126.817 | +1% || 0.17 | 0.17 | -0% | 0.9794 |
-| 7b || 94.273 | 107.722 | +14% || 0.17 | 0.17 | -1% | 0.5520 |
| 7c || 1027.133 | 1061.469 | +3% || 0.17 | 0.17 | -0% | 0.7572 |
+| 8a || 113.591 | 104.729 | -8% || 0.17 | 0.17 | -0% | 0.7562 |
| 8b || 92.627 | 91.615 | -1% || 0.17 | 0.17 | +0% | 0.9557 |
| 8c || 2253.783 | 2208.014 | -2% || 0.17 | 0.17 | -0% | 0.6621 |
-| 8d || 538.596 | 629.594 | +17% || 0.17 | 0.17 | -0% | 0.2898 |
+| 9a || 521.780 | 446.942 | -14% || 0.17 | 0.17 | -0% | 0.3458 |
| 9b || 315.273 | 328.315 | +4% || 0.17 | 0.17 | -0% | 0.8514 |
-| 9c || 679.283 | 710.097 | +5% || 0.17 | 0.17 | -0% | 0.7274 |
| 9d || 1033.543 | 1028.837 | -0% || 0.17 | 0.17 | -0% | 0.9682 |
+---------++----------+----------+--------++----------+----------+--------+---------+
| Sum || 53513.6 | 53564.0 | +0% || | | | |
| Geomean || | | || | | -0% | |
+---------++----------+----------+--------++----------+----------+--------+---------+
