noisepage icon indicating copy to clipboard operation
noisepage copied to clipboard

Add Sqlancer Test Type and Installation Requirements

Open dniu16 opened this issue 3 years ago • 2 comments

This PR adds the sqlancer test type following the instruction here: https://github.com/cmu-db/noisepage/tree/master/script/testing. The test can be run using the command "PYTHONPATH=.. python3 -m script.testing.sqlancer" under the noisepage directory.

In addition, maven is installed in installation/package.sh since it is needed to build sqlancer.

dniu16 avatar Feb 26 '21 02:02 dniu16

Major Decrease in Performance

STOP: this PR has a major negative performance impact

tps (%change) benchmark_type wal_device details
-2.05% tpcc RAM disk
Detailsmaster tps=8894.4, commit tps=8712.46, query_mode=extended, benchmark_type=tpcc, scale_factor=32.0000, terminals=32, client_time=60, weights={'Payment': 43, 'Delivery': 4, 'NewOrder': 45, 'StockLevel': 4, 'OrderStatus': 4}, wal_device=RAM disk, max_connection_threads=32
-1.06% tpcc None
Detailsmaster tps=9997.24, commit tps=9891.11, query_mode=extended, benchmark_type=tpcc, scale_factor=32.0000, terminals=32, client_time=60, weights={'Payment': 43, 'Delivery': 4, 'NewOrder': 45, 'StockLevel': 4, 'OrderStatus': 4}, wal_device=None, max_connection_threads=32
-2.83% tpcc HDD
Detailsmaster tps=491.73, commit tps=477.83, query_mode=extended, benchmark_type=tpcc, scale_factor=32.0000, terminals=32, client_time=60, weights={'Payment': 43, 'Delivery': 4, 'NewOrder': 45, 'StockLevel': 4, 'OrderStatus': 4}, wal_device=HDD, max_connection_threads=32
2.33% tatp RAM disk
Detailsmaster tps=3462.28, commit tps=3543.02, query_mode=extended, benchmark_type=tatp, scale_factor=1.0000, terminals=16, client_time=60, weights={'GetAccessData': 35, 'UpdateLocation': 14, 'GetNewDestination': 10, 'GetSubscriberData': 35, 'DeleteCallForwarding': 2, 'InsertCallForwarding': 2, 'UpdateSubscriberData': 2}, wal_device=RAM disk, max_connection_threads=32
-3.25% tatp None
Detailsmaster tps=3799.59, commit tps=3676.15, query_mode=extended, benchmark_type=tatp, scale_factor=1.0000, terminals=16, client_time=60, weights={'GetAccessData': 35, 'UpdateLocation': 14, 'GetNewDestination': 10, 'GetSubscriberData': 35, 'DeleteCallForwarding': 2, 'InsertCallForwarding': 2, 'UpdateSubscriberData': 2}, wal_device=None, max_connection_threads=32
-7.3% tatp HDD
Detailsmaster tps=351.98, commit tps=326.28, query_mode=extended, benchmark_type=tatp, scale_factor=1.0000, terminals=16, client_time=60, weights={'GetAccessData': 35, 'UpdateLocation': 14, 'GetNewDestination': 10, 'GetSubscriberData': 35, 'DeleteCallForwarding': 2, 'InsertCallForwarding': 2, 'UpdateSubscriberData': 2}, wal_device=HDD, max_connection_threads=32

noisepage-checks[bot] avatar Mar 13 '21 01:03 noisepage-checks[bot]

Codecov Report

Merging #1500 (7309ef8) into master (68c0069) will decrease coverage by 0.01%. The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1500      +/-   ##
==========================================
- Coverage   81.79%   81.78%   -0.02%     
==========================================
  Files         706      706              
  Lines       50425    50425              
==========================================
- Hits        41246    41239       -7     
- Misses       9179     9186       +7     
Impacted Files Coverage Δ
src/storage/arrow_serializer.cpp 80.00% <0.00%> (-6.25%) :arrow_down:
src/include/metrics/metrics_store.h 61.66% <0.00%> (-1.67%) :arrow_down:
src/include/storage/index/bplustree.h 89.49% <0.00%> (-0.34%) :arrow_down:
src/storage/data_table.cpp 98.29% <0.00%> (+1.13%) :arrow_up:
src/include/storage/sql_table.h 100.00% <0.00%> (+2.32%) :arrow_up:
src/transaction/transaction_manager.cpp 95.23% <0.00%> (+2.72%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 68c0069...7309ef8. Read the comment docs.

codecov[bot] avatar Mar 13 '21 01:03 codecov[bot]