redis-benchmarks-specification
redis-benchmarks-specification copied to clipboard
redis
[BUG]: Failures in benchmarking tests that use dataset configured in dbconfig
Describe the bug
I am executing benchmarking tests on the Linux/AMD64 AWS instance.
I am facing issues in all the tests that have dataset configured in dbconfig. For example, consider the test “memtier_benchmark-1key-geo-60M-elements-geohash-pipeline-10.yml”. I ran the test with dataset, following the command below:
redis-benchmarks-spec-client-runner --db_server_host localhost --db_server_port 6379 --client_aggregated_results_folder ./test --flushall_on_every_test_start --run-tests-with-dataset –test memtier_benchmark-1key-geo-60M-elements-geohash-pipeline-10.yml
The failure logs are as below:
2023-06-27 10:43:03 ERROR The total numbers of keys in setup does not match the expected spec: 1!=0. Aborting...
2023-06-27 10:43:03 CRITICAL Some unexpected exception was caught during local work. Failing test....
I found a similar issue here: < https://github.com/redis/redis-benchmarks-specification/issues/206 >, and the solution was this PR: < https://github.com/redis/redis-benchmarks-specification/pull/213/files >.
Following the above PR, I edited the “keyspacelen” from 1 to 0, and the test passes successfully.
Similarly, there are 8 more tests in the suite, that use dataset. Below is the list:
memtier_benchmark-1key-geo-60M-elements-geohash.yml memtier_benchmark-1key-geo-60M-elements-geopos-pipeline-10.yml memtier_benchmark-1key-geo-60M-elements-geopos.yml memtier_benchmark-1key-geo-60M-elements-geosearch-fromlonlat-bybox.yml memtier_benchmark-1key-geo-60M-elements-geodist-pipeline-10.yml
memtier_benchmark-1key-geo-60M-elements-geosearch-fromlonlat-pipeline-10.yml memtier_benchmark-1key-geo-60M-elements-geodist.yml
memtier_benchmark-1key-geo-60M-elements-geosearch-fromlonlat.yml
All the above-mentioned 9 tests use dataset and passes successfully after editing “keyspacelen” from 1 to 0.
Do you think it is reasonable to edit the “keyspacelen” to 0? If yes, I will raise a PR for the same.
Kindly let me know your thoughts on the above.
Environment (please complete the following information):
- OS: Ubuntu 22.04
- PyPi version package pip 23.1.2 from /usr/local/lib/python3.10/dist-packages/pip (python 3.10)
- Platfrom x86_64
