rocksdb icon indicating copy to clipboard operation
rocksdb copied to clipboard

rocksdb 8.9.1 unit test failed

Open adv0518 opened this issue 1 year ago • 5 comments

hello.

system: centos-8-x64, it's docker image, in cetnos7.4. gcc version: 8.3.1 20191121(red hat 8.3.1-5)

  1. db/compaction/compaction_job_stats_test.cc:456 Failure actual:63586 vs 63196.8
  2. db/compaction/compaction_job_stats_test.cc:456 Failure actual:63586 vs 63196.8
  3. db/db_wal_test.cc:2469 Failure actual:18427904 vs 18427904
  4. cache/tiered_secondary_cache_test.cc:524 which is 0 3u which is 3
  5. cache/tiered_secondary_cache_test.cc:382 which is 0 3u which is 3
  6. cache/tiered_secondary_cache_test.cc:267 which is 0 3u which is 1
  7. cache/tiered_secondary_cache_test.cc:623 which is 0 3u which is 8

thanks.

adv0518 avatar Jan 06 '24 02:01 adv0518

system: centos-8-x64, it's docker image, in cetnos7.4.

I am a bit confused about which of the above versions is the system version or the Docker version? Are you running the tests inside Docker? If so, what is the exact Docker image that is used please?

adamretter avatar Jan 10 '24 09:01 adamretter

do command: make check failed info:

Makefile:185: Warning: Compiling in debug mode. Don't use the resulting binary in production Generating parallel test scripts for blob_counting_iterator_test Generating parallel test scripts for blob_file_builder_test Generating parallel test scripts for blob_file_addition_test Generating parallel test scripts for cache_reservation_manager_test Generating parallel test scripts for cache_test Generating parallel test scripts for tiered_secondary_cache_test Generating parallel test scripts for lru_cache_test Generating parallel test scripts for compressed_secondary_cache_test Generating parallel test scripts for blob_file_cache_test Generating parallel test scripts for blob_file_garbage_test Generating parallel test scripts for blob_file_reader_test Generating parallel test scripts for blob_garbage_meter_test Generating parallel test scripts for blob_source_test Generating parallel test scripts for db_blob_basic_test Generating parallel test scripts for db_blob_compaction_test Generating parallel test scripts for db_blob_corruption_test Generating parallel test scripts for db_blob_index_test Generating parallel test scripts for column_family_test Generating parallel test scripts for compact_files_test Generating parallel test scripts for clipping_iterator_test Generating parallel test scripts for compaction_iterator_test Generating parallel test scripts for compaction_job_test Generating parallel test scripts for compaction_job_stats_test Generating parallel test scripts for compaction_picker_test Generating parallel test scripts for compaction_service_test Generating parallel test scripts for tiered_compaction_test Generating parallel test scripts for comparator_db_test Generating parallel test scripts for corruption_test Generating parallel test scripts for cuckoo_table_db_test Generating parallel test scripts for db_basic_test Generating parallel test scripts for db_compaction_filter_test Generating parallel test scripts for db_block_cache_test Generating parallel test scripts for db_compaction_test Generating parallel test scripts for db_clip_test Generating parallel test scripts for db_dynamic_level_test Generating parallel test scripts for db_encryption_test Generating parallel test scripts for db_flush_test Generating parallel test scripts for db_readonly_with_timestamp_test Generating parallel test scripts for import_column_family_test Generating parallel test scripts for db_with_timestamp_basic_test Generating parallel test scripts for db_inplace_update_test Generating parallel test scripts for db_io_failure_test Generating parallel test scripts for db_iter_test Generating parallel test scripts for db_iter_stress_test Generating parallel test scripts for db_iterator_test Generating parallel test scripts for db_log_iter_test Generating parallel test scripts for db_kv_checksum_test Generating parallel test scripts for db_memtable_test Generating parallel test scripts for db_merge_operator_test Generating parallel test scripts for db_merge_operand_test Generating parallel test scripts for db_options_test Generating parallel test scripts for db_properties_test Generating parallel test scripts for db_range_del_test Generating parallel test scripts for db_rate_limiter_test Generating parallel test scripts for db_secondary_test Generating parallel test scripts for db_sst_test Generating parallel test scripts for db_statistics_test Generating parallel test scripts for db_table_properties_test Generating parallel test scripts for db_tailing_iter_test Generating parallel test scripts for db_test Generating parallel test scripts for db_test2 Generating parallel test scripts for db_logical_block_size_cache_test Generating parallel test scripts for db_universal_compaction_test Generating parallel test scripts for db_with_timestamp_compaction_test Generating parallel test scripts for db_wal_test Generating parallel test scripts for db_write_buffer_manager_test Generating parallel test scripts for db_write_test Generating parallel test scripts for dbformat_test Generating parallel test scripts for error_handler_fs_test Generating parallel test scripts for external_sst_file_basic_test Generating parallel test scripts for external_sst_file_test Generating parallel test scripts for fault_injection_test Generating parallel test scripts for file_indexer_test Generating parallel test scripts for filename_test Generating parallel test scripts for flush_job_test Generating parallel test scripts for listener_test Generating parallel test scripts for manual_compaction_test Generating parallel test scripts for log_test Generating parallel test scripts for memtable_list_test Generating parallel test scripts for merge_helper_test Generating parallel test scripts for merge_test Generating parallel test scripts for obsolete_files_test Generating parallel test scripts for options_file_test Generating parallel test scripts for perf_context_test Generating parallel test scripts for periodic_task_scheduler_test Generating parallel test scripts for plain_table_db_test Please install gflags to run this test... Skipping... Generating parallel test scripts for prefix_test Generating parallel test scripts for repair_test Generating parallel test scripts for range_del_aggregator_test Generating parallel test scripts for range_tombstone_fragmenter_test Generating parallel test scripts for seqno_time_test Generating parallel test scripts for table_properties_collector_test Generating parallel test scripts for version_builder_test Generating parallel test scripts for version_edit_test Generating parallel test scripts for version_set_test Generating parallel test scripts for wal_manager_test Generating parallel test scripts for db_wide_basic_test Generating parallel test scripts for wide_column_serialization_test Generating parallel test scripts for wide_columns_helper_test Generating parallel test scripts for write_batch_test Generating parallel test scripts for write_controller_test Generating parallel test scripts for write_callback_test Generating parallel test scripts for env_basic_test Generating parallel test scripts for io_posix_test Generating parallel test scripts for mock_env_test Generating parallel test scripts for delete_scheduler_test Generating parallel test scripts for prefetch_test Generating parallel test scripts for random_access_file_reader_test Generating parallel test scripts for auto_roll_logger_test Generating parallel test scripts for env_logger_test Generating parallel test scripts for event_logger_test Generating parallel test scripts for arena_test Generating parallel test scripts for memory_allocator_test Generating parallel test scripts for skiplist_test Generating parallel test scripts for inlineskiplist_test Generating parallel test scripts for write_buffer_manager_test Generating parallel test scripts for histogram_test Generating parallel test scripts for iostats_context_test Generating parallel test scripts for statistics_test Generating parallel test scripts for stats_history_test Generating parallel test scripts for configurable_test Generating parallel test scripts for options_settable_test Generating parallel test scripts for customizable_test Generating parallel test scripts for options_test Generating parallel test scripts for block_test Generating parallel test scripts for block_based_table_reader_test Generating parallel test scripts for data_block_hash_index_test Generating parallel test scripts for full_filter_block_test Generating parallel test scripts for partitioned_filter_block_test Generating parallel test scripts for cleanable_test Generating parallel test scripts for cuckoo_table_builder_test Please install gflags to run this test... Skipping... Generating parallel test scripts for cuckoo_table_reader_test Generating parallel test scripts for merger_test Generating parallel test scripts for sst_file_reader_test Generating parallel test scripts for block_fetcher_test Generating parallel test scripts for table_test Generating parallel test scripts for testutil_test Please install gflags to run block_cache_trace_analyzer_test Generating parallel test scripts for block_cache_trace_analyzer_test Please install gflags to run io_tracer_parser_test Generating parallel test scripts for io_tracer_parser_test Generating parallel test scripts for ldb_cmd_test Generating parallel test scripts for reduce_levels_test Generating parallel test scripts for sst_dump_test Please install gflags to run trace_analyzer test Generating parallel test scripts for trace_analyzer_test Generating parallel test scripts for block_cache_tracer_test Generating parallel test scripts for io_tracer_test Generating parallel test scripts for autovector_test Please install gflags to run this test... Skipping... Generating parallel test scripts for bloom_test Generating parallel test scripts for coding_test Generating parallel test scripts for crc32c_test Generating parallel test scripts for defer_test Please install gflags to run this test... Skipping... Generating parallel test scripts for dynamic_bloom_test Generating parallel test scripts for filelock_test Generating parallel test scripts for file_reader_writer_test NPHash64 id: 251012e7 Generating parallel test scripts for hash_test Generating parallel test scripts for heap_test Generating parallel test scripts for random_test Generating parallel test scripts for rate_limiter_test Generating parallel test scripts for repeatable_thread_test Generating parallel test scripts for ribbon_test Generating parallel test scripts for slice_test Generating parallel test scripts for slice_transform_test Generating parallel test scripts for timer_queue_test Generating parallel test scripts for timer_test Generating parallel test scripts for thread_list_test Generating parallel test scripts for thread_local_test Generating parallel test scripts for udt_util_test Generating parallel test scripts for work_queue_test Generating parallel test scripts for agg_merge_test Generating parallel test scripts for blob_db_test Generating parallel test scripts for backup_engine_test Generating parallel test scripts for cassandra_format_test Generating parallel test scripts for cassandra_functional_test Generating parallel test scripts for cassandra_row_merge_test Generating parallel test scripts for cassandra_serialize_test Generating parallel test scripts for checkpoint_test Generating parallel test scripts for memory_test Generating parallel test scripts for env_timed_test Generating parallel test scripts for stringappend_test Generating parallel test scripts for object_registry_test Generating parallel test scripts for option_change_migration_test Generating parallel test scripts for options_util_test Generating parallel test scripts for hash_table_test Generating parallel test scripts for persistent_cache_test Generating parallel test scripts for cache_simulator_test Generating parallel test scripts for sim_cache_test Generating parallel test scripts for compact_on_deletion_collector_test Generating parallel test scripts for optimistic_transaction_test Generating parallel test scripts for range_locking_test Generating parallel test scripts for point_lock_manager_test Generating parallel test scripts for transaction_test Generating parallel test scripts for write_prepared_transaction_test Generating parallel test scripts for write_unprepared_transaction_test Generating parallel test scripts for write_committed_transaction_ts_test Generating parallel test scripts for timestamped_snapshot_test Generating parallel test scripts for ttl_test Generating parallel test scripts for util_merge_operators_test Generating parallel test scripts for write_batch_with_index_test make[2]: Leaving directory '/root/rocksdb-8.9.1' make[1]: Leaving directory '/root/rocksdb-8.9.1' GEN check make[1]: Entering directory '/root/rocksdb-8.9.1' $DEBUG_LEVEL is 1, $LIB_MODE is shared Makefile:185: Warning: Compiling in debug mode. Don't use the resulting binary in production

To monitor subtest <duration,pass/fail,name>, run "make watch-log" in a separate window

Computers / CPU cores / Max jobs to run 1:local / 32 / 32

Computer:jobs running/jobs completed/%of started jobs/Average seconds to complete ETA: 133s Left: 10881 AVG: 0.01s local:32/4322/100%/0.0s Note: Google Test filter = CompactionJobStatsTest/CompactionJobStatsTest.CompactionJobStatsTest/0 [==========] Running 1 test from 1 test case. [----------] Global test environment set-up. [----------] 1 test from CompactionJobStatsTest/CompactionJobStatsTest [ RUN ] CompactionJobStatsTest/CompactionJobStatsTest.CompactionJobStatsTest/0 db/compaction/compaction_job_stats_test.cc:456: Failure Expected: (current_stats.total_input_bytes) <= (stats.total_input_bytes * (1.00 + kFileSizeBias)), actual: 63586 vs 63196.8 terminate called after throwing an instance of 'testing::internal::GoogleTestFailureException' what(): db/compaction/compaction_job_stats_test.cc:456: Failure Expected: (current_stats.total_input_bytes) <= (stats.total_input_bytes * (1.00 + kFileSizeBias)), actual: 63586 vs 63196.8 Received signal 6 (Aborted) Invoking GDB for stack trace... GDB failed; falling back on backtrace+addr2line... #0 /lib64/libc.so.6(gsignal+0x10f) [0x7f9fec5b270f] ?? ??:0 #1 /lib64/libc.so.6(abort+0x127) [0x7f9fec59cb25] ?? ??:0 #2 /lib64/libstdc++.so.6(+0x9006b) [0x7f9fecf6706b] ?? ??:0 #3 /lib64/libstdc++.so.6(+0x9650c) [0x7f9fecf6d50c] ?? ??:0 #4 /lib64/libstdc++.so.6(+0x96567) [0x7f9fecf6d567] ?? ??:0 #5 /lib64/libstdc++.so.6(+0x967c8) [0x7f9fecf6d7c8] ?? ??:0 #6 /root/rocksdb-8.9.1/librocksdb_test_debug.so(+0x5e40a) [0x7f9feece540a] ?? ??:0 #7 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZNK7testing8internal12AssertHelperaSERKNS_7MessageE+0x6b) [0x7f9feed4cd6b] ?? ??:0 #8 ./compaction_job_stats_test() [0x41ebb3] rocksdb::CompactionJobStatsChecker::Verify(rocksdb::CompactionJobStats const&, rocksdb::CompactionJobStats const&) /root/rocksdb-8.9.1/db/compaction/compaction_job_stats_test.cc:455 (discriminator 7) #9 ./compaction_job_stats_test() [0x41c9fd] rocksdb::CompactionJobStatsChecker::OnCompactionCompleted(rocksdb::DB*, rocksdb::CompactionJobInfo const&) /usr/include/c++/8/bits/stl_deque.h:1592 #10 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb6DBImpl27NotifyOnCompactionCompletedEPNS_16ColumnFamilyDataEPNS_10CompactionERKNS_6StatusERKNS_18CompactionJobStatsEi+0x240) [0x7f9fee3a4a00] ?? ??:0 #11 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb6DBImpl20BackgroundCompactionEPbPNS_10JobContextEPNS_9LogBufferEPNS0_19PrepickedCompactionENS_3Env8PriorityE+0x10ad) [0x7f9fee3ac72d] ?? ??:0 #12 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb6DBImpl24BackgroundCallCompactionEPNS0_19PrepickedCompactionENS_3Env8PriorityE+0x126) [0x7f9fee3ae546] ?? ??:0 #13 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb6DBImpl16BGWorkCompactionEPv+0x78) [0x7f9fee3aecf8] ?? ??:0 #14 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb14ThreadPoolImpl4Impl8BGThreadEm+0x39d) [0x7f9fee71a97d] ?? ??:0 #15 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb14ThreadPoolImpl4Impl15BGThreadWrapperEPv+0x5e) [0x7f9fee71acfe] ?? ??:0 #16 /lib64/libstdc++.so.6(+0xc2b73) [0x7f9fecf99b73] ?? ??:0 #17 /lib64/libpthread.so.0(+0x82de) [0x7f9fedb472de] ?? ??:0 #18 /lib64/libc.so.6(clone+0x43) [0x7f9fec676e83] ?? ??:0 t/run-compaction_job_stats_test-CompactionJobStatsTest-CompactionJobStatsTest.CompactionJobStatsTest-0: line 4: 1733842 Aborted (core dumped) TEST_TMPDIR=$d ./compaction_job_stats_test --gtest_filter=CompactionJobStatsTest/CompactionJobStatsTest.CompactionJobStatsTest/0 Note: Google Test filter = CompactionJobStatsTest/CompactionJobStatsTest.CompactionJobStatsTest/1 [==========] Running 1 test from 1 test case. [----------] Global test environment set-up. [----------] 1 test from CompactionJobStatsTest/CompactionJobStatsTest [ RUN ] CompactionJobStatsTest/CompactionJobStatsTest.CompactionJobStatsTest/1 db/compaction/compaction_job_stats_test.cc:456: Failure Expected: (current_stats.total_input_bytes) <= (stats.total_input_bytes * (1.00 + kFileSizeBias)), actual: 63586 vs 63196.8 terminate called after throwing an instance of 'testing::internal::GoogleTestFailureException' what(): db/compaction/compaction_job_stats_test.cc:456: Failure Expected: (current_stats.total_input_bytes) <= (stats.total_input_bytes * (1.00 + kFileSizeBias)), actual: 63586 vs 63196.8 Received signal 6 (Aborted) Invoking GDB for stack trace... GDB failed; falling back on backtrace+addr2line... #0 /lib64/libc.so.6(gsignal+0x10f) [0x7fe80d31870f] ?? ??:0 #1 /lib64/libc.so.6(abort+0x127) [0x7fe80d302b25] ?? ??:0 #2 /lib64/libstdc++.so.6(+0x9006b) [0x7fe80dccd06b] ?? ??:0 #3 /lib64/libstdc++.so.6(+0x9650c) [0x7fe80dcd350c] ?? ??:0 #4 /lib64/libstdc++.so.6(+0x96567) [0x7fe80dcd3567] ?? ??:0 #5 /lib64/libstdc++.so.6(+0x967c8) [0x7fe80dcd37c8] ?? ??:0 #6 /root/rocksdb-8.9.1/librocksdb_test_debug.so(+0x5e40a) [0x7fe80fa4b40a] ?? ??:0 #7 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZNK7testing8internal12AssertHelperaSERKNS_7MessageE+0x6b) [0x7fe80fab2d6b] ?? ??:0 #8 ./compaction_job_stats_test() [0x41ebb3] rocksdb::CompactionJobStatsChecker::Verify(rocksdb::CompactionJobStats const&, rocksdb::CompactionJobStats const&) /root/rocksdb-8.9.1/db/compaction/compaction_job_stats_test.cc:455 (discriminator 7) #9 ./compaction_job_stats_test() [0x41c9fd] rocksdb::CompactionJobStatsChecker::OnCompactionCompleted(rocksdb::DB*, rocksdb::CompactionJobInfo const&) /usr/include/c++/8/bits/stl_deque.h:1592 #10 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb6DBImpl27NotifyOnCompactionCompletedEPNS_16ColumnFamilyDataEPNS_10CompactionERKNS_6StatusERKNS_18CompactionJobStatsEi+0x240) [0x7fe80f10aa00] ?? ??:0 #11 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb6DBImpl20BackgroundCompactionEPbPNS_10JobContextEPNS_9LogBufferEPNS0_19PrepickedCompactionENS_3Env8PriorityE+0x10ad) [0x7fe80f11272d] ?? ??:0 #12 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb6DBImpl24BackgroundCallCompactionEPNS0_19PrepickedCompactionENS_3Env8PriorityE+0x126) [0x7fe80f114546] ?? ??:0 #13 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb6DBImpl16BGWorkCompactionEPv+0x78) [0x7fe80f114cf8] ?? ??:0 #14 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb14ThreadPoolImpl4Impl8BGThreadEm+0x39d) [0x7fe80f48097d] ?? ??:0 #15 /root/rocksdb-8.9.1/librocksdb.so.8.9(_ZN7rocksdb14ThreadPoolImpl4Impl15BGThreadWrapperEPv+0x5e) [0x7fe80f480cfe] ?? ??:0 #16 /lib64/libstdc++.so.6(+0xc2b73) [0x7fe80dcffb73] ?? ??:0 #17 /lib64/libpthread.so.0(+0x82de) [0x7fe80e8ad2de] ?? ??:0 #18 /lib64/libc.so.6(clone+0x43) [0x7fe80d3dce83] ?? ??:0 t/run-compaction_job_stats_test-CompactionJobStatsTest-CompactionJobStatsTest.CompactionJobStatsTest-1: line 4: 1733847 Aborted (core dumped) TEST_TMPDIR=$d ./compaction_job_stats_test --gtest_filter=CompactionJobStatsTest/CompactionJobStatsTest.CompactionJobStatsTest/1 ETA: 145s Left: 8157 AVG: 0.02s local:32/7046/100%/0.0s Note: Google Test filter = DBWALTest.TruncateLastLogAfterRecoverWALEmpty [==========] Running 1 test from 1 test case. [----------] Global test environment set-up. [----------] 1 test from DBWALTest [ RUN ] DBWALTest.TruncateLastLogAfterRecoverWALEmpty db/db_wal_test.cc:2472: Failure Expected: (GetAllocatedFileSize(last_log)) < (preallocated_size), actual: 18427904 vs 18427904 terminate called without an active exception Received signal 6 (Aborted) Invoking GDB for stack trace... GDB failed; falling back on backtrace+addr2line... #0 /lib64/libc.so.6(gsignal+0x10f) [0x7f38f85a370f] ?? ??:0 #1 /lib64/libc.so.6(abort+0x127) [0x7f38f858db25] ?? ??:0 #2 /lib64/libstdc++.so.6(+0x9006b) [0x7f38f8f5806b] ?? ??:0 #3 /lib64/libstdc++.so.6(+0x9650c) [0x7f38f8f5e50c] ?? ??:0 #4 /lib64/libstdc++.so.6(+0x96567) [0x7f38f8f5e567] ?? ??:0 #5 ./db_wal_test() [0x454dcf] rocksdb::DBWALTest_TruncateLastLogAfterRecoverWALEmpty_Test::TestBody() /usr/include/c++/8/bits/basic_string.h:312 #6 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8internal35HandleExceptionsInMethodIfSupportedINS_4TestEvEET0_PT_MS4_FS3_vEPKc+0x3e) [0x7f38fad48d1e] ?? ??:0 #7 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing4Test3RunEv+0xd2) [0x7f38fad3f4a2] ?? ??:0 #8 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8TestInfo3RunEv+0x118) [0x7f38fad3f5e8] ?? ??:0 #9 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8TestCase3RunEv+0xb4) [0x7f38fad3f6c4] ?? ??:0 #10 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8internal12UnitTestImpl11RunAllTestsEv+0x474) [0x7f38fad3fbc4] ?? ??:0 #11 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8internal35HandleExceptionsInMethodIfSupportedINS0_12UnitTestImplEbEET0_PT_MS4_FS3_vEPKc+0x3e) [0x7f38fad4925e] ?? ??:0 #12 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8UnitTest3RunEv+0x8b) [0x7f38fad3fcdb] ?? ??:0 #13 ./db_wal_test() [0x42672d] main /root/rocksdb-8.9.1/db/db_wal_test.cc:2680 #14 /lib64/libc.so.6(__libc_start_main+0xf3) [0x7f38f858f6a3] ?? ??:0 #15 ./db_wal_test() [0x42911e] _start ??:? t/run-db_wal_test-DBWALTest.TruncateLastLogAfterRecoverWALEmpty: line 4: 2523881 Aborted (core dumped) TEST_TMPDIR=$d ./db_wal_test --gtest_filter=DBWALTest.TruncateLastLogAfterRecoverWALEmpty ETA: 20s Left: 1510 AVG: 0.01s local:32/13693/100%/0.0s Note: Google Test filter = DBTieredSecondaryCacheTest.BasicMultiGetTest [==========] Running 1 test from 1 test case. [----------] Global test environment set-up. [----------] 1 test from DBTieredSecondaryCacheTest [ RUN ] DBTieredSecondaryCacheTest.BasicMultiGetTest cache/tiered_secondary_cache_test.cc:382: Failure Expected equality of these values: nvm_sec_cache()->num_insert_saved() Which is: 0 3u Which is: 3 terminate called after throwing an instance of 'testing::internal::GoogleTestFailureException' what(): cache/tiered_secondary_cache_test.cc:382: Failure Expected equality of these values: nvm_sec_cache()->num_insert_saved() Which is: 0 3u Which is: 3 Received signal 6 (Aborted) Invoking GDB for stack trace... GDB failed; falling back on backtrace+addr2line... #0 /lib64/libc.so.6(gsignal+0x10f) [0x7fb70a68a70f] ?? ??:0 #1 /lib64/libc.so.6(abort+0x127) [0x7fb70a674b25] ?? ??:0 #2 /lib64/libstdc++.so.6(+0x9006b) [0x7fb70b03f06b] ?? ??:0 #3 /lib64/libstdc++.so.6(+0x9650c) [0x7fb70b04550c] ?? ??:0 #4 /lib64/libstdc++.so.6(+0x96567) [0x7fb70b045567] ?? ??:0 #5 /lib64/libstdc++.so.6(__cxa_rethrow+0x4d) [0x7fb70b04581d] ?? ??:0 #6 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8internal35HandleExceptionsInMethodIfSupportedINS0_12UnitTestImplEbEET0_PT_MS4_FS3_vEPKc+0xe5) [0x7fb70ce30305] ?? ??:0 #7 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8UnitTest3RunEv+0x8b) [0x7fb70ce26cdb] ?? ??:0 #8 ./tiered_secondary_cache_test() [0x40824d] main /root/rocksdb-8.9.1/cache/tiered_secondary_cache_test.cc:711 #9 /lib64/libc.so.6(__libc_start_main+0xf3) [0x7fb70a6766a3] ?? ??:0 #10 ./tiered_secondary_cache_test() [0x40881e] _start ??:? t/run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.BasicMultiGetTest: line 4: 2877125 Aborted (core dumped) TEST_TMPDIR=$d ./tiered_secondary_cache_test --gtest_filter=DBTieredSecondaryCacheTest.BasicMultiGetTest Note: Google Test filter = DBTieredSecondaryCacheTest.BasicTest [==========] Running 1 test from 1 test case. [----------] Global test environment set-up. [----------] 1 test from DBTieredSecondaryCacheTest [ RUN ] DBTieredSecondaryCacheTest.BasicTest cache/tiered_secondary_cache_test.cc:267: Failure Expected equality of these values: nvm_sec_cache()->num_insert_saved() Which is: 0 1u Which is: 1 terminate called after throwing an instance of 'testing::internal::GoogleTestFailureException' what(): cache/tiered_secondary_cache_test.cc:267: Failure Expected equality of these values: nvm_sec_cache()->num_insert_saved() Which is: 0 1u Which is: 1 Received signal 6 (Aborted) Invoking GDB for stack trace... GDB failed; falling back on backtrace+addr2line... #0 /lib64/libc.so.6(gsignal+0x10f) [0x7f3326a4570f] ?? ??:0 #1 /lib64/libc.so.6(abort+0x127) [0x7f3326a2fb25] ?? ??:0 #2 /lib64/libstdc++.so.6(+0x9006b) [0x7f33273fa06b] ?? ??:0 #3 /lib64/libstdc++.so.6(+0x9650c) [0x7f332740050c] ?? ??:0 #4 /lib64/libstdc++.so.6(+0x96567) [0x7f3327400567] ?? ??:0 #5 /lib64/libstdc++.so.6(__cxa_rethrow+0x4d) [0x7f332740081d] ?? ??:0 #6 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8internal35HandleExceptionsInMethodIfSupportedINS0_12UnitTestImplEbEET0_PT_MS4_FS3_vEPKc+0xe5) [0x7f33291eb305] ?? ??:0 #7 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8UnitTest3RunEv+0x8b) [0x7f33291e1cdb] ?? ??:0 #8 ./tiered_secondary_cache_test() [0x40824d] main /root/rocksdb-8.9.1/cache/tiered_secondary_cache_test.cc:711 #9 /lib64/libc.so.6(__libc_start_main+0xf3) [0x7f3326a316a3] ?? ??:0 #10 ./tiered_secondary_cache_test() [0x40881e] _start ??:? t/run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.BasicTest: line 4: 2877202 Aborted (core dumped) TEST_TMPDIR=$d ./tiered_secondary_cache_test --gtest_filter=DBTieredSecondaryCacheTest.BasicTest Note: Google Test filter = DBTieredSecondaryCacheTest.WaitAllTest [==========] Running 1 test from 1 test case. [----------] Global test environment set-up. [----------] 1 test from DBTieredSecondaryCacheTest [ RUN ] DBTieredSecondaryCacheTest.WaitAllTest cache/tiered_secondary_cache_test.cc:524: Failure Expected equality of these values: nvm_sec_cache()->num_insert_saved() Which is: 0 3u Which is: 3 terminate called after throwing an instance of 'testing::internal::GoogleTestFailureException' what(): cache/tiered_secondary_cache_test.cc:524: Failure Expected equality of these values: nvm_sec_cache()->num_insert_saved() Which is: 0 3u Which is: 3 Received signal 6 (Aborted) Invoking GDB for stack trace... GDB failed; falling back on backtrace+addr2line... #0 /lib64/libc.so.6(gsignal+0x10f) [0x7efd1f15270f] ?? ??:0 #1 /lib64/libc.so.6(abort+0x127) [0x7efd1f13cb25] ?? ??:0 #2 /lib64/libstdc++.so.6(+0x9006b) [0x7efd1fb0706b] ?? ??:0 #3 /lib64/libstdc++.so.6(+0x9650c) [0x7efd1fb0d50c] ?? ??:0 #4 /lib64/libstdc++.so.6(+0x96567) [0x7efd1fb0d567] ?? ??:0 #5 /lib64/libstdc++.so.6(__cxa_rethrow+0x4d) [0x7efd1fb0d81d] ?? ??:0 #6 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8internal35HandleExceptionsInMethodIfSupportedINS0_12UnitTestImplEbEET0_PT_MS4_FS3_vEPKc+0xe5) [0x7efd218f8305] ?? ??:0 #7 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8UnitTest3RunEv+0x8b) [0x7efd218eecdb] ?? ??:0 #8 ./tiered_secondary_cache_test() [0x40824d] main /root/rocksdb-8.9.1/cache/tiered_secondary_cache_test.cc:711 #9 /lib64/libc.so.6(__libc_start_main+0xf3) [0x7efd1f13e6a3] ?? ??:0 #10 ./tiered_secondary_cache_test() [0x40881e] _start ??:? t/run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.WaitAllTest: line 4: 2877258 Aborted (core dumped) TEST_TMPDIR=$d ./tiered_secondary_cache_test --gtest_filter=DBTieredSecondaryCacheTest.WaitAllTest Note: Google Test filter = DBTieredSecondaryCacheTest.IterateTest [==========] Running 1 test from 1 test case. [----------] Global test environment set-up. [----------] 1 test from DBTieredSecondaryCacheTest [ RUN ] DBTieredSecondaryCacheTest.IterateTest cache/tiered_secondary_cache_test.cc:623: Failure Expected equality of these values: nvm_sec_cache()->num_insert_saved() Which is: 0 8u Which is: 8 terminate called after throwing an instance of 'testing::internal::GoogleTestFailureException' what(): cache/tiered_secondary_cache_test.cc:623: Failure Expected equality of these values: nvm_sec_cache()->num_insert_saved() Which is: 0 8u Which is: 8 Received signal 6 (Aborted) Invoking GDB for stack trace... GDB failed; falling back on backtrace+addr2line... #0 /lib64/libc.so.6(gsignal+0x10f) [0x7f512e5bf70f] ?? ??:0 #1 /lib64/libc.so.6(abort+0x127) [0x7f512e5a9b25] ?? ??:0 #2 /lib64/libstdc++.so.6(+0x9006b) [0x7f512ef7406b] ?? ??:0 #3 /lib64/libstdc++.so.6(+0x9650c) [0x7f512ef7a50c] ?? ??:0 #4 /lib64/libstdc++.so.6(+0x96567) [0x7f512ef7a567] ?? ??:0 #5 /lib64/libstdc++.so.6(__cxa_rethrow+0x4d) [0x7f512ef7a81d] ?? ??:0 #6 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8internal35HandleExceptionsInMethodIfSupportedINS0_12UnitTestImplEbEET0_PT_MS4_FS3_vEPKc+0xe5) [0x7f5130d65305] ?? ??:0 #7 /root/rocksdb-8.9.1/librocksdb_test_debug.so(_ZN7testing8UnitTest3RunEv+0x8b) [0x7f5130d5bcdb] ?? ??:0 #8 ./tiered_secondary_cache_test() [0x40824d] main /root/rocksdb-8.9.1/cache/tiered_secondary_cache_test.cc:711 #9 /lib64/libc.so.6(__libc_start_main+0xf3) [0x7f512e5ab6a3] ?? ??:0 #10 ./tiered_secondary_cache_test() [0x40881e] _start ??:? t/run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.IterateTest: line 4: 2877201 Aborted (core dumped) TEST_TMPDIR=$d ./tiered_secondary_cache_test --gtest_filter=DBTieredSecondaryCacheTest.IterateTest ETA: 0s Left: 2 AVG: 0.01s local:2/15201/100%/0.0s ETA: 0s Left: 0 AVG: 0.01s local:0/15203/100%/0.0s Seq Host Starttime JobRuntime Send Receive Exitval Signal Command 3522 : 1705129126.818 7.490 0 0 134 0 t/run-compaction_job_stats_test-CompactionJobStatsTest-CompactionJobStatsTest.CompactionJobStatsTest-0 >& t/log-run-compaction_job_stats_test-CompactionJobStatsTest-CompactionJobStatsTest.CompactionJobStatsTest-0 || bash -c "cat t/log-run-compaction_job_stats_test-CompactionJobStatsTest-CompactionJobStatsTest.CompactionJobStatsTest-0; exit $?" 3523 : 1705129126.824 7.492 0 0 134 0 t/run-compaction_job_stats_test-CompactionJobStatsTest-CompactionJobStatsTest.CompactionJobStatsTest-1 >& t/log-run-compaction_job_stats_test-CompactionJobStatsTest-CompactionJobStatsTest.CompactionJobStatsTest-1 || bash -c "cat t/log-run-compaction_job_stats_test-CompactionJobStatsTest-CompactionJobStatsTest.CompactionJobStatsTest-1; exit $?" 7008 : 1705129205.799 6.342 0 0 134 0 t/run-db_wal_test-DBWALTest.TruncateLastLogAfterRecoverWALEmpty >& t/log-run-db_wal_test-DBWALTest.TruncateLastLogAfterRecoverWALEmpty || bash -c "cat t/log-run-db_wal_test-DBWALTest.TruncateLastLogAfterRecoverWALEmpty; exit $?" 13516 : 1705129266.253 4.743 0 0 134 0 t/run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.BasicMultiGetTest >& t/log-run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.BasicMultiGetTest || bash -c "cat t/log-run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.BasicMultiGetTest; exit $?" 13517 : 1705129266.286 4.730 0 0 134 0 t/run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.BasicTest >& t/log-run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.BasicTest || bash -c "cat t/log-run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.BasicTest; exit $?" 13519 : 1705129266.336 4.687 0 0 134 0 t/run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.WaitAllTest >& t/log-run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.WaitAllTest || bash -c "cat t/log-run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.WaitAllTest; exit $?" 13518 : 1705129266.328 4.820 0 0 134 0 t/run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.IterateTest >& t/log-run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.IterateTest || bash -c "cat t/log-run-tiered_secondary_cache_test-DBTieredSecondaryCacheTest.IterateTest; exit $?" make[1]: *** [Makefile:997: check_0] Error 1 make[1]: Leaving directory '/root/rocksdb-8.9.1' make: *** [Makefile:1053: check] Error 2 [root@ff6189696b45 rocksdb-8.9.1]#

adv0518 avatar Jan 13 '24 07:01 adv0518

gcc version:

[root@ff6189696b45 rocksdb-8.9.1]# gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/8/lto-wrapper OFFLOAD_TARGET_NAMES=nvptx-none OFFLOAD_TARGET_DEFAULT=1 Target: x86_64-redhat-linux Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,fortran,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl --disable-libmpx --enable-offload-targets=nvptx-none --without-cuda-driver --enable-gnu-indirect-function --enable-cet --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux Thread model: posix gcc version 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC)

adv0518 avatar Jan 13 '24 07:01 adv0518

operate system info(it's a docker):

[root@ff6189696b45 rocksdb-8.9.1]# more /etc/redhat-release CentOS Linux release 8.5.2111

adv0518 avatar Jan 13 '24 07:01 adv0518

Hello @adv0518 ,

I tested compilation on old unmaintained Centos 8 and I didn't found any problem. I tested two version, latest from main branch and the version you mentioned. I compiled everything with make : make all and then run the test you mentioned : compaction_job_stats_test, db_wal_test, tiered_secondary_cache_test.

Can you please confirm if this problem still persist or if you can provide more info how are you compiling and how we can replicate the issue.

$ cat /etc/redhat-release
CentOS Linux release 8.5.2111

Radek

rhubner avatar Aug 21 '24 12:08 rhubner