redisraft
redisraft copied to clipboard
Published
20 hours ago •
RedisLabs
RedisLabs
Fuzzing redisraft assertion failure
We are a group of researchers testing distributed protocol implementations. While testing redisraft, we encountered the following crash (with a 3 node cluster) and an assertion failure.
logs for node: 1
----- Stdout -----
2313575:C 18 Oct 2023 00:10:56.382 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
2313575:C 18 Oct 2023 00:10:56.382 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
2313575:C 18 Oct 2023 00:10:56.382 * Redis version=255.255.255, bits=64, commit=e04ebdb8, modified=1, pid=2313575, just started
2313575:C 18 Oct 2023 00:10:56.382 * Configuration loaded
2313575:M 18 Oct 2023 00:10:56.382 * monotonic clock: POSIX clock_gettime
2313575:M 18 Oct 2023 00:10:56.382 * Running mode=standalone, port=5091.
2313575:M 18 Oct 2023 00:10:56.386 * <raft> RedisRaft version 255.255.255 [c259f3d2]
2313575:M 18 Oct 2023 00:10:56.391 * <raft> Initializing test network
2313575:M 18 Oct 2023 00:10:56.391 * <raft> Created test network client
2313575:M 18 Oct 2023 00:10:56.395 * <raft> Raft module loaded, state is 'loading'
2313575:M 18 Oct 2023 00:10:56.395 * Module 'raft' loaded from /Fuzzing/redisraft-fuzzing/redisraft.so
2313575:M 18 Oct 2023 00:10:56.395 * Server initialized
2313575:M 18 Oct 2023 00:10:56.395 . The AOF directory appendonlydir doesn't exist
2313575:M 18 Oct 2023 00:10:56.395 * Ready to accept connections tcp
2313575:M 18 Oct 2023 00:10:56.395 . 0 clients connected (0 replicas), 1746752 bytes in use
2313575:M 18 Oct 2023 00:10:56.435 - Accepted 127.0.0.1:37176
2313575:M 18 Oct 2023 00:10:56.435 # Missing implement of connection type tls
2313575:M 18 Oct 2023 00:10:56.495 * <raft> Loading: Redis loading complete, snapshot NOT LOADED
2313575:M 18 Oct 2023 00:10:56.495 * <raft> Cluster Membership: term:0 index:0 nodes: id=1,voting=0,active=1,addr=-
2313575:M 18 Oct 2023 00:10:56.496 * <raft> Test network starting message polling thread
=== REDIS BUG REPORT START: Cut & paste starting from here ===
2313575:M 18 Oct 2023 00:10:56.496 # === ASSERTION FAILED ===
2313575:M 18 Oct 2023 00:10:56.496 # ==> /Fuzzing/redisraft-fuzzing/src/log.c:509 'ety != NULL' is not true
2313575:M 18 Oct 2023 00:10:56.496 # === ASSERTION FAILED ===
2313575:M 18 Oct 2023 00:10:56.496 # ==> /Fuzzing/redisraft-fuzzing/src/log.c:509 'ety != NULL' is not true
------ STACK TRACE ------
Backtrace:
------ STACK TRACE ------
Backtrace:
/Fuzzing/redisraft-fuzzing/redisraft.so(+0x25e80)[0x7f35d9b28e80]
/Fuzzing/redisraft-fuzzing/redisraft.so(+0x25e80)[0x7f35d9b28e80]
/Fuzzing/redisraft-fuzzing/redisraft.so(+0x27208)[0x7f35d9b2a208]
/Fuzzing/redisraft-fuzzing/redisraft.so(+0x27347)[0x7f35d9b2a347]
/Fuzzing/redisraft-fuzzing/redisraft.so(raft_restore_log+0x54)[0x7f35d9b43af4]
/Fuzzing/redisraft-fuzzing/redisraft.so(+0x27208)[0x7f35d9b2a208]
/Fuzzing/redisraft-fuzzing/redisraft.so(callRaftPeriodic+0x131)[0x7f35d9b2fe71]
/Fuzzing/redisraft-fuzzing/redisraft.so(+0x27347)[0x7f35d9b2a347]
/Fuzzing/redisraft-fuzzing/redisraft.so(raft_get_entry_from_idx+0x1a)[0x7f35d9b3e54a]
/Fuzzing/redis/src/redis-server 0.0.0.0:5091(moduleTimerHandler+0x21c)[0x564bc592498c]
/Fuzzing/redisraft-fuzzing/redisraft.so(raft_get_last_log_term+0x36)[0x7f35d9b44576]
/Fuzzing/redisraft-fuzzing/redisraft.so(raft_recv_requestvote+0x25b)[0x7f35d9b3f29b]
/Fuzzing/redisraft-fuzzing/redisraft.so(handleTestNetworkMessage+0x204)[0x7f35d9b38594]
/Fuzzing/redisraft-fuzzing/redisraft.so(poll_redis_test_messages+0x89)[0x7f35d9b38659]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7ea7)[0x7f35d9d6cea7]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7f35d9c8ca2f]
------ INFO OUTPUT ------
/Fuzzing/redis/src/redis-server 0.0.0.0:5091(+0x81705)[0x564bc582e705]
/Fuzzing/redis/src/redis-server 0.0.0.0:5091(aeMain+0x1c8)[0x564bc5838cb8]
/Fuzzing/redis/src/redis-server 0.0.0.0:5091(main+0x3cd)[0x564bc582de2d]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea)[0x7f35d9bb4d0a]
/Fuzzing/redis/src/redis-server 0.0.0.0:5091(_start+0x2a)[0x564bc582e5aa]
------ INFO OUTPUT ------
# Server
redis_version:255.255.255
redis_git_sha1:e04ebdb8
redis_git_dirty:1
redis_build_id:1f78dcc7b4fbdd25
redis_mode:standalone
os:Linux 5.15.111.1.amd64-smp x86_64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:10.2.1
process_id:2313575
process_supervised:no
run_id:fdcd237db91d6f158dcf7a9543a5cfad19c64980
tcp_port:5091
server_time_usec:1697580656495247
uptime_in_seconds:0
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:3081840
executable:/Fuzzing/redis/src/redis-server
config_file:
io_threads_active:0
listener0:name=tcp,bind=0.0.0.0,port=5091
# Clients
connected_clients:1
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:0
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0
total_blocking_keys:0
total_blocking_keys_on_nokey:0
# Memory
used_memory:1816960
used_memory_human:1.73M
used_memory_rss:15511552
used_memory_rss_human:14.79M
used_memory_peak:1816960
used_memory_peak_human:1.73M
used_memory_peak_perc:100.39%
used_memory_overhead:1746792
used_memory_startup:1746608
used_memory_dataset:70168
used_memory_dataset_perc:99.74%
allocator_allocated:2221896
allocator_active:2760704
allocator_resident:10928128
total_system_memory:270386737152
total_system_memory_human:251.82G
used_memory_lua:31744
used_memory_vm_eval:31744
used_memory_lua_human:31.00K
used_memory_scripts_eval:0
number_of_cached_scripts:0
number_of_functions:0
number_of_libraries:0
used_memory_vm_functions:32768
used_memory_vm_total:64512
used_memory_vm_total_human:63.00K
used_memory_functions:184
used_memory_scripts:184
used_memory_scripts_human:184B
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.24
allocator_frag_bytes:538808
allocator_rss_ratio:3.96
allocator_rss_bytes:8167424
rss_overhead_ratio:1.42
rss_overhead_bytes:4583424
mem_fragmentation_ratio:8.88
mem_fragmentation_bytes:13764800
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_total_replication_buffers:0
mem_clients_slaves:0
mem_clients_normal:0
mem_cluster_links:0
mem_aof_buffer:0
mem_allocator:jemalloc-5.3.0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0
# Persistence
loading:0
async_loading:0
current_cow_peak:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1697580656
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_saves:0
rdb_last_cow_size:0
rdb_last_load_keys_expired:0
rdb_last_load_keys_loaded:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_rewrites:0
aof_rewrites_consecutive_failures:0
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0
# Stats
total_connections_received:1
total_commands_processed:7
instantaneous_ops_per_sec:28
total_net_input_bytes:33
total_net_output_bytes:5
total_net_repl_input_bytes:0
total_net_repl_output_bytes:0
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.00
instantaneous_input_repl_kbps:0.00
instantaneous_output_repl_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
evicted_clients:0
total_eviction_exceeded_time:0
current_eviction_exceeded_time:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
pubsubshard_channels:0
latest_fork_usec:0
total_forks:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
total_active_defrag_time:0
current_active_defrag_time:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:0
dump_payload_sanitizations:0
total_reads_processed:1
total_writes_processed:1
io_threaded_reads_processed:0
io_threaded_writes_processed:0
reply_buffer_shrinks:0
reply_buffer_expands:0
acl_access_denied_auth:0
acl_access_denied_cmd:0
acl_access_denied_key:0
acl_access_denied_channel:0
# Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:e43b6d85e76fe44f464509fce81de90da1721b54
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
# CPU
used_cpu_sys:0.000000
used_cpu_user:0.014899
used_cpu_sys_children:0.000000
used_cpu_user_children:0.000000
used_cpu_sys_main_thread:0.000000
used_cpu_user_main_thread:0.000578
# Modules
module:name=raft,ver=1,api=1,filters=1,usedby=[],using=[],options=[]
# Commandstats
cmdstat_auth:calls=1,usec=1,usec_per_call=1.00,rejected_calls=0,failed_calls=0
cmdstat_command:calls=2,usec=1445,usec_per_call=722.50,rejected_calls=0,failed_calls=0
cmdstat_config|set:calls=1,usec=6,usec_per_call=6.00,rejected_calls=0,failed_calls=0
cmdstat_config|get:calls=3,usec=3,usec_per_call=1.00,rejected_calls=0,failed_calls=0
# Errorstats
# Latencystats
latency_percentiles_usec_auth:p50=1.003,p99=1.003,p99.9=1.003
latency_percentiles_usec_command:p50=704.511,p99=745.471,p99.9=745.471
latency_percentiles_usec_config|set:p50=6.015,p99=6.015,p99.9=6.015
latency_percentiles_usec_config|get:p50=1.003,p99=1.003,p99.9=1.003
# Cluster
cluster_enabled:0
# Keyspace
------ CLIENT LIST OUTPUT ------
# Server
redis_version:255.255.255
redis_git_sha1:e04ebdb8
redis_git_dirty:1
redis_build_id:1f78dcc7b4fbdd25
redis_mode:standalone
os:Linux 5.15.111.1.amd64-smp x86_64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:10.2.1
process_id:2313575
process_supervised:no
run_id:fdcd237db91d6f158dcf7a9543a5cfad19c64980
tcp_port:5091
server_time_usec:1697580656495247
uptime_in_seconds:0
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:3081840
executable:/Fuzzing/redis/src/redis-server
config_file:
io_threads_active:0
listener0:name=tcp,bind=0.0.0.0,port=5091
# Clients
connected_clients:1
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:0
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0
total_blocking_keys:0
total_blocking_keys_on_nokey:0
# Memory
used_memory:1825264
used_memory_human:1.74M
used_memory_rss:15511552
used_memory_rss_human:14.79M
used_memory_peak:1825264
used_memory_peak_human:1.74M
used_memory_peak_perc:100.46%
used_memory_overhead:1746792
used_memory_startup:1746608
used_memory_dataset:78472
used_memory_dataset_perc:99.77%
allocator_allocated:2221896
allocator_active:2760704
allocator_resident:10928128
total_system_memory:270386737152
total_system_memory_human:251.82G
used_memory_lua:31744
used_memory_vm_eval:31744
used_memory_lua_human:31.00K
used_memory_scripts_eval:0
number_of_cached_scripts:0
number_of_functions:0
number_of_libraries:0
used_memory_vm_functions:32768
used_memory_vm_total:64512
used_memory_vm_total_human:63.00K
used_memory_functions:184
used_memory_scripts:184
used_memory_scripts_human:184B
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.24
allocator_frag_bytes:538808
allocator_rss_ratio:3.96
allocator_rss_bytes:8167424
rss_overhead_ratio:1.42
rss_overhead_bytes:4583424
mem_fragmentation_ratio:8.88
mem_fragmentation_bytes:13764800
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_total_replication_buffers:0
mem_clients_slaves:0
mem_clients_normal:0
mem_cluster_links:0
mem_aof_buffer:0
mem_allocator:jemalloc-5.3.0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0
# Persistence
loading:0
async_loading:0
current_cow_peak:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1697580656
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_saves:0
rdb_last_cow_size:0
rdb_last_load_keys_expired:0
rdb_last_load_keys_loaded:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_rewrites:0
aof_rewrites_consecutive_failures:0
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0
# Stats
total_connections_received:1
total_commands_processed:7
instantaneous_ops_per_sec:28
total_net_input_bytes:33
total_net_output_bytes:5
total_net_repl_input_bytes:0
total_net_repl_output_bytes:0
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.00
instantaneous_input_repl_kbps:0.00
instantaneous_output_repl_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
evicted_clients:0
total_eviction_exceeded_time:0
current_eviction_exceeded_time:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
pubsubshard_channels:0
latest_fork_usec:0
total_forks:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
total_active_defrag_time:0
current_active_defrag_time:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:0
dump_payload_sanitizations:0
total_reads_processed:1
total_writes_processed:1
io_threaded_reads_processed:0
io_threaded_writes_processed:0
reply_buffer_shrinks:0
reply_buffer_expands:0
acl_access_denied_auth:0
acl_access_denied_cmd:0
acl_access_denied_key:0
acl_access_denied_channel:0
# Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:e43b6d85e76fe44f464509fce81de90da1721b54
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
# CPU
used_cpu_sys:0.000000
used_cpu_user:0.015133
used_cpu_sys_children:0.000000
used_cpu_user_children:0.000000
used_cpu_sys_main_thread:0.000000
used_cpu_user_main_thread:0.013686
# Modules
module:name=raft,ver=1,api=1,filters=1,usedby=[],using=[],options=[]
# Commandstats
cmdstat_auth:calls=1,usec=1,usec_per_call=1.00,rejected_calls=0,failed_calls=0
cmdstat_command:calls=2,usec=1445,usec_per_call=722.50,rejected_calls=0,failed_calls=0
cmdstat_config|set:calls=1,usec=6,usec_per_call=6.00,rejected_calls=0,failed_calls=0
cmdstat_config|get:calls=3,usec=3,usec_per_call=1.00,rejected_calls=0,failed_calls=0
# Errorstats
# Latencystats
latency_percentiles_usec_auth:p50=1.003,p99=1.003,p99.9=1.003
latency_percentiles_usec_command:p50=704.511,p99=745.471,p99.9=745.471
latency_percentiles_usec_config|set:p50=6.015,p99=6.015,p99.9=6.015
latency_percentiles_usec_config|get:p50=1.003,p99=1.003,p99.9=1.003
# Cluster
cluster_enabled:0
# Keyspace
------ CLIENT LIST OUTPUT ------
id=8 addr=127.0.0.1:37176 laddr=127.0.0.1:5091 fd=11 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=37760 events=r cmd=auth user=default redir=-1 resp=2 lib-name= lib-ver=
------ MODULES INFO OUTPUT ------
id=8 addr=127.0.0.1:37176 laddr=127.0.0.1:5091 fd=11 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=37760 events=r cmd=auth user=default redir=-1 resp=2 lib-name= lib-ver=
------ MODULES INFO OUTPUT ------
# raft_version
raft_version:255.255.255
raft_git_sha1:c259f3d2
# raft_general
raft_dbid:3935d749b666111a88fdd724f11ca6fe
raft_node_id:1
raft_state:up
raft_role:follower
raft_is_voting:yes
raft_voted_for:-1
raft_leader_id:-1
raft_current_term:0
raft_num_nodes:1
raft_num_voting_nodes:1
# raft_log
raft_log_entries:8
raft_current_index:8
raft_commit_index:0
raft_last_applied_index:0
raft_file_size:1863
raft_cache_memory_size:0
raft_cache_entries:0
raft_client_attached_entries:0
raft_fsync_count:0
raft_fsync_max_microseconds:0
raft_fsync_avg_microseconds:0
# raft_snapshot
raft_snapshot_filename:redis1.rdb
raft_snapshot_last_idx:0
raft_snapshot_last_term:0
raft_snapshot_size:0
raft_snapshot_time_secs:-1
raft_snapshots_created:0
raft_snapshots_received:0
raft_snapshot_in_progress:no
raft_snapshot_in_progress_last_idx:-1
raft_snapshot_in_progress_last_term:-1
# raft_clients
raft_proxy_reqs:0
raft_proxy_failed_reqs:0
raft_proxy_failed_responses:0
raft_proxy_outstanding_reqs:0
# raft_stats
raft_appendreq_received:0
raft_appendreq_with_entry_received:0
raft_snapshotreq_received:0
raft_exec_throttled:0
raft_num_sessions:0
# raft_version
raft_version:255.255.255
raft_git_sha1:c259f3d2
# raft_general
raft_dbid:3935d749b666111a88fdd724f11ca6fe
raft_node_id:1
raft_state:up
raft_role:follower
raft_is_voting:yes
raft_voted_for:-1
raft_leader_id:-1
raft_current_term:0
raft_num_nodes:1
raft_num_voting_nodes:1
# raft_log
raft_log_entries:8
raft_current_index:8
raft_commit_index:0
raft_last_applied_index:0
raft_file_size:1863
raft_cache_memory_size:0
raft_cache_entries:0
raft_client_attached_entries:0
raft_fsync_count:0
raft_fsync_max_microseconds:0
raft_fsync_avg_microseconds:0
# raft_snapshot
raft_snapshot_filename:redis1.rdb
raft_snapshot_last_idx:0
raft_snapshot_last_term:0
raft_snapshot_size:0
raft_snapshot_time_secs:-1
raft_snapshots_created:0
raft_snapshots_received:0
raft_snapshot_in_progress:no
raft_snapshot_in_progress_last_idx:-1
raft_snapshot_in_progress_last_term:-1
# raft_clients
raft_proxy_reqs:0
raft_proxy_failed_reqs:0
raft_proxy_failed_responses:0
raft_proxy_outstanding_reqs:0
# raft_stats
raft_appendreq_received:0
raft_appendreq_with_entry_received:0
raft_snapshotreq_received:0
raft_exec_throttled:0
raft_num_sessions:0
------ CONFIG DEBUG OUTPUT ------
lazyfree-lazy-user-del no
lazyfree-lazy-eviction no
lazyfree-lazy-server-del no
repl-diskless-sync yes
slave-read-only yes
sanitize-dump-payload no
lazyfree-lazy-expire no
io-threads 1
lazyfree-lazy-user-flush no
repl-diskless-load disabled
client-query-buffer-limit 1gb
io-threads-do-reads no
replica-read-only yes
activedefrag no
proto-max-bulk-len 512mb
list-compress-depth 0
------ FAST MEMORY TEST ------
------ CONFIG DEBUG OUTPUT ------
lazyfree-lazy-user-del no
lazyfree-lazy-eviction no
lazyfree-lazy-server-del no
repl-diskless-sync yes
slave-read-only yes
sanitize-dump-payload no
lazyfree-lazy-expire no
io-threads 1
lazyfree-lazy-user-flush no
repl-diskless-load disabled
client-query-buffer-limit 1gb
io-threads-do-reads no
replica-read-only yes
activedefrag no
proto-max-bulk-len 512mb
list-compress-depth 0
------ FAST MEMORY TEST ------
2313575:M 18 Oct 2023 00:10:56.497 # main thread terminated
2313575:M 18 Oct 2023 00:10:56.497 # Bio worker thread #1 terminated
2313575:M 18 Oct 2023 00:10:56.497 # Bio worker thread #2 terminated
Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.
=== REDIS BUG REPORT END. Make sure to include from START to END. ===
Please report the crash by opening an issue on github:
http://github.com/redis/redis/issues
If a Redis module was involved, please open in the module's repo instead.
Suspect RAM error? Use redis-server --test-memory to verify it.
Some other issues could be detected by redis-server --check-system
----- Stderr -----
logs for node: 2
----- Stdout -----
2313992:C 18 Oct 2023 00:10:56.870 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
2313992:C 18 Oct 2023 00:10:56.870 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
2313992:C 18 Oct 2023 00:10:56.870 * Redis version=255.255.255, bits=64, commit=e04ebdb8, modified=1, pid=2313992, just started
2313992:C 18 Oct 2023 00:10:56.870 * Configuration loaded
2313992:M 18 Oct 2023 00:10:56.871 * monotonic clock: POSIX clock_gettime
2313992:M 18 Oct 2023 00:10:56.871 * Running mode=standalone, port=5092.
2313992:M 18 Oct 2023 00:10:56.876 * <raft> RedisRaft version 255.255.255 [c259f3d2]
2313992:M 18 Oct 2023 00:10:56.881 * <raft> Initializing test network
2313992:M 18 Oct 2023 00:10:56.881 * <raft> Created test network client
2313992:M 18 Oct 2023 00:10:56.884 * <raft> Raft module loaded, state is 'loading'
2313992:M 18 Oct 2023 00:10:56.884 * Module 'raft' loaded from /Fuzzing/redisraft-fuzzing/redisraft.so
2313992:M 18 Oct 2023 00:10:56.885 * Server initialized
2313992:M 18 Oct 2023 00:10:56.885 . The AOF directory appendonlydir doesn't exist
2313992:M 18 Oct 2023 00:10:56.885 * Ready to accept connections tcp
2313992:M 18 Oct 2023 00:10:56.885 . 0 clients connected (0 replicas), 1746688 bytes in use
2313992:M 18 Oct 2023 00:10:56.985 * <raft> Loading: Redis loading complete, snapshot NOT LOADED
2313992:M 18 Oct 2023 00:10:56.985 * <raft> Cluster Membership: term:0 index:0 nodes: id=2,voting=0,active=1,addr=-
2313992:M 18 Oct 2023 00:10:56.986 * <raft> Test network starting message polling thread
2313992:M 18 Oct 2023 00:10:56.986 * <raft> Cluster Membership: term:0 index:4 nodes: id=2,voting=0,active=1,addr=- id=1,voting=1,active=1,addr=localhost:5091
2313992:M 18 Oct 2023 00:10:56.987 * <raft> Cluster Membership: term:0 index:4 nodes: id=2,voting=0,active=1,addr=- id=1,voting=1,active=1,addr=localhost:5091 id=3,voting=0,active=1,addr=localhost:5093
2313992:M 18 Oct 2023 00:10:56.987 * <raft> Raft state after loading log: log_count=4, first_idx=1, current_idx=4, last_applied_idx=0
2313992:M 18 Oct 2023 00:10:56.987 . <raft> Raft term=1, vote=-1
2313992:M 18 Oct 2023 00:10:57.287 * <raft> State change: Election starting, node is now a pre-candidate, term 1
2313992:M 18 Oct 2023 00:10:57.287 * <raft> Cluster Membership: term:1 index:4 nodes: id=2,voting=0,active=1,addr=- id=1,voting=1,active=1,addr=localhost:5091 id=3,voting=0,active=1,addr=localhost:5093
2313992:M 18 Oct 2023 00:10:57.327 * <raft> State change: Node is now a follower, term 1
2313992:M 18 Oct 2023 00:10:57.327 * <raft> Cluster Membership: term:1 index:4 nodes: id=2,voting=0,active=1,addr=- id=1,voting=1,active=1,addr=localhost:5091 id=3,voting=0,active=1,addr=localhost:5093
2313992:M 18 Oct 2023 00:10:57.382 - Accepted 127.0.0.1:37074
2313992:M 18 Oct 2023 00:10:57.382 # Missing implement of connection type tls
----- Stderr -----
logs for node: 3
----- Stdout -----
2314245:C 18 Oct 2023 00:10:57.266 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
2314245:C 18 Oct 2023 00:10:57.266 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
2314245:C 18 Oct 2023 00:10:57.266 * Redis version=255.255.255, bits=64, commit=e04ebdb8, modified=1, pid=2314245, just started
2314245:C 18 Oct 2023 00:10:57.266 * Configuration loaded
2314245:M 18 Oct 2023 00:10:57.267 * monotonic clock: POSIX clock_gettime
2314245:M 18 Oct 2023 00:10:57.267 * Running mode=standalone, port=5093.
2314245:M 18 Oct 2023 00:10:57.271 * <raft> RedisRaft version 255.255.255 [c259f3d2]
2314245:M 18 Oct 2023 00:10:57.276 * <raft> Initializing test network
2314245:M 18 Oct 2023 00:10:57.276 * <raft> Created test network client
2314245:M 18 Oct 2023 00:10:57.279 * <raft> Raft module loaded, state is 'loading'
2314245:M 18 Oct 2023 00:10:57.279 * Module 'raft' loaded from /Fuzzing/redisraft-fuzzing/redisraft.so
2314245:M 18 Oct 2023 00:10:57.279 * Server initialized
2314245:M 18 Oct 2023 00:10:57.279 . The AOF directory appendonlydir doesn't exist
2314245:M 18 Oct 2023 00:10:57.279 * Ready to accept connections tcp
2314245:M 18 Oct 2023 00:10:57.279 . 0 clients connected (0 replicas), 1746816 bytes in use
2314245:M 18 Oct 2023 00:10:57.288 - Accepted 127.0.0.1:47478
2314245:M 18 Oct 2023 00:10:57.288 # Missing implement of connection type tls
2314245:M 18 Oct 2023 00:10:57.379 * <raft> Loading: Redis loading complete, snapshot NOT LOADED
2314245:M 18 Oct 2023 00:10:57.380 * <raft> Cluster Membership: term:0 index:0 nodes: id=3,voting=0,active=1,addr=-
2314245:M 18 Oct 2023 00:10:57.380 * <raft> Test network starting message polling thread
2314245:M 18 Oct 2023 00:10:57.381 * <raft> Cluster Membership: term:0 index:4 nodes: id=3,voting=0,active=1,addr=- id=1,voting=1,active=1,addr=localhost:5091
2314245:M 18 Oct 2023 00:10:57.381 * <raft> Cluster Membership: term:0 index:4 nodes: id=3,voting=0,active=1,addr=- id=1,voting=1,active=1,addr=localhost:5091 id=2,voting=0,active=1,addr=localhost:5092
2314245:M 18 Oct 2023 00:10:57.381 * <raft> Raft state after loading log: log_count=4, first_idx=1, current_idx=4, last_applied_idx=0
2314245:M 18 Oct 2023 00:10:57.381 . <raft> Raft term=1, vote=-1
2314245:M 18 Oct 2023 00:10:57.390 - Client closed connection id=8 addr=127.0.0.1:47478 laddr=127.0.0.1:5093 fd=11 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=16384 rbp=0 obl=0 oll=0 omem=0 tot-mem=37760 events=r cmd=auth user=default redir=-1 resp=2 lib-name= lib-ver=
----- Stderr -----
We've instrumented the code to capture the messages exchanged, the calls handleTestNetworkMessage, poll_redis_test_messages are related to the instrumentation. We have been able to reliably reproduce the bug over several iterations.
The same assertion failure occurs due to a different call.
logs for node: 1
----- Stdout -----
1233342:C 18 Oct 2023 06:10:40.390 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1233342:C 18 Oct 2023 06:10:40.391 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1233342:C 18 Oct 2023 06:10:40.391 * Redis version=255.255.255, bits=64, commit=e04ebdb8, modified=1, pid=1233342, just started
1233342:C 18 Oct 2023 06:10:40.391 * Configuration loaded
1233342:M 18 Oct 2023 06:10:40.391 * monotonic clock: POSIX clock_gettime
1233342:M 18 Oct 2023 06:10:40.391 * Running mode=standalone, port=5071.
1233342:M 18 Oct 2023 06:10:40.396 * <raft> RedisRaft version 255.255.255 [c259f3d2]
1233342:M 18 Oct 2023 06:10:40.401 * <raft> Initializing test network
1233342:M 18 Oct 2023 06:10:40.401 * <raft> Created test network client
1233342:M 18 Oct 2023 06:10:40.404 * <raft> Raft module loaded, state is 'loading'
1233342:M 18 Oct 2023 06:10:40.404 * Module 'raft' loaded from /Fuzzing/redisraft-fuzzing/redisraft.so
1233342:M 18 Oct 2023 06:10:40.405 * Server initialized
1233342:M 18 Oct 2023 06:10:40.405 . The AOF directory appendonlydir doesn't exist
1233342:M 18 Oct 2023 06:10:40.405 * Ready to accept connections tcp
1233342:M 18 Oct 2023 06:10:40.405 . 0 clients connected (0 replicas), 1746720 bytes in use
1233342:M 18 Oct 2023 06:10:40.500 - Accepted 127.0.0.1:48310
1233342:M 18 Oct 2023 06:10:40.500 # Missing implement of connection type tls
1233342:M 18 Oct 2023 06:10:40.505 * <raft> Loading: Redis loading complete, snapshot NOT LOADED
1233342:M 18 Oct 2023 06:10:40.505 * <raft> Cluster Membership: term:0 index:0 nodes: id=1,voting=0,active=1,addr=-
1233342:M 18 Oct 2023 06:10:40.506 * <raft> Test network starting message polling thread
1233342:M 18 Oct 2023 06:10:40.506 * <raft> Cluster Membership: term:0 index:5 nodes: id=1,voting=1,active=1,addr=- id=2,voting=0,active=1,addr=localhost:5072
1233342:M 18 Oct 2023 06:10:40.507 * <raft> Cluster Membership: term:0 index:5 nodes: id=1,voting=1,active=1,addr=- id=2,voting=0,active=1,addr=localhost:5072 id=3,voting=0,active=1,addr=localhost:5073
1233342:M 18 Oct 2023 06:10:40.507 * <raft> Raft state after loading log: log_count=5, first_idx=1, current_idx=5, last_applied_idx=0
1233342:M 18 Oct 2023 06:10:40.507 . <raft> Raft term=1, vote=-1
----- Stderr -----
logs for node: 2
----- Stdout -----
1233155:C 18 Oct 2023 06:10:40.083 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1233155:C 18 Oct 2023 06:10:40.083 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1233155:C 18 Oct 2023 06:10:40.083 * Redis version=255.255.255, bits=64, commit=e04ebdb8, modified=1, pid=1233155, just started
1233155:C 18 Oct 2023 06:10:40.083 * Configuration loaded
1233155:M 18 Oct 2023 06:10:40.084 * monotonic clock: POSIX clock_gettime
1233155:M 18 Oct 2023 06:10:40.084 * Running mode=standalone, port=5072.
1233155:M 18 Oct 2023 06:10:40.088 * <raft> RedisRaft version 255.255.255 [c259f3d2]
1233155:M 18 Oct 2023 06:10:40.093 * <raft> Initializing test network
1233155:M 18 Oct 2023 06:10:40.094 * <raft> Created test network client
1233155:M 18 Oct 2023 06:10:40.097 * <raft> Raft module loaded, state is 'loading'
1233155:M 18 Oct 2023 06:10:40.097 * Module 'raft' loaded from /Fuzzing/redisraft-fuzzing/redisraft.so
1233155:M 18 Oct 2023 06:10:40.097 * Server initialized
1233155:M 18 Oct 2023 06:10:40.097 . The AOF directory appendonlydir doesn't exist
1233155:M 18 Oct 2023 06:10:40.097 * Ready to accept connections tcp
1233155:M 18 Oct 2023 06:10:40.097 . 0 clients connected (0 replicas), 1746688 bytes in use
1233155:M 18 Oct 2023 06:10:40.197 * <raft> Loading: Redis loading complete, snapshot NOT LOADED
1233155:M 18 Oct 2023 06:10:40.198 * <raft> Cluster Membership: term:0 index:0 nodes: id=2,voting=0,active=1,addr=-
1233155:M 18 Oct 2023 06:10:40.198 * <raft> Test network starting message polling thread
1233155:M 18 Oct 2023 06:10:40.199 * <raft> Cluster Membership: term:0 index:4 nodes: id=2,voting=0,active=1,addr=- id=1,voting=1,active=1,addr=localhost:5071
1233155:M 18 Oct 2023 06:10:40.199 * <raft> Cluster Membership: term:0 index:4 nodes: id=2,voting=0,active=1,addr=- id=1,voting=1,active=1,addr=localhost:5071 id=3,voting=0,active=1,addr=localhost:5073
1233155:M 18 Oct 2023 06:10:40.199 * <raft> Raft state after loading log: log_count=4, first_idx=1, current_idx=4, last_applied_idx=0
1233155:M 18 Oct 2023 06:10:40.199 . <raft> Raft term=1, vote=-1
1233155:M 18 Oct 2023 06:10:40.499 * <raft> State change: Election starting, node is now a pre-candidate, term 1
1233155:M 18 Oct 2023 06:10:40.499 * <raft> Cluster Membership: term:1 index:4 nodes: id=2,voting=0,active=1,addr=- id=1,voting=1,active=1,addr=localhost:5071 id=3,voting=0,active=1,addr=localhost:5073
1233155:M 18 Oct 2023 06:10:40.507 - Accepted 127.0.0.1:40056
1233155:M 18 Oct 2023 06:10:40.507 # Missing implement of connection type tls
1233155:M 18 Oct 2023 06:10:40.604 - Client closed connection id=8 addr=127.0.0.1:40056 laddr=127.0.0.1:5072 fd=13 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=16384 rbp=0 obl=0 oll=0 omem=0 tot-mem=37760 events=r cmd=auth user=default redir=-1 resp=2 lib-name= lib-ver=
----- Stderr -----
logs for node: 3
----- Stdout -----
1233378:C 18 Oct 2023 06:10:40.451 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
1233378:C 18 Oct 2023 06:10:40.451 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1233378:C 18 Oct 2023 06:10:40.451 * Redis version=255.255.255, bits=64, commit=e04ebdb8, modified=1, pid=1233378, just started
1233378:C 18 Oct 2023 06:10:40.451 * Configuration loaded
1233378:M 18 Oct 2023 06:10:40.452 * monotonic clock: POSIX clock_gettime
1233378:M 18 Oct 2023 06:10:40.452 * Running mode=standalone, port=5073.
1233378:M 18 Oct 2023 06:10:40.456 * <raft> RedisRaft version 255.255.255 [c259f3d2]
1233378:M 18 Oct 2023 06:10:40.460 * <raft> Initializing test network
1233378:M 18 Oct 2023 06:10:40.461 * <raft> Created test network client
1233378:M 18 Oct 2023 06:10:40.464 * <raft> Raft module loaded, state is 'loading'
1233378:M 18 Oct 2023 06:10:40.464 * Module 'raft' loaded from /Fuzzing/redisraft-fuzzing/redisraft.so
1233378:M 18 Oct 2023 06:10:40.465 * Server initialized
1233378:M 18 Oct 2023 06:10:40.465 . The AOF directory appendonlydir doesn't exist
1233378:M 18 Oct 2023 06:10:40.465 * Ready to accept connections tcp
1233378:M 18 Oct 2023 06:10:40.465 . 0 clients connected (0 replicas), 1746848 bytes in use
1233378:M 18 Oct 2023 06:10:40.500 - Accepted 127.0.0.1:59274
1233378:M 18 Oct 2023 06:10:40.500 # Missing implement of connection type tls
1233378:M 18 Oct 2023 06:10:40.507 - Accepted 127.0.0.1:59286
1233378:M 18 Oct 2023 06:10:40.564 * <raft> Loading: Redis loading complete, snapshot NOT LOADED
1233378:M 18 Oct 2023 06:10:40.565 * <raft> Cluster Membership: term:0 index:0 nodes: id=3,voting=0,active=1,addr=-
1233378:M 18 Oct 2023 06:10:40.565 * <raft> Test network starting message polling thread
=== REDIS BUG REPORT START: Cut & paste starting from here ===
1233378:M 18 Oct 2023 06:10:40.565 # === ASSERTION FAILED ===
1233378:M 18 Oct 2023 06:10:40.565 # ==> /Fuzzing/redisraft-fuzzing/src/log.c:509 'ety != NULL' is not true
1233378:M 18 Oct 2023 06:10:40.565 # === ASSERTION FAILED ===
1233378:M 18 Oct 2023 06:10:40.565 # ==> /Fuzzing/redisraft-fuzzing/src/log.c:509 'ety != NULL' is not true
------ STACK TRACE ------
Backtrace:
------ STACK TRACE ------
/Fuzzing/redisraft-fuzzing/redisraft.so(+0x25e80)[0x7fd28576ee80]
Backtrace:
/Fuzzing/redisraft-fuzzing/redisraft.so(+0x27208)[0x7fd285770208]
/Fuzzing/redisraft-fuzzing/redisraft.so(+0x27347)[0x7fd285770347]
/Fuzzing/redisraft-fuzzing/redisraft.so(raft_get_entry_from_idx+0x1a)[0x7fd28578454a]
/Fuzzing/redisraft-fuzzing/redisraft.so(+0x25e80)[0x7fd28576ee80]
/Fuzzing/redisraft-fuzzing/redisraft.so(raft_get_last_log_term+0x36)[0x7fd28578a576]
/Fuzzing/redisraft-fuzzing/redisraft.so(+0x27208)[0x7fd285770208]
/Fuzzing/redisraft-fuzzing/redisraft.so(raft_recv_requestvote+0x25b)[0x7fd28578529b]
/Fuzzing/redisraft-fuzzing/redisraft.so(handleTestNetworkMessage+0x204)[0x7fd28577e594]
/Fuzzing/redisraft-fuzzing/redisraft.so(poll_redis_test_messages+0x89)[0x7fd28577e659]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x7ea7)[0x7fd2859b2ea7]
/Fuzzing/redisraft-fuzzing/redisraft.so(+0x27347)[0x7fd285770347]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x7fd2858d2a2f]
------ INFO OUTPUT ------
/Fuzzing/redisraft-fuzzing/redisraft.so(raft_restore_log+0x54)[0x7fd285789af4]
/Fuzzing/redisraft-fuzzing/redisraft.so(callRaftPeriodic+0x131)[0x7fd285775e71]
/Fuzzing/redis/src/redis-server 0.0.0.0:5073(moduleTimerHandler+0x21c)[0x561dc320398c]
/Fuzzing/redis/src/redis-server 0.0.0.0:5073(+0x81705)[0x561dc310d705]
/Fuzzing/redis/src/redis-server 0.0.0.0:5073(aeMain+0x1c8)[0x561dc3117cb8]
/Fuzzing/redis/src/redis-server 0.0.0.0:5073(main+0x3cd)[0x561dc310ce2d]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea)[0x7fd2857fad0a]
# Server
redis_version:255.255.255
redis_git_sha1:e04ebdb8
redis_git_dirty:1
redis_build_id:1f78dcc7b4fbdd25
redis_mode:standalone
os:Linux 5.15.111.1.amd64-smp x86_64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:10.2.1
process_id:1233378
process_supervised:no
run_id:caed6a64524ebb992655d104d014593dcfd22b48
tcp_port:5073
server_time_usec:1697602240564640
uptime_in_seconds:0
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:3103424
executable:/Fuzzing/redis/src/redis-server
config_file:
io_threads_active:0
listener0:name=tcp,bind=0.0.0.0,port=5073
# Clients
connected_clients:2
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:0
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0
total_blocking_keys:0
total_blocking_keys_on_nokey:0
# Memory
used_memory:1855400
used_memory_human:1.77M
used_memory_rss:15405056
used_memory_rss_human:14.69M
used_memory_peak:1855400
used_memory_peak_human:1.77M
used_memory_peak_perc:100.38%
used_memory_overhead:1746888
used_memory_startup:1746704
used_memory_dataset:108512
used_memory_dataset_perc:99.83%
allocator_allocated:2060480
allocator_active:2613248
allocator_resident:10887168
total_system_memory:270386737152
total_system_memory_human:251.82G
used_memory_lua:31744
used_memory_vm_eval:31744
used_memory_lua_human:31.00K
used_memory_scripts_eval:0
number_of_cached_scripts:0
number_of_functions:0
number_of_libraries:0
used_memory_vm_functions:32768
used_memory_vm_total:64512
used_memory_vm_total_human:63.00K
used_memory_functions:184
used_memory_scripts:184
used_memory_scripts_human:184B
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.27
allocator_frag_bytes:552768
allocator_rss_ratio:4.17
allocator_rss_bytes:8273920
rss_overhead_ratio:1.41
rss_overhead_bytes:4517888
mem_fragmentation_ratio:8.82
mem_fragmentation_bytes:13658208
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_total_replication_buffers:0
mem_clients_slaves:0
mem_clients_normal:0
mem_cluster_links:0
mem_aof_buffer:0
mem_allocator:jemalloc-5.3.0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0
# Persistence
loading:0
async_loading:0
current_cow_peak:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1697602240
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_saves:0
rdb_last_cow_size:0
rdb_last_load_keys_expired:0
rdb_last_load_keys_loaded:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_rewrites:0
aof_rewrites_consecutive_failures:0
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0
# Stats
total_connections_received:2
total_commands_processed:8
instantaneous_ops_per_sec:28
total_net_input_bytes:66
total_net_output_bytes:10
total_net_repl_input_bytes:0
total_net_repl_output_bytes:0
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.00
instantaneous_input_repl_kbps:0.00
instantaneous_output_repl_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
evicted_clients:0
total_eviction_exceeded_time:0
current_eviction_exceeded_time:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
pubsubshard_channels:0
latest_fork_usec:0
total_forks:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
total_active_defrag_time:0
current_active_defrag_time:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:0
dump_payload_sanitizations:0
total_reads_processed:2
total_writes_processed:2
io_threaded_reads_processed:0
io_threaded_writes_processed:0
reply_buffer_shrinks:0
reply_buffer_expands:0
acl_access_denied_auth:0
acl_access_denied_cmd:0
acl_access_denied_key:0
acl_access_denied_channel:0
# Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:ccfde12aaf62118799b491c8d221a0de2b07feaa
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
# CPU
used_cpu_sys:0.012332
used_cpu_user:0.002758
used_cpu_sys_children:0.000000
used_cpu_user_children:0.000000
used_cpu_sys_main_thread:0.000000
used_cpu_user_main_thread:0.000619
# Modules
module:name=raft,ver=1,api=1,filters=1,usedby=[],using=[],options=[]
# Commandstats
cmdstat_auth:calls=2,usec=3,usec_per_call=1.50,rejected_calls=0,failed_calls=0
cmdstat_config|get:calls=3,usec=4,usec_per_call=1.33,rejected_calls=0,failed_calls=0
cmdstat_config|set:calls=1,usec=6,usec_per_call=6.00,rejected_calls=0,failed_calls=0
cmdstat_command:calls=2,usec=1431,usec_per_call=715.50,rejected_calls=0,failed_calls=0
# Errorstats
# Latencystats
latency_percentiles_usec_auth:p50=1.003,p99=2.007,p99.9=2.007
latency_percentiles_usec_config|get:p50=1.003,p99=2.007,p99.9=2.007
latency_percentiles_usec_config|set:p50=6.015,p99=6.015,p99.9=6.015
latency_percentiles_usec_command:p50=704.511,p99=729.087,p99.9=729.087
# Cluster
cluster_enabled:0
# Keyspace
------ CLIENT LIST OUTPUT ------
/Fuzzing/redis/src/redis-server 0.0.0.0:5073(_start+0x2a)[0x561dc310d5aa]
------ INFO OUTPUT ------
id=8 addr=127.0.0.1:59274 laddr=127.0.0.1:5073 fd=11 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=37760 events=r cmd=auth user=default redir=-1 resp=2 lib-name= lib-ver=
id=9 addr=127.0.0.1:59286 laddr=127.0.0.1:5073 fd=12 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 ssub=0 multi=-1 qbuf=0 qbuf-free=20474 argv-mem=0 multi-mem=0 rbs=16384 rbp=16384 obl=0 oll=0 omem=0 tot-mem=37760 events=r cmd=auth user=default redir=-1 resp=2 lib-name= lib-ver=
------ MODULES INFO OUTPUT ------
# raft_version
raft_version:255.255.255
raft_git_sha1:c259f3d2
# raft_general
raft_dbid:c83a5fe7dd6849c6e864f25fde59aace
raft_node_id:3
raft_state:up
raft_role:follower
raft_is_voting:no
raft_voted_for:-1
raft_leader_id:-1
raft_current_term:0
raft_num_nodes:1
raft_num_voting_nodes:0
# raft_log
raft_log_entries:5
raft_current_index:5
raft_commit_index:0
raft_last_applied_index:0
raft_file_size:1531
raft_cache_memory_size:0
raft_cache_entries:0
raft_client_attached_entries:0
raft_fsync_count:0
raft_fsync_max_microseconds:0
raft_fsync_avg_microseconds:0
# raft_snapshot
raft_snapshot_filename:redis3.rdb
raft_snapshot_last_idx:0
raft_snapshot_last_term:0
raft_snapshot_size:0
raft_snapshot_time_secs:-1
raft_snapshots_created:0
raft_snapshots_received:0
raft_snapshot_in_progress:no
raft_snapshot_in_progress_last_idx:-1
raft_snapshot_in_progress_last_term:-1
# raft_clients
raft_proxy_reqs:0
raft_proxy_failed_reqs:0
raft_proxy_failed_responses:0
raft_proxy_outstanding_reqs:0
# raft_stats
raft_appendreq_received:0
raft_appendreq_with_entry_received:0
raft_snapshotreq_received:0
raft_exec_throttled:0
raft_num_sessions:0
------ CONFIG DEBUG OUTPUT ------
io-threads-do-reads no
repl-diskless-load disabled
proto-max-bulk-len 512mb
client-query-buffer-limit 1gb
lazyfree-lazy-expire no
activedefrag no
list-compress-depth 0
lazyfree-lazy-server-del no
replica-read-only yes
sanitize-dump-payload no
lazyfree-lazy-user-flush no
slave-read-only yes
lazyfree-lazy-user-del no
io-threads 1
repl-diskless-sync yes
lazyfree-lazy-eviction no
------ FAST MEMORY TEST ------
# Server
redis_version:255.255.255
redis_git_sha1:e04ebdb8
redis_git_dirty:1
redis_build_id:1f78dcc7b4fbdd25
redis_mode:standalone
os:Linux 5.15.111.1.amd64-smp x86_64
arch_bits:64
monotonic_clock:POSIX clock_gettime
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:10.2.1
process_id:1233378
process_supervised:no
run_id:caed6a64524ebb992655d104d014593dcfd22b48
tcp_port:5073
server_time_usec:1697602240564640
uptime_in_seconds:0
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:3103424
executable:/Fuzzing/redis/src/redis-server
config_file:
io_threads_active:0
listener0:name=tcp,bind=0.0.0.0,port=5073
# Clients
connected_clients:2
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:0
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0
total_blocking_keys:0
total_blocking_keys_on_nokey:0
# Memory
used_memory:1868904
used_memory_human:1.78M
used_memory_rss:15405056
used_memory_rss_human:14.69M
used_memory_peak:1868904
used_memory_peak_human:1.78M
used_memory_peak_perc:100.73%
used_memory_overhead:1746888
used_memory_startup:1746704
used_memory_dataset:122016
used_memory_dataset_perc:99.85%
allocator_allocated:2060480
allocator_active:2613248
allocator_resident:10887168
total_system_memory:270386737152
total_system_memory_human:251.82G
used_memory_lua:31744
used_memory_vm_eval:31744
used_memory_lua_human:31.00K
used_memory_scripts_eval:0
number_of_cached_scripts:0
number_of_functions:0
number_of_libraries:0
used_memory_vm_functions:32768
used_memory_vm_total:64512
used_memory_vm_total_human:63.00K
used_memory_functions:184
used_memory_scripts:184
used_memory_scripts_human:184B
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.27
allocator_frag_bytes:552768
allocator_rss_ratio:4.17
allocator_rss_bytes:8273920
rss_overhead_ratio:1.41
rss_overhead_bytes:4517888
mem_fragmentation_ratio:8.82
mem_fragmentation_bytes:13658208
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_total_replication_buffers:0
mem_clients_slaves:0
mem_clients_normal:0
mem_cluster_links:0
mem_aof_buffer:0
mem_allocator:jemalloc-5.3.0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0
# Persistence
loading:0
async_loading:0
current_cow_peak:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1697602240
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_saves:0
rdb_last_cow_size:0
rdb_last_load_keys_expired:0
rdb_last_load_keys_loaded:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_rewrites:0
aof_rewrites_consecutive_failures:0
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0
# Stats
total_connections_received:2
total_commands_processed:8
instantaneous_ops_per_sec:28
total_net_input_bytes:66
total_net_output_bytes:10
total_net_repl_input_bytes:0
total_net_repl_output_bytes:0
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.00
instantaneous_input_repl_kbps:0.00
instantaneous_output_repl_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
evicted_clients:0
total_eviction_exceeded_time:0
current_eviction_exceeded_time:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
pubsubshard_channels:0
latest_fork_usec:0
total_forks:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
total_active_defrag_time:0
current_active_defrag_time:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:0
dump_payload_sanitizations:0
total_reads_processed:2
total_writes_processed:2
io_threaded_reads_processed:0
io_threaded_writes_processed:0
reply_buffer_shrinks:0
reply_buffer_expands:0
acl_access_denied_auth:0
acl_access_denied_cmd:0
acl_access_denied_key:0
acl_access_denied_channel:0
# Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:ccfde12aaf62118799b491c8d221a0de2b07feaa
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
# CPU
used_cpu_sys:0.012332
used_cpu_user:0.003153
used_cpu_sys_children:0.000000
used_cpu_user_children:0.000000
used_cpu_sys_main_thread:0.009143
used_cpu_user_main_thread:0.004571
# Modules
module:name=raft,ver=1,api=1,filters=1,usedby=[],using=[],options=[]
# Commandstats
cmdstat_auth:calls=2,usec=3,usec_per_call=1.50,rejected_calls=0,failed_calls=0
cmdstat_config|get:calls=3,usec=4,usec_per_call=1.33,rejected_calls=0,failed_calls=0
cmdstat_config|set:calls=1,usec=6,usec_per_call=6.00,rejected_calls=0,failed_calls=0
cmdstat_command:calls=2,usec=1431,usec_per_call=715.50,rejected_calls=0,failed_calls=0
# Errorstats
# Latencystats
latency_percentiles_usec_auth:p50=1.003,p99=2.007,p99.9=2.007
latency_percentiles_usec_config|get:p50=1.003,p99=2.007,p99.9=2.007
latency_percentiles_usec_config|set:p50=6.015,p99=6.015,p99.9=6.015
latency_percentiles_usec_command:p50=704.511,p99=729.087,p99.9=729.087
# Cluster
cluster_enabled:0
# Keyspace
l_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:0
dump_payload_sanitizations:0
total_reads_processed:2
total_writes_processed:2
io_threaded_reads_processed:0
io_threaded_writes_processed:0
reply_buffer_shrinks:0
reply_buffer_expands:0
acl_access_denied_auth:0
acl_access_denied_cmd:0
acl_access_denied_key:0
acl_access_denied_channel:0
# Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:ccfde12aaf62118799b491c8d221a0de2b07feaa
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
# CPU
used_cpu_sys:0.012332
used_cpu_user:0.003153
used_cpu_sys_children:0.000000
used_cpu_user_children:0.000000
used_cpu_sys_main_thread:0.009143
used_cpu_user_main_thread:0.004571
# Modules
module:name=raft,ver=1,api=1,filters=1,usedby=[],using=[],options=[]
# Commandstats
cmdstat_auth:calls=2,usec=3,usec_per_call=1.50,rejected_calls=0,failed_calls=0
cmdstat_config|get:calls=3,usec=4,usec_per_call=1.33,rejected_calls=0,failed_calls=0
cmdstat_config|set:calls=1,usec=6,usec_per_call=6.00,rejected_calls=0,failed_calls=0
cmdstat_command:calls=2,usec=1431,usec_per_call=715.50,rejected_calls=0,failed_calls=0
# Errorstats
# Latencystats
latency_percentiles_usec_auth:p50=1.003,p99=2.007,p99.9=2.007
latency_percentiles_usec_config|get:p50=1.003,p99=2.007,p99.9=2.007
latency_percentiles_usec_config|set:p50=6.015,p99=6.015,p99.9=6.015
latency_percentiles_usec_command:p50=704.511,p99=729.087,p99.9=729.087
# Cluster
cluster_enabled:0
# Keyspace
1233378:M 18 Oct 2023 06:10:40.566 # main thread terminated
1233378:M 18 Oct 2023 06:10:40.566 # Bio worker thread #0 terminated
1233378:M 18 Oct 2023 06:10:40.566 # Bio worker thread #1 terminated
1233378:M 18 Oct 2023 06:10:40.566 # Bio worker thread #2 terminated
Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.
=== REDIS BUG REPORT END. Make sure to include from START to END. ===
Please report the crash by opening an issue on github:
http://github.com/redis/redis/issues
If a Redis module was involved, please open in the module's repo instead.
Suspect RAM error? Use redis-server --test-memory to verify it.
Some other issues could be detected by redis-server --check-system
----- Stderr -----