Dragonfly missing reported information in bull-board

[ThatOneCalculator]

Describe the bug A clear and concise description of what the bug is.

Probably related to #782 , in bull-board, Dragonfly does not report the OS or fragmentation ratio properly. It also seems to be misreporting memory usage information, as it reports ~7MB, when in reality it seems to be using anywhere from 200-250MB.

To Reproduce Steps to reproduce the behavior:

1. Launch Dragonfly with ./dragonfly --default_lua_flags='allow-undeclared-keys disable-atomicity' --alsologtostderr
2. Go to bull board
3. View results

Expected behavior OS is reported and memory is correctly reported. Reports only ~7MB used despite actually using ~200-250MB.

Screenshots

Environment (please complete the following information):

• OS: Ubuntu 22.04.2 LTS x86_64
• Kernel: 5.15.0-53-generic
• RAM: 15.6GB total, 12GB used by Dragonfly
• Containerized?: Bare Metal, built from source
• Dragonfly Version: Commit e3086f56a106ef4d7d2da4cc3748127a93def69e Output of dragonfly --version:

dragonfly dev-0000000
build time: bigbang

• Bull board version: 5.2.0
• Bull version: 4.10.4

Additional context Add any other context about the problem here.

Using Dragonfly with Calckey

[royjacobson]

Could you kindly add the way you use to measure the 200MB-250MB of memory usage and the output of the INFO command on you instance?

[ThatOneCalculator]

I got the info by looking at top and dividing the percentage by the amount of total ram. Here's INFO:

127.0.0.1:6379> INFO
# Server
redis_version:6.2.11
dfly_version:df-dev
redis_mode:standalone
arch_bits:64
multiplexing_api:iouring
tcp_port:6379
uptime_in_seconds:1012
uptime_in_days:0

# Clients
connected_clients:234
client_read_buf_capacity:1353216
blocked_clients:64

# Memory
used_memory:6886880
used_memory_human:6.57MiB
used_memory_peak:7471824
comitted_memory:304283648
used_memory_rss:250556416
used_memory_rss_human:238.95MiB
object_used_memory:328281
table_used_memory:4228560
num_buckets:100800
num_entries:5946
inline_keys:0
strval_bytes:826496
updateval_amount:34392245291
listpack_blobs:3477
listpack_bytes:104292
small_string_bytes:335728
pipeline_cache_bytes:48660
maxmemory:13420671795
maxmemory_human:12.50GiB
cache_mode:store
maxmemory_policy:noeviction

# Stats
total_connections_received:234
total_commands_processed:4696311
instantaneous_ops_per_sec:3022
total_pipelined_commands:1107668
total_net_input_bytes:732704294
total_net_output_bytes:452208940
instantaneous_input_kbps:-1
instantaneous_output_kbps:-1
rejected_connections:-1
expired_keys:128154
evicted_keys:0
hard_evictions:0
garbage_checked:0
garbage_collected:0
bump_ups:0
stash_unloaded:0
oom_rejections:0
traverse_ttl_sec:2
delete_ttl_sec:2
keyspace_hits:3021719
keyspace_misses:413154
total_reads_processed:1493473
total_writes_processed:2201864
async_writes_count:1152490
parser_err_count:0
defrag_attempt_total:0
defrag_realloc_total:0
defrag_task_invocation_total:0

# Replication
role:master
connected_slaves:0
master_replid:34324d0c1cb7b90ffa93e5a8b9930222503a4cc6

# Keyspace
db0:keys=5946,expires=1350,avg_ttl=-1

# CPU
used_cpu_sys:838.24205
used_cpu_user:633.940812
used_cpu_sys_children:0.0
used_cpu_user_children:0.0
used_cpu_sys_main_thread:55.786214
used_cpu_user_main_thread:41.118513

# Cluster
cluster_enabled:0
127.0.0.1:6379> 

[ThatOneCalculator]

# Memory
used_memory:6886880
used_memory_human:6.57MiB
used_memory_peak:7471824
comitted_memory:304283648
used_memory_rss:250556416
used_memory_rss_human:238.95MiB

Interesting... what's the difference between used_memory_human and used_memory_rss?

[royjacobson]

used_memory is (should be?) the memory used for the data in the database, but dragonfly itself has additional constant overhead and per-thread overhead which probably explains the difference here.

[chakaz]

used_memory_rss is the usage observed by the operating system. It includes stuff we don't take into account, like the executable's memory (data, instructions, etc) and other stuff that we don't account for (like ephemeral memory needed to serve requests, that is not used in the "database itself"). In the case of a (nearly) empty database, such as the one showing here, the gap is bigger because we essentially don't count anything, but the diff should be insignificant for real world usage where the DB is full.

[ThatOneCalculator]

Got it. It is real world usage (running in prod with many users), but it just gets cleared because it's being used as a queue rather than a persistent store.

[royjacobson]

Thanks! I'm closing this issue, please comment if you have further questions.

[ThatOneCalculator]

@royjacobson please reopen this issue, as memory wasn't the only problem. As per the issue and screenshot, Dragonfly does not report the OS or fragmentation ratio at all.

[royjacobson]

ah, sorry. you're right :)