Suspend mode connect: connection refused

[javieramirez1]

Environment info

• NooBaa Version: VERSION c83f2-dan8-hs200.test.net: noobaa-core-5.17.0-20241016.el9.x86_64 c83f2-dan10-hs200.test.net: noobaa-core-5.17.0-20241016.el9.x86_64

Actual behavior

1.The following workload was executed with a bucket previously put in suspend mode (the errors are also seen when executing warp stat and warp list) warp get --host=172.20.100.6{0...9}:6443 --access-key="$access_key" --secret-key="$secret_key" --obj.size=1k --concurrent=1000 --duration=30m --bucket=bucket1$j --insecure --tls

arp: <ERROR> download error:Get "https://172.20.100.66:6443/bucket11/RS12F%28xN/3.VOaX3zJIp805QLN%29.rnd": dial tcp 172.20.100.66:6443: connect: connection refused
warp: <ERROR> download error:Get "https://172.20.100.67:6443/bucket11/AE5otFFM/1.6WqmwEJd0re19jG6.rnd": dial tcp 172.20.100.67:6443: connect: connection refused
warp: <ERROR> download error:Get "https://172.20.100.67:6443/bucket11/UgiXqpIj/5.Wct8ORJhXx5iyO20.rnd": dial tcp 172.20.100.67:6443: connect: connection refused
warp: <ERROR> download error:Get "https://172.20.100.64:6443/bucket11/YzjuGWbj/1.gHPA46P9I3pKkUO0.rnd": dial tcp 172.20.100.64:6443: connect: connection refused
----------------------------------------
Operation: PUT. Concurrency: 1000
* Average:0.09 MiB/s, 93.16 obj/s

Throughput by host:
 * https://172.20.100.60:6443: Avg: 0.01 MiB/s, 10.09 obj/s
 * https://172.20.100.61:6443: Avg: 0.01 MiB/s, 9.58 obj/s
 * https://172.20.100.62:6443: Avg: 0.01 MiB/s, 9.50 obj/s
 * https://172.20.100.63:6443: Avg: 0.01 MiB/s, 9.58 obj/s
 * https://172.20.100.64:6443: Avg: 0.01 MiB/s, 9.07 obj/s
 * https://172.20.100.65:6443: Avg: 0.01 MiB/s, 9.46 obj/s
 * https://172.20.100.66:6443: Avg: 0.01 MiB/s, 8.18 obj/s
 * https://172.20.100.67:6443: Avg: 0.01 MiB/s, 8.48 obj/s
 * https://172.20.100.68:6443: Avg: 0.01 MiB/s, 9.93 obj/s
 * https://172.20.100.69:6443: Avg: 0.01 MiB/s, 8.69 obj/s

Throughput, split into 22 x 1s:
 * Fastest:706.5KiB/s, 724.46 obj/s
 * 50% Median:46.4KiB/s, 48.00 obj/s
 * Slowest:46.2KiB/s, 47.88 obj/s

----------------------------------------
Operation: GET. Concurrency: 1000
Errors:614
* Average:5.97 MiB/s, 6265.07 obj/s

Throughput by host:
 * https://172.20.100.60:6443: Avg: 0.59 MiB/s, 622.45 obj/s
 * https://172.20.100.61:6443: Avg: 0.60 MiB/s, 632.92 obj/s, 125 errors
Errors:125
 * https://172.20.100.62:6443: Avg: 0.59 MiB/s, 620.88 obj/s
 * https://172.20.100.63:6443: Avg: 0.60 MiB/s, 630.26 obj/s
 * https://172.20.100.64:6443: Avg: 0.60 MiB/s, 627.16 obj/s, 119 errors
Errors:119
 * https://172.20.100.65:6443: Avg: 0.60 MiB/s, 626.47 obj/s
 * https://172.20.100.66:6443: Avg: 0.59 MiB/s, 622.15 obj/s, 128 errors
Errors:128
 * https://172.20.100.67:6443: Avg: 0.59 MiB/s, 618.20 obj/s, 121 errors
Errors:121
 * https://172.20.100.68:6443: Avg: 0.60 MiB/s, 632.47 obj/s
 * https://172.20.100.69:6443: Avg: 0.60 MiB/s, 632.12 obj/s, 121 errors
Errors:121

Throughput, split into 359 x 5s:
 * Fastest:7.3MiB/s, 7702.89 obj/s
 * 50% Median:6.0MiB/s, 6314.32 obj/s
 * Slowest:0.00 obj/s

Expected behavior

1.workload completed without problems

Steps to reproduce

1.warp get --host=172.20.100.6{0...9}:6443 --access-key="$access_key" --secret-key="$secret_key" --obj.size=1k --concurrent=1000 --duration=30m --bucket=bucket1$j --insecure --tls

More information - Screenshots / Logs / Other output

cluster time at which the workload started Thu Oct 17 01:54:35 AM EDT 2024 the failures were seen after 19 minutes of execution The log is too large, so I'll add it to the Slack channel.