Ceph nano crashes when downloading large file

[nduytg]

Hi guys,

I'm encountering an issue. Whenever I download large file (in my case, an 2.5GB video), ceph nano will crash.

I'm using huge flavor now and It works well with small files and text files

How to reproduce this

Start Ceph nano

./cn cluster start my-first-cluster -f huge

Use mc client to set the bucket publicly accessible

mc policy set public mybucket/mybucket

Then download the files from browser directly => Ceph nano crashes

Docker logs

++/opt/ceph-container/bin/docker_exec.sh:149: exec(): ceph --cluster ceph -w
 * Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
  cluster:
    id:     6ddca59e-d870-4dea-8260-3530542530d5
    health: HEALTH_OK
 
  services:
    mon: 1 daemons, quorum ceph-nano-my-first-cluster-faa32aebf00b (age 3s)
    mgr: ceph-nano-my-first-cluster-faa32aebf00b(active, since 5m)
    osd: 1 osds: 1 up (since 5m), 1 in (since 4d)
    rgw: 1 daemon active (ceph-nano-my-first-cluster-faa32aebf00b)
 
  task status:
 
  data:
    pools:   9 pools, 209 pgs
    objects: 930 objects, 2.5 GiB
    usage:   3.6 GiB used, 96 GiB / 100 GiB avail
    pgs:     209 active+clean
 
  io:
    client:   26 MiB/s rd, 6 op/s rd, 0 op/s wr
 

2020-06-15T07:52:37.030096+0000 mon.ceph-nano-my-first-cluster-faa32aebf00b [INF] Active manager daemon ceph-nano-my-first-cluster-faa32aebf00b restarted
2020-06-15T07:52:37.042020+0000 mon.ceph-nano-my-first-cluster-faa32aebf00b [INF] Activating manager daemon ceph-nano-my-first-cluster-faa32aebf00b
2020-06-15T07:52:37.123684+0000 mon.ceph-nano-my-first-cluster-faa32aebf00b [INF] Manager daemon ceph-nano-my-first-cluster-faa32aebf00b is now available
+++/opt/ceph-container/bin/docker_exec.sh:1: exec(): _chld
+++/opt/ceph-container/bin/docker_exec.sh:117: _chld(): teardown SIGCHLD 0
+++/opt/ceph-container/bin/docker_exec.sh:34: teardown(): trap - SIGINT SIGILL SIGABRT SIGFPE SIGSEGV SIGTERM SIGBUS SIGCHLD SIGKILL ERR
+++/opt/ceph-container/bin/docker_exec.sh:39: teardown(): local signal_name=SIGCHLD
+++/opt/ceph-container/bin/docker_exec.sh:40: teardown(): local exit_code=0
+++/opt/ceph-container/bin/docker_exec.sh:41: teardown(): echo 'teardown: managing teardown after SIGCHLD'
teardown: managing teardown after SIGCHLD
+++/opt/ceph-container/bin/docker_exec.sh:45: teardown(): '[' SIGCHLD = SIGTERM ']'
+++/opt/ceph-container/bin/docker_exec.sh:52: teardown(): '[' -e /proc/395 ']'
+++/opt/ceph-container/bin/docker_exec.sh:52: teardown(): echo 'teardown: Sending SIGTERM to PID 395'
+++/opt/ceph-container/bin/docker_exec.sh:52: teardown(): kill -TERM 395
teardown: Sending SIGTERM to PID 395
/opt/ceph-container/bin/docker_exec.sh: line 32: warning: run_pending_traps: bad value in trap_list[17]: 0x55f1c1e917b0
+++/opt/ceph-container/bin/docker_exec.sh:54: teardown(): echo -n 'teardown: Waiting PID 395 to terminate '
+++/opt/ceph-container/bin/docker_exec.sh:55: teardown(): local MAX_TIMEOUT=50
+++/opt/ceph-container/bin/docker_exec.sh:56: teardown(): local timeout=0
+++/opt/ceph-container/bin/docker_exec.sh:57: teardown(): '[' -e /proc/395 ']'
+++/opt/ceph-container/bin/docker_exec.sh:68: teardown(): echo
+++/opt/ceph-container/bin/docker_exec.sh:69: teardown(): echo 'teardown: Process 395 is terminated'
+++/opt/ceph-container/bin/docker_exec.sh:71: teardown(): [[ SIGCHLD =~ SIGTERM|SIGCHLD ]]
teardown: Waiting PID 395 to terminate 
teardown: Process 395 is terminated
+++/opt/ceph-container/bin/docker_exec.sh:73: teardown(): declare -F sigterm_cleanup_post
+++/opt/ceph-container/bin/docker_exec.sh:79: teardown(): do_stayalive
+++/opt/ceph-container/bin/docker_exec.sh:7: do_stayalive(): '[' -z '' ']'
+++/opt/ceph-container/bin/docker_exec.sh:8: do_stayalive(): return
teardown: Bye Bye, container will die with return code 0
+++/opt/ceph-container/bin/docker_exec.sh:81: teardown(): echo 'teardown: Bye Bye, container will die with return code 0'
+++/opt/ceph-container/bin/docker_exec.sh:82: teardown(): [[ 0 -ne 0 ]]
+++/opt/ceph-container/bin/docker_exec.sh:85: teardown(): exit 0

[nduytg]

I tested again with 600mb file, crashed half-way too