gemini icon indicating copy to clipboard operation
gemini copied to clipboard

Could not initialize seastar when starting scylla 5.2.9

Open qiuqiu-007 opened this issue 1 year ago • 12 comments

It occured when I do gemini test at the beginning of starting scylla image with the version of 5.2.9. Msg is Could not initialize seastar: std::system_error (error system:1, Operation not permitted), all detatiled msg is attached in file

qiuqiu-007 avatar Dec 13 '23 01:12 qiuqiu-007

It occured when I do gemini test at the beginning of starting scylla image with the version of 5.2.9. Msg is Could not initialize seastar: std::system_error (error system:1, Operation not permitted), all detatiled msg is attached in file

log.txt

qiuqiu-007 avatar Dec 13 '23 01:12 qiuqiu-007

Can you provide more information on how you are trying to run it? OS, kernel, container/VM, versions?

mykaul avatar Dec 13 '23 08:12 mykaul

Can you provide more information on how you are trying to run it? OS, kernel, container/VM, versions?

  • I was trying to run gemini test, and i just run ./script/test.sh scylla
  • Env: CentOS 7
  • gemini : https://github.com/scylladb/gemini/tree/master/scripts
  • I just modified the version of scylla image from 5.2.2 to 5.2.9
  • `[root@centos02 gemini]# cat scripts/docker-compose-scylla.yml version: '3.5'

networks: gemini: name: gemini driver: bridge ipam: driver: default config: - subnet: 192.168.100.0/24

services: gemini-oracle: image: scylladb/scylla:5.2.9 container_name: gemini-oracle command: --smp 2 --memory 512M --api-address 0.0.0.0 networks: gemini:

gemini-test: image: scylladb/scylla:5.2.9 container_name: gemini-test command: --smp 2 --memory 512M --api-address 0.0.0.0 networks: gemini: `

qiuqiu-007 avatar Dec 13 '23 09:12 qiuqiu-007

I think we can move the discussion to Gemini repo, no? (I can move the issue, I believe)

mykaul avatar Dec 13 '23 09:12 mykaul

  • Env: CentOS 7

This is a bit of an old environment and kernel (and Docker...)

mykaul avatar Dec 13 '23 09:12 mykaul

  • Env: CentOS 7

This is a bit of an old environment and kernel (and Docker...)

Is it the potential cause of this issue ?

qiuqiu-007 avatar Dec 13 '23 09:12 qiuqiu-007

I think we can move the discussion to Gemini repo, no? (I can move the issue, I believe)

I agree.

qiuqiu-007 avatar Dec 13 '23 09:12 qiuqiu-007

@qiuqiu-007

2023-12-13T01:19:33.687533571Z WARN  2023-12-13 01:19:33,684 seastar - Requested AIO slots too large, please increase request capacity in /proc/sys/fs/aio-max-nr. available:65536 requested:102052
2023-12-13T01:19:33.687550162Z WARN  2023-12-13 01:19:33,684 seastar - max-networking-io-control-blocks adjusted from 50000 to 31742, since AIO slots are unavailable
2023-12-13T01:19:33.687554111Z INFO  2023-12-13 01:19:33,684 seastar - Reactor backend: epoll

this should fix it:

echo "fs.aio-max-nr = 1048576" >> /etc/sysctl.conf

fruch avatar Dec 13 '23 09:12 fruch

@fruch I tried, but it doesn't work. log2.txt

qiuqiu-007 avatar Dec 13 '23 10:12 qiuqiu-007

@fruch I tried, but it doesn't work. log2.txt

forgot the actual update, that would get it updated:

sysctl -p 

fruch avatar Dec 13 '23 10:12 fruch

@fruch I tried, but it doesn't work. log2.txt

forgot the actual update, that would get it updated:

sysctl -p 

The aio fixed. But others. log3.txt

qiuqiu-007 avatar Dec 14 '23 01:12 qiuqiu-007

I guess the issue is:

2023-12-14T01:15:35.620341245Z [0.013s][warning][os,thread] Failed to start thread "VM Thread" - pthread_create failed (EPERM) for attributes: stacksize: 1024k, guardsize: 4k, detached.
2023-12-14T01:15:35.620368712Z Error occurred during initialization of VM

I wonder if it happens with newer kernels.

mykaul avatar Dec 22 '23 11:12 mykaul