kubeblocks icon indicating copy to clipboard operation
kubeblocks copied to clipboard
verified publisher icon apecloud

[BUG] vastbase create failed on VKE: /var/lib/vastbase/reload_config.lock: Permission denied

Open JashBook opened this issue 5 months ago • 2 comments

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error
kubectl logs vastbase-oxipoi-vastbase-0 
Defaulted container "vastbase" out of: vastbase, exporter, lorry, config-manager, init-lorry (init)
+ CONFIG_VOLUME=/etc/vastbase
+ VASTBASE_CM_SERVER_PORT=1028
+ VASTBASE_DB_PORT=1025
+ VASTBASE_DB_REPL_PORT=1026
+ VASTBASE_DB_HEARTBEAT_PORT=1027
+ db_conn_str=postgres://%2Ftmp:1025
+ trap _stop INT TERM
+ pwd
/
+ id
uid=0(root) gid=0(root) groups=0(root)
+ hostname=vastbase-oxipoi-vastbase-0
+ hostname_underscore=vastbase_oxipoi_vastbase_0
++ echo vastbase-oxipoi-vastbase-0
++ awk -F - '{print $(NF)}'
+ ordinal=0
+ export NODE_ID=1
+ NODE_ID=1
+ export DB_DIR=/var/lib/vastbase/dn
+ DB_DIR=/var/lib/vastbase/dn
+ export GAUSSLOG=/var/lib/vastbase/log
+ GAUSSLOG=/var/lib/vastbase/log
+ echo 'export GAUSSLOG="/var/lib/vastbase/log"'
++ id -u
+ [[ 0 = \0 ]]
+ echo 'create dirs'
create dirs
+ mkdir -p /var/lib/vastbase/cm/cm_server/ /var/lib/vastbase/cm/cm_agent/
+ mkdir -p /var/lib/vastbase/log/cm/cm_server/ /var/lib/vastbase/log/cm/cm_agent/
+ mkdir -p /var/lib/vastbase/dn
+ chmod -R 700 /var/lib/vastbase/dn
+ chown -R vastbase:vastbase /var/lib/vastbase/dn
+ chmod -R 700 /var/lib/vastbase/log
+ chown -R vastbase:vastbase /var/lib/vastbase/log
+ chmod -R 700 /var/lib/vastbase/cm
+ chown -R vastbase:vastbase /var/lib/vastbase/cm
+ echo 'restart script as vastbase user'
+ exec gosu vastbase /scripts/start_masterslave.sh
restart script as vastbase user
+ CONFIG_VOLUME=/etc/vastbase
+ VASTBASE_CM_SERVER_PORT=1028
+ VASTBASE_DB_PORT=1025
+ VASTBASE_DB_REPL_PORT=1026
+ VASTBASE_DB_HEARTBEAT_PORT=1027
+ db_conn_str=postgres://%2Ftmp:1025
+ trap _stop INT TERM
+ pwd
+ id
/
uid=1000(vastbase) gid=1000(vastbase) groups=1000(vastbase)
+ hostname=vastbase-oxipoi-vastbase-0
+ hostname_underscore=vastbase_oxipoi_vastbase_0
++ echo vastbase-oxipoi-vastbase-0
++ awk -F - '{print $(NF)}'
+ ordinal=0
+ export NODE_ID=1
+ NODE_ID=1
+ export DB_DIR=/var/lib/vastbase/dn
+ DB_DIR=/var/lib/vastbase/dn
+ export GAUSSLOG=/var/lib/vastbase/log
+ GAUSSLOG=/var/lib/vastbase/log
+ echo 'export GAUSSLOG="/var/lib/vastbase/log"'
++ id -u
+ [[ 1000 = \0 ]]
++ ls -A /var/lib/vastbase/dn
data dir empty, initializing
+ '[' -z '' ']'
+ echo 'data dir empty, initializing'
+ vb_initdb -D /var/lib/vastbase/dn --nodename vastbase_oxipoi_vastbase_0 -w bu70gWf82d7h4B60 --dbcompatibility=PG
The files belonging to this database system will be owned by user "vastbase".
This user must also own the server process.

The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".

fixing permissions on existing directory /var/lib/vastbase/dn ... ok
creating subdirectories ... in ordinary occasionok
creating configuration files ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 1024MB
Begin init undo subsystem meta.
[INIT UNDO] Init undo subsystem meta successfully.
creating template1 database in /var/lib/vastbase/dn/base/1 ... The core dump path is an invalid directory
2025-07-23 09:06:11.097 [unknown] [unknown] localhost 140115284538112 0[0:0#0]  [BACKEND] WARNING:  macAddr is 22/1044955607, sysidentifier is 1457736/3252132064, randomNum is 444242144
ok
initializing pg_authid ... ok
setting password ... 
system admin[vbadmin] initial password: Cee6b(e6
security admin[vbsso] initial password: Y4ee86@a
audit admin[vbaudit] initial password: Aab&2ea0
Please change your initial password immediately after your first successful login.
ok
initializing dependencies ... ok
loading PL/pgSQL server-side language ... ok
creating system views ... ok
creating performance views ... ok
loading system objects' descriptions ... ok
creating postgresql object ... ok
creating collations ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
initialize global configure for bucketmap length ... ok
creating information schema ... ok
loading security plugin ... ok
loading pg_zhtrgm extension ... ok
update system tables ... ok
creating snapshots catalog ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to vastbase ... ok
copying template1 to postgres ... ok
freezing database template0 ... ok
freezing database template1 ... ok
freezing database postgres ... ok

WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run vb_initdb.

Success. You can now start the database server of single node using:

    vastbase -D /var/lib/vastbase/dn
or
    vb_ctl start -D /var/lib/vastbase/dn -Z single_node -l logfile

+ cp -f /etc/vastbase/postgresql.conf /var/lib/vastbase/dn/postgresql.conf
+ gs_guc set -D /var/lib/vastbase/dn -c 'license_path='\''/license/vb.key'\'''
The vb_guc run with the following arguments: [gs_guc -D /var/lib/vastbase/dn -c license_path='/license/vb.key' set ].
expected instance path: [/var/lib/vastbase/dn/postgresql.conf]
gs_guc set: license_path='/license/vb.key': [/var/lib/vastbase/dn/postgresql.conf]

Total instances: 1. Failed instances: 0.
Success to perform gs_guc!

+ gs_guc set -D /var/lib/vastbase/dn -c port=1025
The vb_guc run with the following arguments: [gs_guc -D /var/lib/vastbase/dn -c port=1025 set ].
expected instance path: [/var/lib/vastbase/dn/postgresql.conf]
gs_guc set: port=1025: [/var/lib/vastbase/dn/postgresql.conf]

Total instances: 1. Failed instances: 0.
Success to perform gs_guc!

+ gs_guc set -D /var/lib/vastbase/dn -h 'host replication root 0.0.0.0/0 sha256'
realpath(/home/vastbase/app/bin/cluster_static_config) failed : No such file or directory!
The vb_guc run with the following arguments: [gs_guc -D /var/lib/vastbase/dn -h host replication root 0.0.0.0/0 sha256 set ].
expected instance path: [/var/lib/vastbase/dn/pg_hba.conf]
realpath(/var/lib/vastbase/dn/pg_hba.conf.lock) failed : No such file or directory!
gs_guc sethba: host replication root 0.0.0.0/0 sha256: [/var/lib/vastbase/dn/pg_hba.conf]

Total instances: 1. Failed instances: 0.
Success to perform gs_guc!

+ gs_guc set -D /var/lib/vastbase/dn -h 'host all all 0.0.0.0/0 sha256'
The vb_guc run with the following arguments: [gs_guc -D /var/lib/vastbase/dn -h host all all 0.0.0.0/0 sha256 set ].
realpath(/home/vastbase/app/bin/cluster_static_config) failed : No such file or directory!
expected instance path: [/var/lib/vastbase/dn/pg_hba.conf]
gs_guc sethba: host all all 0.0.0.0/0 sha256: [/var/lib/vastbase/dn/pg_hba.conf]

Total instances: 1. Failed instances: 0.
Success to perform gs_guc!

+ '[' '!' -f /var/lib/vastbase/cm/cm_server/cm_server.conf ']'
+ echo 'initialze cm config'
initialze cm config
+ cp /home/vastbase/app/share/config/cm_server.conf.sample /var/lib/vastbase/cm/cm_server/cm_server.conf
+ cp /home/vastbase/app/share/config/cm_agent.conf.sample /var/lib/vastbase/cm/cm_agent/cm_agent.conf
+ retry call_reload_config
+ vb_agent
+ command=call_reload_config
+ true
+ call_reload_config
++ echo vastbase-oxipoi-vastbase-0.vastbase-oxipoi-vastbase-headless.default.svc.cluster.local,vastbase-oxipoi-vastbase-1.vastbase-oxipoi-vastbase-headless.default.svc.cluster.local,vastbase-oxipoi-vastbase-2.vastbase-oxipoi-vastbase-headless.default.svc.cluster.local
++ tr , '\n'
+ pod_fqdns='vastbase-oxipoi-vastbase-0.vastbase-oxipoi-vastbase-headless.default.svc.cluster.local
vastbase-oxipoi-vastbase-1.vastbase-oxipoi-vastbase-headless.default.svc.cluster.local
vastbase-oxipoi-vastbase-2.vastbase-oxipoi-vastbase-headless.default.svc.cluster.local'
+ for pod_fqdn in '${pod_fqdns}'
+ url=http://vastbase-oxipoi-vastbase-0.vastbase-oxipoi-vastbase-headless.default.svc.cluster.local:1031/reloadConfig
++ curl -X POST http://vastbase-oxipoi-vastbase-0.vastbase-oxipoi-vastbase-headless.default.svc.cluster.local:1031/reloadConfig --connect-timeout 10 --retry 3 --retry-delay 10 -H 'Content-Type: application/json' -d '{"podFqdns": "vastbase-oxipoi-vastbase-0.vastbase-oxipoi-vastbase-headless.default.svc.cluster.local,vastbase-oxipoi-vastbase-1.vastbase-oxipoi-vastbase-headless.default.svc.cluster.local,vastbase-oxipoi-vastbase-2.vastbase-oxipoi-vastbase-headless.default.svc.cluster.local"}'
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   432  100   156  100   276  67154   116k --:--:-- --:--:-- --:--:--  210k
+ result='{"goErr":"exit status 1","stdout":"","stderr":"/scripts/reload_config.sh: line 98: /var/lib/vastbase/reload_config.lock: Permission denied\n","exitCode":1}'
+ echo '{"goErr":"exit status 1","stdout":"","stderr":"/scripts/reload_config.sh: line 98: /var/lib/vastbase/reload_config.lock: Permission denied\n","exitCode":1}'
+ jq -e .goErr
"exit status 1"
++ echo '{"goErr":"exit status 1","stdout":"","stderr":"/scripts/reload_config.sh: line 98: /var/lib/vastbase/reload_config.lock: Permission denied\n","exitCode":1}'
++ jq --raw-output .goErr
+ [[ exit status 1 != '' ]]
call reload config failed
+ echo 'call reload config failed'
+ return 1
+ exit_code=1
+ echo 'command '\''call_reload_config'\'' failed with exit code 1. retrying in 10s'
+ sleep 20
command 'call_reload_config' failed with exit code 1. retrying in 10s

Expected behavior A clear and concise description of what you expected to happen.

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Additional context Add any other context about the problem here.

JashBook avatar Jul 23 '25 09:07 JashBook

This issue has been marked as stale because it has been open for 30 days with no activity

github-actions[bot] avatar Aug 25 '25 00:08 github-actions[bot]

This issue has been marked as stale because it has been open for 30 days with no activity

github-actions[bot] avatar Nov 10 '25 00:11 github-actions[bot]

fixed via https://github.com/apecloud/apecloud-addons/commit/0434266e79d1712df24c041f7625994966c9745b

JashBook avatar Nov 27 '25 09:11 JashBook