emqx-rel icon indicating copy to clipboard operation
emqx-rel copied to clipboard

Published 20 hours ago • verified publisher icon emqx

helm install emqx Custom EMQX_LOADED_PLUGINS

Open liuxingjun opened this issue 2 years ago • 13 comments

Environment

  • EMQ: 4.3.5

Description

###Repetition steps

# helm3 search repo emqx
NAME            CHART VERSION   APP VERSION     DESCRIPTION                             
emqx/emqx       4.3.5           4.3.5           A Helm chart for EMQ X                  
emqx/emqx-ee    4.2.6           4.2.6           A Helm chart for EMQ X Enterprise       
emqx/kuiper     0.9.0           0.9.0           A lightweight IoT edge analytic software

add env EMQX_LOADED_PLUGINS to values.yaml

emqxConfig:
  EMQX_CLUSTER__K8S__APISERVER: "https://kubernetes.default.svc:443"
  ## The address type is used to extract host from k8s service.
  ## Value: ip | dns | hostname
  ## Note：Hostname is only supported after v4.0-rc.2
  EMQX_CLUSTER__K8S__ADDRESS_TYPE: "hostname"
  EMQX_CLUSTER__K8S__SUFFIX: "svc.cluster.local"
  ## if EMQX_CLUSTER__K8S__ADDRESS_TYPE eq dns
  # EMQX_CLUSTER__K8S__SUFFIX: "pod.cluster.local"
  EMQX_LOADED_PLUGINS: "emqx_dashboard"

run helm3 install connliu -n connliu emqx/emqx -f values.yaml

# kubectl get pod -n connliu
NAME             READY   STATUS             RESTARTS   AGE
connliu-emqx-0   0/1     CrashLoopBackOff   1          8s

# kubectl logs -n connliu  connliu-emqx-0 emqx 
EMQX_LOADED_PLUGINS=emqx_dashboard
/usr/bin/docker-entrypoint.sh: line 101: /opt/emqx/data/loaded_plugins: Read-only file system

Normal helm install

# helm3 install  connliu -n connliu  emqx/emqx
NAME: connliu
LAST DEPLOYED: Fri Jul 16 11:39:49 2021
NAMESPACE: connliu
STATUS: deployed
REVISION: 1
TEST SUITE: None

# kubectl get pod -n connliu 
NAME             READY   STATUS    RESTARTS   AGE
connliu-emqx-0   1/1     Running   0          13s

# kubectl exec -it -n connliu connliu-emqx-0 -- ls -l /opt/emqx/data/
total 24
drwxrwxr-x    2 emqx     emqx          4096 Jul 16 03:39 configs
-rw-r--r--    1 root     emqx           177 Jul 16 03:39 loaded_modules
-rw-r--r--    1 root     emqx           169 Jul 16 03:39 loaded_plugins
drwxrwsrwx    3 root     emqx          4096 Jul 16 03:39 mnesia
drwxrwxr-x    2 emqx     emqx          4096 Jun 28 06:57 patches
drwxrwxr-x    2 emqx     emqx          4096 Jun 28 06:57 scripts

The permissions of directories loaded_modules and loaded_plugins are different from others

Custom EMQX_LOADED_PLUGINS docker install

# docker run -d --name emqx -p 18083:18083 -e EMQX_LOADED_PLUGINS="emqx_dashboard" emqx/emqx:4.3.5
189f7061d4e8ca2a96211baf3930ff0770e5f6deb3c264f3405287b59ac4e20d

# docker exec emqx ls -l /opt/emqx/data/
total 24
drwxrwxr-x    2 emqx     emqx          4096 Jul 16 03:34 configs
-rw-rw-r--    1 emqx     emqx            58 Jun 28 06:54 loaded_modules
-rw-rw-r--    1 emqx     emqx           192 Jul 16 03:34 loaded_plugins
drwxrwxr-x    3 emqx     emqx          4096 Jul 16 03:34 mnesia
drwxrwxr-x    2 emqx     emqx          4096 Jun 28 06:57 patches
drwxrwxr-x    2 emqx     emqx          4096 Jun 28 06:57 scripts

liuxingjun avatar Jul 16 '21 04:07 liuxingjun

Hi， did you solved your issue? I hope to start "emqx_auth_mnesia" with helm commands, and result in "CrashLoopBackOff".

iMadriver avatar Jul 20 '21 01:07 iMadriver

Hi， did you solved your issue? I hope to start "emqx_auth_mnesia" with helm commands, and result in "CrashLoopBackOff".

You can use kubectl logs to see if this is the same problem as mine

liuxingjun avatar Jul 20 '21 01:07 liuxingjun

Hi， did you solved your issue? I hope to start "emqx_auth_mnesia" with helm commands, and result in "CrashLoopBackOff".

I use 4.2.13 temporarily available

liuxingjun avatar Jul 20 '21 01:07 liuxingjun

My emqx/emqx version is 4.3.5. And after I added EMQX_LOADED_PLUGINS: "emqx_auth_mnesia" in values.yaml, it seems ok but nothing changes. The emqx_auth_mnesia is still not started, verifired in dashboard.

iMadriver avatar Jul 20 '21 01:07 iMadriver

Hi， did you solved your issue? I hope to start "emqx_auth_mnesia" with helm commands, and result in "CrashLoopBackOff".

You can use kubectl logs to see if this is the same problem as mine

I checked the log and got [error] [Plugins] Write File "data/loaded_plugins" Error: erofs, seems the same as your issue

iMadriver avatar Jul 20 '21 03:07 iMadriver

Hi， did you solved your issue? I hope to start "emqx_auth_mnesia" with helm commands, and result in "CrashLoopBackOff".

You can use kubectl logs to see if this is the same problem as mine

I checked the log and got [error] [Plugins] Write File "data/loaded_plugins" Error: erofs, seems the same as your issue

use version 4.2.13 for the time being helm3 install test emqx/emqx -f values.yaml --version 4.2.13

liuxingjun avatar Jul 20 '21 03:07 liuxingjun

Hi， did you solved your issue? I hope to start "emqx_auth_mnesia" with helm commands, and result in "CrashLoopBackOff".

You can use kubectl logs to see if this is the same problem as mine

I checked the log and got [error] [Plugins] Write File "data/loaded_plugins" Error: erofs, seems the same as your issue

use version 4.2.13 for the time being helm3 install test emqx/emqx -f values.yaml --version 4.2.13

wonderful!

iMadriver avatar Jul 20 '21 05:07 iMadriver

Update, I have same issue with 4.3.8 need to rollback to 4.2.13

lpaulusLohr avatar Nov 16 '21 12:11 lpaulusLohr

mark, I have same issue with 4.3.11

gcdd1993 avatar Jan 07 '22 06:01 gcdd1993

Same issue as well with 4.4.0

barisvelioglu avatar Feb 25 '22 10:02 barisvelioglu

Adding

EMQX_LOADED_PLUGINS: "emqx_dashboard"

gives me the same error. But when I try to load it by using

emqxLoadedPlugins: >

on values.yaml, it works fine

## EMQX configuration item, see the documentation (https://hub.docker.com/r/emqx/emqx)
emqxConfig:
  EMQX_NAME:  "{{ .Release.Name }}"

  ## Cluster discovery by dns
  # EMQX_CLUSTER__DISCOVERY: "dns"
  # EMQX_CLUSTER__DNS__NAME: "{{ .Release.Name }}-headless.{{ .Release.Namespace }}.svc.cluster.local"
  # EMQX_CLUSTER__DNS__APP: "{{ .Release.Name }}"
  # EMQX_CLUSTER__DNS__TYPE: "srv"

  ## Cluster discovery by k8s
  EMQX_CLUSTER__DISCOVERY: "k8s"
  EMQX_CLUSTER__K8S__APP_NAME: "{{ .Release.Name }}"
  EMQX_CLUSTER__K8S__APISERVER: "https://kubernetes.default.svc:443"
  EMQX_CLUSTER__K8S__SERVICE_NAME: "{{ .Release.Name }}-headless"
  EMQX_CLUSTER__K8S__NAMESPACE: "{{ .Release.Namespace }}"
  ## The address type is used to extract host from k8s service.
  ## Value: ip | dns | hostname
  ## Note：Hostname is only supported after v4.0-rc.2
  EMQX_CLUSTER__K8S__ADDRESS_TYPE: "hostname"
  EMQX_CLUSTER__K8S__SUFFIX: "svc.cluster.local"
  EMQX_ALLOW_ANONYMOUS: false
  EMQX_AUTH__MONGO__SERVER: "mongodb:27017"
  EMQX_AUTH__MONGO__LOGIN: "root"
  EMQX_AUTH__MONGO__PASSWORD: "mycrazystrongpublicpassword"
  EMQX_AUTH__MONGO__AUTH_SOURCE: "admin"
  EMQX_AUTH__MONGO__AUTH_QUERY__PASSWORD_HASH: "sha256"
  ## if EMQX_CLUSTER__K8S__ADDRESS_TYPE eq dns
  # EMQX_CLUSTER__K8S__SUFFIX: "pod.cluster.local"

## --------------------------------------------------------------------
##  [ACL](https://docs.emqx.io/broker/latest/en/advanced/acl-file.html)

##  -type(who() :: all | binary() |
##                 {ipaddr, esockd_access:cidr()} |
##                 {client, binary()} |
##                 {user, binary()}).

##  -type(access() :: subscribe | publish | pubsub).

##  -type(topic() :: binary()).

##  -type(rule() :: {allow, all} |
##                  {allow, who(), access(), list(topic())} |
##                  {deny, all} |
##                  {deny, who(), access(), list(topic())}).
## --------------------------------------------------------------------
emqxAclConfig: >
  {allow, {user, "dashboard"}, subscribe, ["$SYS/#"]}.
  {allow, {ipaddr, "127.0.0.1"}, pubsub, ["$SYS/#", "#"]}.
  {deny, all, subscribe, ["$SYS/#", {eq, "#"}]}.
  {allow, all}.

emqxLoadedPlugins: >
  {emqx_management, true}.
  {emqx_recon, true}.
  {emqx_retainer, true}.
  {emqx_dashboard, true}.
  {emqx_telemetry, true}.
  {emqx_rule_engine, true}.
  {emqx_bridge_mqtt, false}.
  {emqx_auth_mongo, true}.

emqxLoadedModules: >
  {emqx_mod_acl_internal, true}.
  {emqx_mod_presence, true}.
  {emqx_mod_trace, false}.
  {emqx_mod_st_statistics, false}.
  {emqx_mod_delayed, true}.
  {emqx_mod_rewrite, true}.
  {emqx_mod_subscription, true}.
  {emqx_mod_topic_metrics, true}.

barisvelioglu avatar Feb 26 '22 09:02 barisvelioglu

kubectl edit configmap xxxx--loaded-plugins

ylcrow avatar Apr 13 '22 08:04 ylcrow

Same issue as well with 4.4.2

xiemeilong avatar Apr 15 '22 13:04 xiemeilong

Same issue as well with latest version 4.4.19, but docker run work fine.

zhonghuali avatar Dec 06 '23 04:12 zhonghuali

Hi guys, now EMQ X no longer uses this repository since version 4.3. Use https://github.com/emqx/emqx/ instead.

Rory-Z avatar Dec 06 '23 06:12 Rory-Z