thingsboard-ce-k8s icon indicating copy to clipboard operation
thingsboard-ce-k8s copied to clipboard

Published 20 hours ago • verified publisher icon thingsboard

How to scale tb-node in monolith type

Open xiddjp opened this issue 1 year ago • 9 comments

How to scale tb-node in monolith type? When i scale tb-node to 2 or 3 pod, telemetry not update any more.

xiddjp avatar Aug 21 '23 06:08 xiddjp

listening

dkim-stratuscent avatar Sep 08 '23 19:09 dkim-stratuscent

Can't seem to figure this out neither. Any thoughts?

icanos avatar Sep 17 '23 16:09 icanos

I dont't know，it's hard to fix.

xiddjp avatar Sep 18 '23 02:09 xiddjp

How to scale tb-node in monolith type? When i scale tb-node to 2 or 3 pod, telemetry not update any more.

What do you mean by "not update anymore"? In dashboard charts (real time or history?), in device latest values tab or what?

aistisdev avatar Oct 02 '23 13:10 aistisdev

How to scale tb-node in monolith type? When i scale tb-node to 2 or 3 pod, telemetry not update any more.

What do you mean by "not update anymore"? In dashboard charts (real time or history?), in device latest values tab or what? In device details page, Latest telemetry tab. telemetry data will no longer be updated. Reopening this tab will display the telemetry value for the current time again, but it will not continue to be updated

xiddjp avatar Oct 09 '23 01:10 xiddjp

In device details page, Latest telemetry tab. telemetry data will no longer be updated. Reopening this tab will display the telemetry value for the current time again, but it will not continue to be updated

I think you need to use zookeeper+kafka for the scaled pods to communicate with each other (even on monolith mode), so to get real time updates on websocket queries. Otherwise the nodes will have their own queues and rule chain exectution logic, which will not trigger the websocket services on one of the nodes that is not receiving data. Also you might not be able to register new rule chains on both nodes if they have no way to communicate with each other.

aistisdev avatar Oct 11 '23 08:10 aistisdev

I use zookeeper and kafka accord thingsboard-ce-k8s/helm folder. but when i scale tb-node up to 2 or 3 node telemetry not update.

xiddjp avatar Oct 11 '23 08:10 xiddjp

Do the telemetry values also not update in real time dashboards? Also when you go to systemadmin home page, do you see all your scaled nodes likes this?: image

aistisdev avatar Oct 12 '23 13:10 aistisdev

I've tested scaling by changing the default value of the environment variable "TB_SERVICE_TYPE" from "monolith" to "tb-core" on the tb-node StatefulSet, then deployed another StatefulSet for the rule engine with TB_SERVICE_TYPE set to "tb-rule-engine". Still haven't configured the liveness probe for the rule engine StatefulSet.

image

jairohg avatar Mar 26 '24 14:03 jairohg