thingsboard
[BUG] Gateway connectors not visible in frontend when added via docker volumes
Describe the bug After deploying ThingsBoard with docker-compose and setting up a plug & play proof of concept using scripts and makefiles, the Modbus connectors are not visible in the UI under Paneles > ThingsBoard IoT Gateways > IoT Gateway > Connectors configuration, despite being present in the Docker volume. However, if the connectors are added manually via the UI, they appear correctly.
Your Server Environment
- own setup
- Deployment: Microservices
- Deployment type: docker-compose
- ThingsBoard Version 3.6.4 (Gateway version: 3.5)
- Community Edition
- OS Name and Version
Docker compose file:
version: '3.0'
services:
#########################
###### THINGSBOARD ######
#########################
thingsboard:
restart: always
image: "thingsboard/tb-postgres:${THINGSBOARD_VERSION:-3.6.2}" # 3.4.1 Also can use thingsboard/tb-cassandra (need minimum 4GB of RAM, but is recommended 8GB)
ports:
- "${THINGSBOARD_PORT:-8080}:9090" # HTTP port
- "${MQTT_PORT}:1883" # MQTT Port
- "${RPC_PORT}:7070" # Edge RPC Port
- "${UDP_PORT_MIN}-${UDP_PORT_MAX}:5683-5688/udp" # UDP Port (to exposed internal COAP and LwM2m2 port)
environment:
TB_QUEUE_TYPE: ${TB_QUEUE_TYPE:-in-memory} # Can use an external queue, for example mqtt (need more environment config). In memory is not suitable for clusters.
SSL_ENABLED: ${THINGSBOARD_USE_SSL:-false}
MONITORING_ENABLED : ${MONITORING_ENABLED:-false}
DATABASE: ${DATABASE:-postgres}
DATABASE_TS_TYPE: ${TB_DATABASE_TS_TYPE:-sql} # sql, cassandra or timescale
DATABASE_TS_LATEST_TYPE: ${TB_DATABASE_TS_LATEST_TYPE:-sql} # sql, cassandra or timescale
volumes:
- tb-data:/data
- tb-logs:/var/log/thingsboard
networks:
- network_default
thingsboard-setup:
build:
context: ./client
dockerfile: dockerfile
environment:
THINGSBOARD_HOST: ${THINGSBOARD_HOST:-thingsboard}
THINGSBOARD_PORT: 9090
THINGSBOARD_USE_SSL: ${THINGSBOARD_USE_SSL:-False}
THINGSBOARD_USER: ${THINGSBOARD_USER}
THINGSBOARD_PASSWORD: ${THINGSBOARD_PASS}
THINGSBOARD_GW_NAME: ${THINGSBOARD_GW_NAME:-IoT_Gateway}
TB_GW_CREDENTIAL_TYPE: ${TB_GW_CREDENTIAL_TYPE:-ACCESS_TOKEN}
TB_GW_CREDENTIAL_VALUE: ${TB_GW_CREDENTIAL_VALUE}
depends_on:
- thingsboard
deploy:
restart_policy:
condition: on-failure
delay: 20s
max_attempts: 20
window: 60s
networks:
- network_default
#########################
###### GatewayIoT ######
#########################
tb-gateway:
image: thingsboard/tb-gateway:${THINGSBOARD_GW_VERSION:-3.4.5}
container_name: tb-gateway
restart: always
#command: ["/bin/sh", "-c", "pip3 install -r requirements.txt && pip3 install pyserial-asyncio && pip install 'pymodbus<3.4.0' --force-reinstall && python3 thingsboard_gateway/tb_gateway.py $TB_HOT_RELOAD"]
# Ports bindings - required by some connectors
ports:
- "${GATEWAY_PORT}:5000" # Comment if you don't use REST connector and change if you use another port
# Uncomment and modify the following ports based on connector usage:
# - "1052:1052" # BACnet connector
# - "5026:5026" # Modbus TCP connector (Modbus Slave)
# - "50000:50000/tcp" # Socket connector with type TCP
# - "50000:50000/udp" # Socket connector with type UDP
# Environment variables
environment:
host: thingsboard
port: 1883
accessToken: ${TB_GW_CREDENTIAL_VALUE}
DEBUG: "True"
# Volumes bind
volumes:
- tb-gw-config:/thingsboard_gateway/config
- tb-gw-logs:/thingsboard_gateway/logs
- tb-gw-extensions:/thingsboard_gateway/extensions
depends_on:
- thingsboard-setup
networks:
- network_default
#########################
###### Volumes ######
#########################
volumes:
tb-data:
name: tb-data
tb-logs:
name: tb-logs
tb-gw-config:
name: tb-gw-config
tb-gw-logs:
name: tb-gw-logs
tb-gw-extensions:
name: tb-gw-extensions
networks:
network_default:
driver: bridge
Gateway configuration:
{
"thingsboard": {
"host": "thingsboard",
"port": 1883,
"remoteShell": false,
"remoteConfiguration": false,
"statistics": {
"enable": false,
"statsSendPeriodInSeconds": 3600,
"configuration": "statistics.json"
},
"maxPayloadSizeBytes": 1024,
"minPackSendDelayMS": 0,
"checkConnectorsConfigurationInSeconds": 60,
"handleDeviceRenaming": true,
"checkingDeviceActivity": {
"checkDeviceInactivity": false,
"inactivityTimeoutSeconds": 120,
"inactivityCheckPeriodSeconds": 10
},
"security": {
"accessToken": "XXXXXXXXXXXXXXXXXXXXXXXXXXXX"
},
"qos": 1
},
"storage": {
"type": "memory",
"read_records_count": 100,
"max_records_count": 100000
},
"grpc": {
"enabled": false,
"serverPort": 9595,
"keepaliveTimeMs": 10000,
"keepaliveTimeoutMs": 5000,
"keepalivePermitWithoutCalls": true,
"maxPingsWithoutData": 0,
"minTimeBetweenPingsMs": 10000,
"minPingIntervalWithoutDataMs": 5000
},
"connectors": [
{
"name": "Simulator MQTT controller",
"type": "mqtt",
"configuration": "simulator_mqtt.json"
},
{
"name": "Huawei inverter Connector",
"type": "modbus",
"configuration": "huawei_solar_inverter.json"
},
{
"name": "Victron inverter Connector",
"type": "modbus",
"configuration": "victron_solar_inverter.json"
},
{
"name": "Pylontech battery controller",
"type": "modbus",
"configuration": "pylontech_batteries.json"
}
]
}
Your Client Environment **Desktop
- OS: Ubuntu 22.04
To Reproduce Steps to reproduce the behavior:
- Deploy ThingsBoard and Gateway using docker-compose.
- Load the Modbus connector configurations into the Docker volume (using script).
- Register the gateway on the ThingsBoard platform (using client).
- Navigate to Paneles > ThingsBoard IoT Gateways > IoT Gateway > Connectors configuration.
- Observe that the connectors are not visible in the UI, but it works.
Expected behavior The Modbus connectors added via scripts should be visible in the UI under the Connectors configuration, just as they are when added manually through the UI.
Screenshots
Additional context Service script for registering the gateway:
def main():
# Creating the REST client object with context manager to get auto token refresh
with RestClientCE(base_url=url) as rest_client:
try:
# Auth with credentials
logging.info(f"usermane={TB_USERNAME} | password={TB_PASSWORD}")
rest_client.login(username=TB_USERNAME, password=TB_PASSWORD)
# Creating a Device
default_device_profile_id = rest_client.get_default_device_profile_info().id
device = Device(name=THINGSBOARD_GW_NAME, device_profile_id=default_device_profile_id, additional_info={"gateway": True, "overwriteActivityTime": False, "description": ""})
credentials = DeviceCredentials(credentials_type=TB_GW_CREDENTIAL_TYPE, credentials_id=TB_GW_CREDENTIAL_VALUE)
create_device_request = SaveDeviceWithCredentialsRequest(device=device, credentials=credentials)
device = rest_client.save_device_with_credentials(create_device_request)
logging.info(" Device was created:\n%r\n", device)
except ApiException as e:
logging.exception(e)
Hi @javier-alvarez-ctic,
Thank you for your interest in the ThingsBoard IoT Gateway. We know about this situation, and at the moment there is no mechanism to synchronize manually created connectors with UI. We are going to add this ability, but it requires changes on the ThingsBoard side as well as changes in the gateway. This functionality may be available in one of the future releases of ThingsBoard.
Thank you very much for your reply and for all the work. I am very interested in the new feature. Synchronizing the connectors added with the client (or added manually on Docker volumes) to be able to see them through the UI would be a great idea. This would allow me to build a Plug & Play deployment in my company, allowing our technicians to easily make the necessary changes to the connector configurations or add new ones if needed.
I look forward to this enhancement. Thanks very much again for all.
