configuration plugin does not support multiple operations at once

[toewsar]

Describe the bug If the device is offline (or the plugin is not started) and c8y requests more than one configuration file, one of these operations will be lost when the plugin is started.

At start of the cunfigurations plugin it asks for all pending operations, but is does not expect that the operations are send with in one MQTT message sparated by a new line:

c8y/s/ds 526,10000000b1bc91d3,t3k_PlcList
526,10000000b1bc91d3,c8y-configuration-plugin

To Reproduce

1. systemctl stop c8y-configuration-plugin
2. Get more than one configuration snapshot from device
3. systemctl start c8y-configuration-plugin
4. Only one configuration file is uploaded to c8y

Expected behavior Consideration of multilined SmartRest operations.

Environment (please complete the following information):

• Raspi
• thin-edge.io version [0.7.3]

[albinsuresh]

AC

• The plugin to filter the messages meant for itself only (524 and 526 for config management) and ignore the rest. The ignored operations will be handled by their own respective plugins/mappers.
• The c8y-log-plugin and c8y-mapper(for software management) should also be fixed along the same lines
• The pending operations spread across multiple lines must be parsed and split first, before they are fed to the operation handling main loop of the plugin. We should avoid handling multi-line requests and individual requests separately

[rina23q]

I opened a PR #1483 to fix the issue for c8y_log_plugin, c8y_configuration_plugin, and tedge_mapper c8y.

[rina23q]

How to test

c8y_configuration_plugin

Prerequisites:

1. tedge and c8y_configuration_plugin are installed on a device. (it doesn't matter if more packages are installed)
2. DTU is configured to connect a Cumulocity tenant. And DTU is connected to c8y (sudo tedge connect c8y is done successfully)
3. c8y-configuration-plugin.service is stopped. (confirmed by sudo systemctl status c8y-configuration-plugin.service. If not, please stop by sudo systemctl stop c8y-configuration-plugin.service)

Steps:

1. Edit /etc/tedge/c8y/c8y-configuration-plugin.toml. At least two different types must be enabled. Example:

# Add the configurations to be managed by c8y-configuration-plugin

files = [
    { path = '/etc/tedge/tedge.toml' },
    { path = '/etc/tedge/mosquitto-conf/c8y-bridge.conf', type = 'c8y-bridge.conf' },
#    { path = '/etc/tedge/mosquitto-conf/tedge-mosquitto.conf', type = 'tedge-mosquitto.conf' },
#    { path = '/etc/mosquitto/mosquitto.conf', type = 'mosquitto.conf' },
#    { path = '/etc/tedge/c8y/example.txt', type = 'example', user = 'tedge', group = 'tedge', mode = 0o444 }
]

2. Start c8y-configuration-plugin.service by sudo systemctl start c8y-configuration-plugin.service.
3. Confirm you have the configuration types visible in your c8y tenants "Configuration" tab. In this example, you should see like this screenshot.
4. Stop c8y-configuration-plugin.service by sudo systemctl stop c8y-configuration-plugin.service.
5. Press "Get snapshot from device" on c8y UI for at least two different types. For example, "c8y-configuration-plugin" and "c8y-bridge.conf".
6. Confirm the operations that you created in step 5 are marked "PENDING" on "Control" tab on c8y UI.
7. Start c8y-configuration-plugin.service again by sudo systemctl start c8y-configuration-plugin.service.
8. Check the status of the operations created by step 5 on "Control" tab on c8y UI.
9. If they are marked "successful" (green), then test is succeeded.

[rina23q]

How to test

c8y_log_plugin

Prerequisites:

1. tedge and c8y_log_plugin are installed on a device. (it doesn't matter if more packages are installed)
2. DTU is configured to connect a Cumulocity tenant. And DTU is connected to c8y (sudo tedge connect c8y is done successfully)
3. c8y-log-plugin.service is stopped. (confirmed by sudo systemctl status c8y-log-plugin.service. If not, please stop by sudo systemctl stop c8y-log-plugin.service)

Steps:

1. Create at least two request log file operations from "Logs" tab on c8y UI. (the Type of log "software-management, others don't matter, use the default)
2. Confirm the operations that you created in step 1 are marked "PENDING" on "Control" tab on c8y UI.
3. Start c8y-log-plugin.service by sudo systemctl start c8y-log-plugin.service.
4. Publish a message by tedge mqtt pub c8y/s/us 500 on your terminal.
5. Check the status of the operations created by step 1 on "Control" tab on c8y UI.
6. If they are marked either "successful" (green) or "failed" (red), then the test is succeeded.

Note: This test checks if the operation status changes to the final state (successful or failed). Therefore, successful or failed doesn't matter.

[rina23q]

How to test

tedge_mapper c8y

Prerequisites:

1. tedge and tedge_mapper are installed on a device. (it doesn't matter if more packages are installed)

Steps:

1. Run sudo tedge config set c8y.smartrest.templates testTemp to set a custom SmartREST template.
2. Create /etc/tedge/operations/c8y_Test file owned by tedge:tedge, with the content below.

[exec]
  topic = "c8y/s/dc/testTemp"
  on_message = "abc"
  command = "/bin/pwd"

3. Disconnect to c8y in order to ensure if it's not connected (sudo tedge disconnect c8y )
4. Connect to c8y (sudo tedge connect c8y is done successfully)
5. Check if tedge-mapper-c8y.service is active. (confirmed by sudo systemctl status tedge-mapper-c8y.service)
6. Run tedge mqtt sub "c8y/s/us"
7. Run these two commands.

STR=$'abc\nabc'
tedge mqtt pub c8y/s/dc/testTemp "$STR"

8. Check these 4 lines are recorded by the subscribed on Step 6. Then the test is succeeded.

[c8y/s/us] 501,c8y_Test
[c8y/s/us] 503,c8y_Test
[c8y/s/us] 501,c8y_Test
[c8y/s/us] 503,c8y_Test

[rina23q]

Build number #868

[rina23q]

We fixed it in 0.7.7. @toewsar Can we close this issue? You can try either 0.7.7 or 0.8.0.