iotagent-node-lib icon indicating copy to clipboard operation
iotagent-node-lib copied to clipboard

Published 20 hours ago • verified publisher icon telefonicaid

Reverse option sends the notification twice

Open danielvillalbamota opened this issue 3 years ago • 10 comments

Environment: tutorials.IoT-Agent

IoTA version:

{
    "libVersion": "2.15.0",
    "port": "4041",
    "baseRoot": "/",
    "version": "1.16.0"
}

CB version:

{
    "orion": {
        "version": "3.0.0",
        "uptime": "0 d, 1 h, 14 m, 22 s",
        "git_hash": "d6f8f4c6c766a9093527027f0a4b3f906e7f04c4",
        "compile_time": "Mon Apr 12 14:48:44 UTC 2021",
        "compiled_by": "root",
        "compiled_in": "f307ca0746f5",
        "release_date": "Mon Apr 12 14:48:44 UTC 2021",
        "machine": "x86_64",
        "doc": "https://fiware-orion.rtfd.io/en/3.0.0/",
        "libversions": {
            "boost": "1_66",
            "libcurl": "libcurl/7.61.1 OpenSSL/1.1.1g zlib/1.2.11 nghttp2/1.33.0",
            "libmicrohttpd": "0.9.70",
            "openssl": "1.1",
            "rapidjson": "1.1.0",
            "mongoc": "1.17.4",
            "bson": "1.17.4"
        }
    }
}

Device registration with endpoint webhook to test the commands (service group already set):

{
     "devices": [
    {
      "device_id": "tracker005",
      "entity_name": "car:005",
      "entity_type": "Car",
      "protocol": "IoTA-UL",
      "transport": "HTTP",
      "endpoint": "https://webhook.site/d897aa60-2758-43ba-9df7-f16895114220",
      "attributes": [
        {
            "object_id": "t",
            "name":"turn",
            "type":"Text",
            "expression": "${@t}",
            "reverse": [
                {
                "object_id":"t",
                "type": "Text",
                "expression": "${@turn}"
                }
            ]
        }
      ]
    }
  ]
}

When I send data from the CB:

curl --location --request PUT 'http://localhost:1026/v2/entities/car:005/attrs/turn' \
--header 'Fiware-Service: fiware_zone' \
--header 'Fiware-ServicePath: /' \
--header 'X-Auth-Token: null' \
--header 'Content-Type: application/json' \
--data-raw '{
    "value": "left"
}'

The operation sends two different commands (seconds of difference) to the endpoint:

06/07/2021 6:35:17 PM

tracker005@turn|left

06/07/2021 6:35:22 PM

tracker005@t|left

I would expect just one notification from the "reverse" expression, only the last command tracker005@t|left, as it is configured that t is the attribute that understand the "device". But it seems that the agent is also considering the attribute outside the reverse configuration.

danielvillalbamota avatar Jun 07 '21 16:06 danielvillalbamota

Trying to get deeper using the latest version and a simpler example I got this:

Environment:

  • FIWARE Tutorials IoT Agents Code: https://github.com/FIWARE/tutorials.IoT-Agent/tree/a0f0e769781e600bebb9ce094c4fdfa0d96dbd2c
  • env:
# Orion variables
ORION_PORT=1026
ORION_VERSION=3.0.0

# MongoDB variables
MONGO_DB_PORT=27017
MONGO_DB_VERSION=4.4

# IoT Agent Ultralight Variables
ULTRALIGHT_VERSION=latest
IOTA_NORTH_PORT=4041
IOTA_SOUTH_PORT=7896
IOTA_AUTOCAST=false

# Tutorial variables
TUTORIAL_APP_PORT=3000
TUTORIAL_DUMMY_DEVICE_PORT=3001
  • IoTAgent-ul:
{
    "libVersion": "2.15.0-next",
    "port": "4041",
    "baseRoot": "/",
    "version": "1.16.0-next"
}

Although last tagged version is 1.16.2 the version shown by the agent is "1.16.0-next". I suppose I am using the very latest.

  • env inside the "iot-agent" container:
YARN_VERSION=1.22.5
IOTA_LOG_LEVEL=DEBUG
HOSTNAME=iot-agent
IOTA_CB_PORT=1026
IOTA_TIMESTAMP=true
IOTA_CB_NGSI_VERSION=v2
NODE_ENV=production
IOTA_REGISTRY_TYPE=mongodb
IOTA_MONGO_PORT=27017
IOTA_DEFAULT_RESOURCE=/iot/d
PWD=/opt/iotaul
HOME=/home/node
IOTA_NORTH_PORT=4041
IOTA_AUTOCAST=false
IOTA_CB_HOST=orion
NODE_VERSION=12.21.0
TERM=xterm
IOTA_PROVIDER_URL=http://iot-agent:4041
SHLVL=1
IOTA_HTTP_PORT=7896
IOTA_MONGO_DB=iotagentul
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
IOTA_MONGO_HOST=mongo-db
_=/usr/bin/env

Test executed:

  • Service group provision:
curl --location --request POST 'http://localhost:4041/iot/services' \
--header 'Fiware-Service: fiware_zone' \
--header 'Fiware-ServicePath: /' \
--header 'Content-Type: application/json' \
--data-raw '{
    "services": [
        {
            "attributes": [],
            "static_attributes": [],
            "protocol": [
                "IoTA-UL"
            ],  
            "apikey":      "4jggokgpepnvsb2uv4s40d59ov",
            "cbroker":     "http://orion:1026",
            "entity_type": "bell",
            "resource":    "/iot/d"
        }
      ]
}'
  • Device provision: Entities and attributes has no real meaning.
{
     "devices": [
    {
      "device_id": "tracker008",
      "entity_name": "car:008",
      "entity_type": "Car",
      "protocol": "IoTA-UL",
      "transport": "HTTP",
      "endpoint": "https://webhook.site/d897aa60-2758-43ba-9df7-...",
      "attributes": [
        {
            "object_id": "h",
            "name":"humidity",
            "type":"Number",
            "reverse": [
                {
                "object_id":"h",
                "type": "Number",
                "expression": "${@humidity}"
                }
            ]
        }
      ]
    }
  ]
}

I use webhook service to see the notifications. I keep the example simple without expressions in the attribute coming from the southbound.

I send the update on the humidity attribute through the CB:

curl --location --request PUT 'http://localhost:1026/v2/entities/car:008/attrs/humidity/value' \
--header 'Fiware-Service: fiware_zone' \
--header 'Fiware-ServicePath: /' \
--header 'X-Auth-Token: null' \
--header 'Content-Type: text/plain' \
--data-raw '24'

I get two notifications in the webhook.site: At 06/15/2021 10:37:06 AM

curl -X 'POST' 'https://webhook.site/d897aa60-2758-43ba-9df7-...' -H 'connection: close' -H 'content-length: 22' -H 'host: webhook.site' -H 'fiware-servicepath: /' -H 'fiware-service: fiware_zone' -H 'content-type: ' -d $'tracker008@humidity|24'

And other at 06/15/2021 10:37:07 AM

curl -X 'POST' 'https://webhook.site/d897aa60-2758-43ba-9df7-...' -H 'connection: close' -H 'content-length: 15' -H 'host: webhook.site' -H 'fiware-servicepath: /' -H 'fiware-service: fiware_zone' -H 'content-type: ' -d $'tracker008@h|24'

I would expect just the last one as the notification has the attribute in the device's format.

The IoTAgent log:

...
time=2021-06-15T08:37:06.411Z | lvl=DEBUG | corr=ddcc069e-cdb4-11eb-952d-0242ac120103; cbnotif=1 | trans=6cb0ac10-4935-4b52-9fe3-abaf732e4ff6 | op=IoTAgentNGSI.GenericMiddlewares | from=n/a | srv=fiware_zone | subsrv=/ | msg=Body:

{
    "subscriptionId": "60c863661a8da57c325ff0fd",
    "data": [
        {
            "id": "car:008",
            "type": "Car",
            "humidity": {
                "type": "Number",
                "value": 24,
                "metadata": {}
            }
        }
    ]
}

 | comp=IoTAgent
time=2021-06-15T08:37:06.416Z | lvl=DEBUG | corr=ddcc069e-cdb4-11eb-952d-0242ac120103; cbnotif=1 | trans=6cb0ac10-4935-4b52-9fe3-abaf732e4ff6 | op=IoTAgentNGSI.ContextServer-v2 | from=n/a | srv=fiware_zone | subsrv=/ | msg=Handling notification from [iot-agent:4041] | comp=IoTAgent
time=2021-06-15T08:37:06.420Z | lvl=DEBUG | corr=ddcc069e-cdb4-11eb-952d-0242ac120103; cbnotif=1 | trans=6cb0ac10-4935-4b52-9fe3-abaf732e4ff6 | op=IoTAgentNGSI.MongoDBDeviceRegister | from=n/a | srv=fiware_zone | subsrv=/ | msg=Looking for device with name [car:008]. | comp=IoTAgent
time=2021-06-15T08:37:06.468Z | lvl=DEBUG | corr=2d148120-c462-42bd-a175-74ebdd5579c1 | trans=2d148120-c462-42bd-a175-74ebdd5579c1 | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=n/a | subsrv=n/a | msg=Looking for group params ["service","subservice","type","apikey"] with queryObj {"service":"fiware_zone","subservice":"/","type":"Car"} | comp=IoTAgent
time=2021-06-15T08:37:06.476Z | lvl=DEBUG | corr=2d148120-c462-42bd-a175-74ebdd5579c1 | trans=2d148120-c462-42bd-a175-74ebdd5579c1 | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=n/a | subsrv=n/a | msg=Device group for fields [["service","subservice","type","apikey"]] not found: [{"service":"fiware_zone","subservice":"/","type":"Car"}] | comp=IoTAgent
time=2021-06-15T08:37:06.476Z | lvl=DEBUG | corr=2d148120-c462-42bd-a175-74ebdd5579c1 | trans=2d148120-c462-42bd-a175-74ebdd5579c1 | op=IoTAgentNGSI.BidirectionalPlugin | from=n/a | srv=n/a | subsrv=n/a | msg=Processing active attributes notification | comp=IoTAgent
time=2021-06-15T08:37:06.482Z | lvl=DEBUG | corr=2d148120-c462-42bd-a175-74ebdd5579c1 | trans=2d148120-c462-42bd-a175-74ebdd5579c1 | op=IoTAgentNGSI.BidirectionalPlugin | from=n/a | srv=n/a | subsrv=n/a | msg=Got the following transformations: [{"object_id":"h","type":"Number","expression":"${@humidity}"}] | comp=IoTAgent
time=2021-06-15T08:37:06.483Z | lvl=DEBUG | corr=2d148120-c462-42bd-a175-74ebdd5579c1 | trans=2d148120-c462-42bd-a175-74ebdd5579c1 | op=IoTAgentNGSI.BidirectionalPlugin | from=n/a | srv=n/a | subsrv=n/a | msg=Got the following transformations: [] | comp=IoTAgent
time=2021-06-15T08:37:06.488Z | lvl=DEBUG | corr=2d148120-c462-42bd-a175-74ebdd5579c1 | trans=2d148120-c462-42bd-a175-74ebdd5579c1 | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=n/a | subsrv=n/a | msg=Looking for bindings for the function [notificationHandler] and protocol [HTTP] | comp=IoTAgent
time=2021-06-15T08:37:06.489Z | lvl=DEBUG | corr=2d148120-c462-42bd-a175-74ebdd5579c1 | trans=2d148120-c462-42bd-a175-74ebdd5579c1 | op=IoTAgentNGSI.MongoDBGroupRegister | from=n/a | srv=n/a | subsrv=n/a | msg=Binding found for function [notificationHandler] and protocol [HTTP] | comp=IoTAgent
time=2021-06-15T08:37:06.679Z | lvl=DEBUG | corr=c04203a2-7258-4307-a3ce-f3153d36e04b | trans=c04203a2-7258-4307-a3ce-f3153d36e04b | op=IoTAgentNGSI.GenericMiddlewares | from=n/a | srv=n/a | subsrv=n/a | msg=Request for path [/iot/about] from [localhost:4041] | comp=IoTAgent
time=2021-06-15T08:37:06.683Z | lvl=DEBUG | corr=c04203a2-7258-4307-a3ce-f3153d36e04b | trans=c04203a2-7258-4307-a3ce-f3153d36e04b | op=IoTAgentNGSI.DomainControl | from=n/a | srv=n/a | subsrv=n/a | msg=response-time: 5 | comp=IoTAgent
time=2021-06-15T08:37:07.343Z | lvl=DEBUG | corr=ddcc069e-cdb4-11eb-952d-0242ac120103; cbnotif=1 | trans=6cb0ac10-4935-4b52-9fe3-abaf732e4ff6 | op=IoTAgentNGSI.DomainControl | from=n/a | srv=fiware_zone | subsrv=/ | msg=response-time: 944 | comp=IoTAgent
...

I see two op=IoTAgentNGSI.BidirectionalPlugin, but I am not sure if this is related.

If you need more information let me know.

danielvillalbamota avatar Jun 15 '21 09:06 danielvillalbamota

I am interested to work in this issue.

Sunny-NEC avatar Oct 01 '21 09:10 Sunny-NEC

@Sunny-NEC thanks for your willingness to work on this issue! Please go ahead and create the pull request with your contribution, so we can evaluate and provide feedback.

fgalan avatar Oct 01 '21 10:10 fgalan

@danielvillalbamota @fgalan I am investigation in this issue and trying to reproduce the issue.

  • By using below curl command Device registration successfully take place .

curl -X POST -H "Fiware-Service: myHome" -H "Fiware-ServicePath: /environment" -H "Content-Type: application/json" -H "Cache-Control: no-cache" -d '{

 "devices": [
{
  "device_id": "tracker005",
  "entity_name": "car:005",
  "entity_type": "Car",
  "protocol": "IoTA-UL",
  "transport": "HTTP",
  "attributes": [
    {
        "object_id": "t",
        "name":"turn",
        "type":"Text",
        "expression": "${@t}",
        "reverse": [
            {
            "object_id":"t",
            "type": "Text",
            "expression": "${@turn}"
            }
        ]
    }
  ]
}

] }

' 'http://localhost:4041/iot/devices'

  • After the device registration When I send data from the CB by using curl command :

curl --location --request PUT 'http://localhost:1026/v2/entities/car:005/attrs/turn'
--header 'Fiware-Service: fiware_zone'
--header 'Fiware-ServicePath: /'
--header 'X-Auth-Token: null'
--header 'Content-Type: application/json'
--data-raw '{ "value": "left" }'

OUTPUT : getting an error : {"error":"MethodNotAllowed","description":"method not allowed"}.

Is their any additional steps required to reproduce the issue. please let me know.

Sunny-NEC avatar Oct 05 '21 08:10 Sunny-NEC

Hello, @Sunny-NEC

I see FIWARE Service and ServicePath do not match with the provided in the Device registration.

Try using Fiware-Service: myHome and Fiware-ServicePath: /environment as the entity is in that scope.

You will also need an endpoint in the device's provision to receive the commands

danielvillalbamota avatar Oct 15 '21 12:10 danielvillalbamota

@danielvillalbamota thankyou for your response. i tried Fiware-Service: myHome and Fiware-ServicePath: /environment as the entity is in that scope. but still getting a same error {"error":"MethodNotAllowed","description":"method not allowed"}.

Sunny-NEC avatar Oct 19 '21 10:10 Sunny-NEC

I still experience this issue in version 1.25.0

Commands are send twice during notification

Blobonat avatar Nov 09 '22 19:11 Blobonat

@Blobonat can you confirm you are having this issue just when using the reverse option?

mapedraza avatar Nov 15 '22 11:11 mapedraza

@mapedraza During a PoC I noticed that commands always are sent twice using following attribute configuration within the IOTA:

{
                "name": "warningOn",
                "type": "Property",
                "reverse": [
                    {
                        "object_id": "warningOn",
                        "type": "Text",
                        "expression": "${@warningOn}"
                    }
                ]
            }

Subscriptions in the NGSI-LD broker are correctly created only once.

I can't test a device setup with commands since the PoC setup is based on Scorpio, which does not support the IOTA-like actuation yet.

Blobonat avatar Nov 15 '22 11:11 Blobonat

Can you provide more information about how the IoT is deployed (config, environment vars, log output, group provision etc) that helps to reproduce the issue?

mapedraza avatar Nov 15 '22 15:11 mapedraza