Can't serialize Unsigned number from BACPypes (Support for AnyAtomic needed)

[tnesztler]

Description of Issue

JSON serializer is unable to convert Unsigned number to a valid JSON string. Could this be the same issue as #1650?

Affected Version

master branch (2018-04-06) (9ef6b29)

Additional Details

2018-04-06` 10:40:02,298 (bacnet_proxyagent-0.4 3400) <stderr> ERROR: Traceback (most recent call last):
2018-04-06 10:40:02,299 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:   File "/home/XXXX/volttron/env/local/lib/python2.7/site-packages/gevent/greenlet.py", line 534, in run
2018-04-06 10:40:02,300 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:     result = self._run(*self.args, **self.kwargs)
2018-04-06 10:40:02,301 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:   File "/home/XXXX/volttron/volttron/platform/vip/agent/subsystems/rpc.py", line 296, in _handle_subsystem
2018-04-06 10:40:02,302 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:     dispatch(bytes(msg), message) for msg in message.args) if response]
2018-04-06 10:40:02,302 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:   File "/home/XXXX/volttron/volttron/platform/vip/agent/subsystems/rpc.py", line 296, in <genexpr>
2018-04-06 10:40:02,303 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:     dispatch(bytes(msg), message) for msg in message.args) if response]
2018-04-06 10:40:02,304 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:   File "/home/XXXX/volttron/volttron/platform/jsonrpc.py", line 335, in dispatch
2018-04-06 10:40:02,305 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:     return self.serialize(response)
2018-04-06 10:40:02,306 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:   File "/home/XXXX/volttron/volttron/platform/vip/agent/subsystems/rpc.py", line 79, in serialize
2018-04-06 10:40:02,307 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:     return jsonapi.dumps(json_obj)
2018-04-06 10:40:02,308 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:   File "/home/XXXX/volttron/volttron/platform/agent/json.py", line 49, in dumps
2018-04-06 10:40:02,309 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:     return _dumps(data, **kwargs)
2018-04-06 10:40:02,310 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:   File "/home/XXXX/volttron/env/local/lib/python2.7/site-packages/zmq/utils/jsonapi.py", line 40, in dumps
2018-04-06 10:40:02,310 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:     s = jsonmod.dumps(o, **kwargs)
2018-04-06 10:40:02,312 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:   File "/home/XXXX/volttron/env/local/lib/python2.7/site-packages/simplejson/__init__.py", line 397, in dumps
2018-04-06 10:40:02,312 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:     **kw).encode(obj)
2018-04-06 10:40:02,313 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:   File "/home/XXXX/volttron/env/local/lib/python2.7/site-packages/simplejson/encoder.py", line 275, in encode
2018-04-06 10:40:02,313 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:     chunks = self.iterencode(o, _one_shot=True)
2018-04-06 10:40:02,314 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:   File "/home/XXXX/volttron/env/local/lib/python2.7/site-packages/simplejson/encoder.py", line 357, in iterencode
2018-04-06 10:40:02,315 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:     return _iterencode(o, 0)
2018-04-06 10:40:02,316 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:   File "/home/XXXX/volttron/env/local/lib/python2.7/site-packages/simplejson/encoder.py", line 252, in default
2018-04-06 10:40:02,316 (bacnet_proxyagent-0.4 3400) <stderr> ERROR:     raise TypeError(repr(o) + " is not JSON serializable")
2018-04-06 10:40:02,317 (bacnet_proxyagent-0.4 3400) <stderr> ERROR: TypeError: <bacpypes.primitivedata.Unsigned object at 0x7f5f2c3f6cd0> is not JSON serializable
2018-04-06 10:40:02,318 (bacnet_proxyagent-0.4 3400) <stderr> ERROR: <Greenlet at 0x7f5f2cdb6f50: _handle_subsystem(<volttron.platform.vip.agent.subsystems.rpc.RPC ob, Message(**{'peer': 'platform.driver', 'subsystem':)> failed with TypeError
2018-04-06 10:40:02,318 (bacnet_proxyagent-0.4 3400) <stderr> ERROR: 

[kmonson]

Looks like a case where we do not cast the bacpypes object to a python int. I'll look into that.

I have no way of testing the fix so I may send you a patch to try out.

[kmonson]

Is this happening during a normal scrape of the device? Have you set the "use_read_multiple" setting in the settings for the bacnet device?

[kmonson]

Also, could you send me the register csv and device configuration for the device. I need to make sure I'm chasing the right problem.

[tnesztler]

Is this happening during a normal scrape of the device? Have you set the "use_read_multiple" setting in the settings for the bacnet device?

I would say yes. I did not specifically set the use_read_multiple settings. See below my config for the bacnet device and master driver.

Also, could you send me the register csv and device configuration for the device. I need to make sure I'm chasing the right problem.

I replaced the building sensitive information every time by ABCXYZ

"bldg55/ABCXYZ/Schedule-501": {
"type": "csv",
"data": "Reference Point Name,Volttron Point Name,Units,Unit Details,BACnet Object Type,Property,Writable,Index,Write Priority,Notes\nABCXYZ-NAE01/Schedule.AHU 4 FLR 5-6-7 SCHEDULE,ABCXYZ-NAE01/Schedule.AHU 4 FLR 5-6-7 SCHEDULE,UNKNOWN UNITS,,schedule,presentValue,False,3005414,,\nABCXYZ-NAE01/Schedule.ABCXYZ BALLROOM AHU SCHEDULE,ABCXYZ-NAE01/Schedule.ABCXYZ BALLROOM AHU SCHEDULE,UNKNOWN UNITS,,schedule,presentValue,False,3005415,,\nABCXYZ-NAE01/Schedule.1ST FLOOR SW AHU,ABCXYZ-NAE01/Schedule.1ST FLOOR SW AHU,UNKNOWN UNITS,,schedule,presentValue,False,3005412,,\nABCXYZ-NAE01/Schedule.AHU EAST BALLROOM,ABCXYZ-NAE01/Schedule.AHU EAST BALLROOM,UNKNOWN UNITS,,schedule,presentValue,False,3007040,,\nABCXYZ-NAE01/Schedule.AHU-4 LOBBY SCHEDULE,ABCXYZ-NAE01/Schedule.AHU-4 LOBBY SCHEDULE,UNKNOWN UNITS,,schedule,presentValue,False,3005413,,\n"
},
"devices/bldg55/ABCXYZ/Schedule-501": {
"type": "json",
"data": "{\"driver_type\": \"bacnet\", \"interval\": 300, \"timezone\": \"America/Chicago\", \"driver_config\": {\"device_address\": \"192.168.1.3\", \"device_id\": 501}, \"group\": 0, \"registry_config\": \"config://bldg55/ABCXYZ/Schedule-501\"}"
},
"config": {
"type": "json",
"data": "{\n    \"driver_scrape_interval\": 1,\n    \"publish_depth_first_all\": true,\n    \"publish_breadth_first_all\": false,\n    \"publish_depth_first\": false,\n    \"publish_breadth_first\": false,\n    \"group_offset_interval\": 0.0\n}"
},

Let me know if you need something else.

[kmonson]

The short version is that we don't currently support schedule objects. The core problem is that the BACnet Proxy Agent doesn't know how to deal with properties of the AnyAtomic type.

[kmonson]

Here is a drop in replacement for the proxy's agent.py file. Replace agent.py and reinstall the proxy and give it a shot. If you get any errors let me know and I'll see what I can do. It just tries to handle the AnyAtomic data type based on my reading of the bacpypes code.

https://gist.github.com/kmonson/ee1434b91647269809ba04f9347d9121

I have no way of testing this as I don't have a device that it will test this code.

Also, if the schedule presentValue data type is actually a date or time it will return a list of values with 255 meaning "ignore this field". Both have 4 values: Date (year, month, day, day_of_week) , Time (hour, minute, second, hundredth). You'll need to know what it is in order to interpret it correctly.

[tnesztler]

After installing this version, the error was no longer showing up in the logs so I would assume it fixed the problem. I installed it using python scripts/install-agent.py -s services/core/BACnetProxy -c services/core/BACnetProxy/config -t bacnet_proxy --start while replacing agent.py in services/core/BACnetProxy/bacnet_proxy.

However, the data mover no longer received data from it. I had to revert to the agent.py from master to get the data collection going again. FYI, the data mover itself was working since points like cpu were still being pushed to the VC instance.