RedMap
RedMap copied to clipboard
Add Heat node for server side heatmap
As discussed in #196. Added a server-side heat node, based on the same logic as tracks.
Heatmap is periodically "normalized" to account for stationary objects that seldom report positions. This is done by recording stationary time whenever an object is stationary, and normalizing the plot so that the position with longest stationary time has an intensity of 1.0, with the rest linearly decreasing in intensity based on their recorded stationary time.
Hi - hmm - not sure I'm understanding the purpose and benefit of this ... do you have a demo flow that will show how to use it ? How does it interact with existing heatmap functionality ?
I tried just uses random positions and they show up on the existing heatmap - but seem to get wiped after a while and not be rewritten so you end up with a blotchy effect as below.

I realize my explanation is lacking, I will elaborate.
The basic idea is that a heatmap is going to provide you some added analytical "value" other than just showing tracks. In my mind this depends on what you are looking for and what kind of sensors you are using to provide geo-data. I also think it is important to differentiate between a sensor and the object that is being tracked by the sensor. In order for the heat-map to provide any value (other than looking fancy) it should provide some relevant analytical insight of the objects (not the sensors?).
To exemplify this imagine two sensors:
Sensor A Sends positional data with a fixed interval at all times (also when stationary).
Sensor B Sends positional data with the same interval as A when moving, but enters "sleep mode" when stationary where it sends updates at a much longer interval. This is a common strategy to conserve power for battery powered devices.
For this example the objects carrying Sensor A and B, lets call them Object A and B are of equal interest, and their movements should carry the same significance into the heat-map. With the current approach (as-is) all positions registered are of equal value. This gives some analytical insight into common movement patterns, and common stationary locations. The problem is that if Object A and B have the same movement pattern, and are stationary the same amount of time, they will not be equally represented. Object A with fixed updates, will get much more "intensity" or "bleed" in the heat-map than Object B that sends an update every so often. The purpose of my normalization method is to mitigate this by detecting stationary objects and treat them equal for the amount of time spent in a stationary position, not the amount of updates sent.
I realize that not everyone are interested in stationary locations, ideally this normalization should also include movement. However that would demand a much more complex system, including interpolation of movement patterns etc.
If you agree to the logic i could implement three modes for the heatmap:
- Normal: all updates are given equal weight (as-is).
- Stationary priority: Normalize time spent in stationary positions (this PR)
- Movement priority: Ignore updates sent from stationary locations.
As requested a demo flow for current functionality. The dummy sensors go stationary every so often (shown in debug log):
[{"id":"76b2995499ab8ad7","type":"tab","label":"Heat - Example","disabled":false,"info":"","env":[]},{"id":"4db475e75279f0d6","type":"group","z":"76b2995499ab8ad7","style":{"stroke":"#2e333a","stroke-opacity":"1","fill":"#383c45","fill-opacity":"0.5","label":true,"label-position":"nw","color":"#a4a4a4"},"nodes":["7cc41528365691a1","0b30d75608884714","a7fe35ca9b5d57d4"],"x":174,"y":499,"w":332,"h":122},{"id":"dbfacfaea93109ec","type":"group","z":"76b2995499ab8ad7","style":{"stroke":"#2e333a","stroke-opacity":"1","fill":"#383c45","fill-opacity":"0.5","label":true,"label-position":"nw","color":"#a4a4a4"},"nodes":["3cc6e4628f2965f7","e9ffed76dae4120d","e4a7a8dfbd3ceec2"],"x":174,"y":359,"w":332,"h":122},{"id":"85f6b35515b08dd9","type":"group","z":"76b2995499ab8ad7","style":{"stroke":"#2e333a","stroke-opacity":"1","fill":"#383c45","fill-opacity":"0.5","label":true,"label-position":"nw","color":"#a4a4a4"},"nodes":["d0e9176a92d907e8","3c1536f400f2dee1","8c4014bf871837f8"],"x":174,"y":639,"w":332,"h":122},{"id":"2c490c3d7b95bfbd","type":"worldmap-heat","z":"76b2995499ab8ad7","name":"","staticThreshold":"0.5","bleedIntensity":"0.1","minNormalize":"30","radius":"15","maxZoom":"16","x":810,"y":300,"wires":[["fe6985ed3a393205"]]},{"id":"fe6985ed3a393205","type":"worldmap","z":"76b2995499ab8ad7","name":"","lat":"","lon":"","zoom":"16","layer":"OSMG","cluster":"","maxage":"","usermenu":"show","layers":"show","panit":"false","panlock":"false","zoomlock":"false","hiderightclick":"false","coords":"deg","showgrid":"false","allowFileDrop":"false","path":"/worldmap","overlist":"DR,CO,RA,DN,HM","maplist":"OSMG,OSMC,EsriC,EsriS,EsriT,EsriDG,UKOS","mapname":"","mapurl":"","mapopt":"","mapwms":false,"x":1010,"y":400,"wires":[]},{"id":"29dca3987a5623a8","type":"inject","z":"76b2995499ab8ad7","name":"normalize","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"{\"command\":{\"heatmap\":{\"normalize\":true}}}","payloadType":"json","x":580,"y":260,"wires":[["2c490c3d7b95bfbd"]]},{"id":"f33accbdbc480538","type":"inject","z":"76b2995499ab8ad7","name":"clear","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"{\"command\":{\"clear\":true}}","payloadType":"json","x":590,"y":220,"wires":[["2c490c3d7b95bfbd"]]},{"id":"7cc41528365691a1","type":"function","z":"76b2995499ab8ad7","g":"4db475e75279f0d6","name":"dummy2","func":"let point = [51.05,-1.36];\nlet brg = 200;\nlet initialSpeed = 50;\nlet maxSpeed = 100;\nlet name = 'dummy2';\nlet lastFix = null;\nlet interval = 5;\nlet timer = context.get('timer') || null;\nlet stationary = 0;\n\nlet speed = initialSpeed;\n\nif (timer && msg.kill) {\n clearTimeout(timer);\n node.status({});\n return null;\n}\n\nfunction update() {\n let date = new Date();\n\n speed = speed + (Math.random() - 0.5);\n brg = brg + 2 + ((Math.random() - 0.5) * 10);\n if (speed > maxSpeed || speed <= 0) {\n speed = initialSpeed;\n }\n \n while (brg >= 360) {\n brg = brg - 360;\n }\n if (stationary > 0) {\n stationary--;\n } else {\n if (lastFix) {\n let ruler = new cheapRuler(point[0]);\n let lonlat = [point[1],point[0]];\n let distance = speed * ((date.getTime() - lastFix.getTime()) / (1000 * 60 * 60));\n nextlonlat = ruler.destination(lonlat, distance, brg);\n point = [nextlonlat[1], nextlonlat[0]];\n }\n if (Math.random() > 0.99) {\n node.warn(\"STATIONARY\");\n stationary = Math.round(Math.random() * 100);\n }\n }\n \n lastFix = date;\n \n msg.payload = {\n \"name\": name,\n \"icon\": \"arrow\",\n \"lat\": point[0],\n \"lon\": point[1],\n \"speed\": speed,\n \"heading\": brg,\n \"lastfix\": lastFix.toISOString()\n };\n node.send(msg);\n timer = setTimeout(update, interval * 1000);\n context.set('timer', timer);\n}\n\nnode.status({fill:\"green\",shape:\"dot\",text:\"running\"});\nupdate();","outputs":1,"noerr":0,"initialize":"","finalize":"// Code added here will be run when the\n// node is being stopped or re-deployed.\nnode.status({});","libs":[{"var":"cheapRuler","module":"cheap-ruler"}],"x":420,"y":560,"wires":[["2c490c3d7b95bfbd","8b806bbe1f03612e"]]},{"id":"0b30d75608884714","type":"inject","z":"76b2995499ab8ad7","g":"4db475e75279f0d6","name":"start","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":270,"y":540,"wires":[["7cc41528365691a1"]]},{"id":"a7fe35ca9b5d57d4","type":"inject","z":"76b2995499ab8ad7","g":"4db475e75279f0d6","name":"kill","props":[{"p":"kill","v":"true","vt":"bool"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":270,"y":580,"wires":[["7cc41528365691a1"]]},{"id":"3cc6e4628f2965f7","type":"function","z":"76b2995499ab8ad7","g":"dbfacfaea93109ec","name":"dummy1","func":"let point = [51.05,-1.39];\nlet brg = 70;\nlet initialSpeed = 3;\nlet maxSpeed = 12;\nlet name = 'dummy1';\nlet lastFix = null;\nlet interval = 5;\nlet timer = context.get('timer') || null;\nlet stationary = 0;\n\nlet speed = initialSpeed;\n\nif (timer && msg.kill) {\n clearTimeout(timer);\n node.status({});\n return null;\n}\n\nfunction update() {\n let date = new Date();\n\n speed = speed + (Math.random() - 0.5);\n brg = brg + 2 + ((Math.random() - 0.5) * 10);\n if (speed > maxSpeed || speed <= 0) {\n speed = initialSpeed;\n }\n \n while (brg >= 360) {\n brg = brg - 360;\n }\n if (stationary > 0) {\n stationary--;\n } else {\n if (lastFix) {\n let ruler = new cheapRuler(point[0]);\n let lonlat = [point[1],point[0]];\n let distance = speed * ((date.getTime() - lastFix.getTime()) / (1000 * 60 * 60));\n nextlonlat = ruler.destination(lonlat, distance, brg);\n point = [nextlonlat[1], nextlonlat[0]];\n }\n if (Math.random() > 0.99) {\n node.warn(\"STATIONARY\");\n stationary = Math.round(Math.random() * 100);\n }\n }\n \n lastFix = date;\n \n msg.payload = {\n \"name\": name,\n \"icon\": \"arrow\",\n \"lat\": point[0],\n \"lon\": point[1],\n \"speed\": speed,\n \"heading\": brg,\n \"lastfix\": lastFix.toISOString()\n };\n node.send(msg);\n timer = setTimeout(update, interval * 1000);\n context.set('timer', timer);\n}\n\nnode.status({fill:\"green\",shape:\"dot\",text:\"running\"});\nupdate();","outputs":1,"noerr":0,"initialize":"","finalize":"// Code added here will be run when the\n// node is being stopped or re-deployed.\nnode.status({});","libs":[{"var":"cheapRuler","module":"cheap-ruler"}],"x":420,"y":420,"wires":[["2c490c3d7b95bfbd","8b806bbe1f03612e"]]},{"id":"e9ffed76dae4120d","type":"inject","z":"76b2995499ab8ad7","g":"dbfacfaea93109ec","name":"start","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":270,"y":400,"wires":[["3cc6e4628f2965f7"]]},{"id":"e4a7a8dfbd3ceec2","type":"inject","z":"76b2995499ab8ad7","g":"dbfacfaea93109ec","name":"kill","props":[{"p":"kill","v":"true","vt":"bool"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":270,"y":440,"wires":[["3cc6e4628f2965f7"]]},{"id":"d0e9176a92d907e8","type":"function","z":"76b2995499ab8ad7","g":"85f6b35515b08dd9","name":"dummy3","func":"let point = [51.053,-1.37];\nlet brg = 140;\nlet initialSpeed = 70;\nlet maxSpeed = 100;\nlet name = 'dummy3';\nlet lastFix = null;\nlet interval = 5;\nlet timer = context.get('timer') || null;\nlet stationary = 0;\n\nlet speed = initialSpeed;\n\nif (timer && msg.kill) {\n clearTimeout(timer);\n node.status({});\n return null;\n}\n\nfunction update() {\n let date = new Date();\n\n speed = speed + (Math.random() - 0.5);\n brg = brg + 2 + ((Math.random() - 0.5) * 10);\n if (speed > maxSpeed || speed <= 0) {\n speed = initialSpeed;\n }\n \n while (brg >= 360) {\n brg = brg - 360;\n }\n if (stationary > 0) {\n stationary--;\n } else {\n if (lastFix) {\n let ruler = new cheapRuler(point[0]);\n let lonlat = [point[1],point[0]];\n let distance = speed * ((date.getTime() - lastFix.getTime()) / (1000 * 60 * 60));\n nextlonlat = ruler.destination(lonlat, distance, brg);\n point = [nextlonlat[1], nextlonlat[0]];\n }\n if (Math.random() > 0.99) {\n node.warn(\"STATIONARY\");\n stationary = Math.round(Math.random() * 100);\n }\n }\n \n lastFix = date;\n \n msg.payload = {\n \"name\": name,\n \"icon\": \"arrow\",\n \"lat\": point[0],\n \"lon\": point[1],\n \"speed\": speed,\n \"heading\": brg,\n \"lastfix\": lastFix.toISOString()\n };\n node.send(msg);\n timer = setTimeout(update, interval * 1000);\n context.set('timer', timer);\n}\n\nnode.status({fill:\"green\",shape:\"dot\",text:\"running\"});\nupdate();","outputs":1,"noerr":0,"initialize":"","finalize":"// Code added here will be run when the\n// node is being stopped or re-deployed.\nnode.status({});","libs":[{"var":"cheapRuler","module":"cheap-ruler"}],"x":420,"y":700,"wires":[["2c490c3d7b95bfbd","8b806bbe1f03612e"]]},{"id":"3c1536f400f2dee1","type":"inject","z":"76b2995499ab8ad7","g":"85f6b35515b08dd9","name":"start","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":270,"y":680,"wires":[["d0e9176a92d907e8"]]},{"id":"8c4014bf871837f8","type":"inject","z":"76b2995499ab8ad7","g":"85f6b35515b08dd9","name":"kill","props":[{"p":"kill","v":"true","vt":"bool"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":270,"y":720,"wires":[["d0e9176a92d907e8"]]},{"id":"8b806bbe1f03612e","type":"function","z":"76b2995499ab8ad7","name":"addtoheatmap: false","func":"msg.payload['addtoheatmap'] = false;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":800,"y":520,"wires":[["fe6985ed3a393205"]]}]
Example output (in the second one you can clearly see stationary positions being highlighted: