neolink
neolink copied to clipboard
thirtythreeforty
Use camera's motion detection
Some camera's come with motion detection. I would like to have some way for neolink to signal that motion was detected.
For now I think just printing to std-out but ultimately I am thinking of some sort of control interface (maybe a rest API) would be good. I envisage we can add command and control to this control interface, PTZ turning the camera on and off etc etc.
Here are the relevent wireshark packets for motion detection
It is just a BC packet of the msg-id 33:
- Motion start
<?xml version="1.0" encoding="UTF-8" ?>
<body>
<AlarmEventList version="1.1">
<AlarmEvent version="1.1">
<channelId>0</channelId>
<status>MD</status>
<recording>0</recording>
<timeStamp>0</timeStamp>
</AlarmEvent>
</AlarmEventList>
</body>
- Motion stop
<?xml version="1.0" encoding="UTF-8" ?>
<body>
<AlarmEventList version="1.1">
<AlarmEvent version="1.1">
<channelId>0</channelId>
<status>none</status>
<recording>0</recording>
<timeStamp>0</timeStamp>
</AlarmEvent>
</AlarmEventList>
</body>
Agree, it's good to have a tracking issue.
The industry standard here is ONVIF, and if Neolink were an ONVIF bridge you could connect any client to it and it would "just work" with all these more advanced features. ONVIF is also somewhat long in the tooth, and I don't see many open source libraries for being an ONVIF server (except one in Javascript of all things).
Ideas for other standard APIs are welcome, but I'd like to avoid rolling something custom for Neolink - it'll just make work for anyone who wants to consume it.
Yes I actually wanted ONVIF but as you say I too couldn't find a server version.
Could this be helpful to you Guys?
https://github.com/EliasKotlyar/Xiaomi-Dafang-Hacks/issues/78
I had a look at their PR that added it but it seems to be a binary commit so I can't read the source and the changes. Some of the links in that issue may be useful though.
The actual code is in Dafang-Hacks/Main#41. Their overall approach is to use Gsoap to generate bindings to the ONVIF spec.
Thanks I've looked at that now my c++ is rusty (no pun intended) but it seems somewhat doable but quite some work.
It would be best if we could find a soap implementation in rust ive found this but it seems quite minimal. But perhaps a good starting point. It will take soap scheme and generate code. It's designed for a client though not a server.
We could alternatively generate c code with gsoap then either bind or port it.
MQTT might be easier to implement than ONVIF too should be able to connect to home assistant as well.
I plan to add ONVIF when I have time but will also look into MQTT too
Great... I've been bugging manufacturers over and over about some simple MQTT publishes and subscribes. Would be awesome for doing integrations with NodeRed, HomeAssistant, or whatever.
I'm thinking to make the MQTT messages complimant enough to work with this HA plugin and use this rust library for the server side code
As I have understand the plugin we set the topic to the cameraname the subtopic to motion and the message as either motion_start motion_stop would that work with the design in nodered you had?
I don't have the plugin. Pretty wide open to however it would be implemented. A good idea might be to do a prefix configuration. Maybe something like
neolink/reardoor/state/motion with a payload on or off
Thinking ahead where other stuff could be output as states, like LWT, etc. Then if it is possible, the ability to send commands to the camera like to turn on the white LEDs on the Lumus (not sure if that is possible)
send an on payload to neolink/reardoor/command/light
Might a bit much but you get the idea.
I don't mind but this would be going down the route of rolling our own interface, it might be better if there were a standard but it seems from most mqtt projects I have seen that it is more transport than protocol and they often roll there own messages
At least on the plus side the mqtt clients like nodered and HA are configurable in which message to listen for
I do eventually plan to get controls set up too but thats a longer project
@thirtythreeforty Could I get some recommendations on where to implement the subscribe for msg_id 33 (motion)
Currently we subscribe to id 3 in the start_video function. Then we loop polling the rx with a timeout. Do you think I should add a second subscription here in the loop polling id 33 as well?
@digiblur I think I have a working implementation in #78 would you be willing to test it?
@digiblur I think I have a working implementation in #78 would you be willing to test it?
Sure!
Cool, could you checkout the last version from my github assets here
Then add this to your neolink.toml
[[mqtt.servers]]
port = 8883
next_connection_delay_ms = 10
connection_timeout_ms = 100
max_client_id_len = 256
max_connections = 1
throttle_delay_ms = 0
disk_persistence = false
disk_retention_size = 100
disk_retention_time_sec = 1000
auto_save_interval_sec = 1000
max_payload_size = 2048
max_inflight_count = 100
max_inflight_size = 1024
instant_ack = false
# this enables tls connection
#cert_path = "tlsfiles/server.cert.pem"
#key_path = "tlsfiles/server.key.pem"
# provide ca_path to enable client authentication
#ca_path = "tlsfiles/ca-chain.cert.pem"
[mqtt.router]
id = 0
dir = "/tmp/rumqttd"
# A commitlog read will pull full segment. Make sure that a segment isn't
# too big as async tcp writes readiness of one connection might affect tail
# latencies of other connection. Not a problem with prempting runtimes (tokio)
max_segment_size = 10240
max_segment_count = 10
max_connections = 10001
Probably don't need all of those settings they are just the ones that I copy pasted from here
If you put neolink into debug mode with
export RUST_LOG=error,neolink=debug
Then you will see this for the debug messages
Compiling neolink v0.3.0 (/Users/awk21/Projects/Software/neolink)
Finished dev [unoptimized + debuginfo] target(s) in 16.53s
Running `target/debug/neolink --config target/test.toml`
[2020-10-03T08:56:01Z INFO neolink] Neolink v0.3.0-140-g0f83c7d debug
[2020-10-03T08:56:01Z DEBUG neolink] Create mqtt
[2020-10-03T08:56:01Z DEBUG neolink] Start mqtt server
[2020-10-03T08:56:01Z DEBUG neolink] Start polling mqtt
[2020-10-03T08:56:01Z DEBUG neolink] Send mqtt test
[2020-10-03T08:56:01Z INFO neolink::mqtt::live] Starting MQTT Server
[2020-10-03T08:56:01Z DEBUG neolink] mqtt ready
[2020-10-03T08:56:01Z DEBUG neolink::mqtt::live] Incoming. Topic = /neolink, Payload = [b"start"]
[2020-10-03T08:56:01Z DEBUG neolink::gst] Permitting anonymous to access /Cammy02, /Cammy02/mainStream
[2020-10-03T08:56:01Z INFO neolink] Cammy02: Connecting to camera at 192.168.1.101:9000
[2020-10-03T08:56:01Z INFO neolink] Cammy02: Connected and logged in
[2020-10-03T08:56:01Z DEBUG neolink::bc_protocol::connection] Ignoring uninteresting message ID 78
[2020-10-03T08:56:01Z DEBUG neolink::bc_protocol::connection] Ignoring uninteresting message ID 79
[2020-10-03T08:56:01Z INFO neolink] Cammy02: Camera time is already set: 2020-10-03 15:56:05 +7
[2020-10-03T08:56:01Z INFO neolink] Cammy02: Starting video stream mainStream
Look out for these mqtt messages in the log to check its working
[2020-10-03T08:56:01Z DEBUG neolink] Create mqtt
[2020-10-03T08:56:01Z DEBUG neolink] Start mqtt server
[2020-10-03T08:56:01Z DEBUG neolink] Start polling mqtt
[2020-10-03T08:56:01Z DEBUG neolink] Send mqtt test
[2020-10-03T08:56:01Z INFO neolink::mqtt::live] Starting MQTT Server
[2020-10-03T08:56:01Z DEBUG neolink] mqtt ready
[2020-10-03T08:56:01Z DEBUG neolink::mqtt::live] Incoming. Topic = /neolink, Payload = [b"start"]
You should be able to connect on port 8883 with your mqtt client and listen to messages
Topic = /neolink/Cammy02/status/motion, Payload = "on"
Topic = /neolink/Cammy02/status/motion, Payload = "off"
Where Cammy02 is your camera name
Where do you set the mqtt server at? Mine is just a simple auth one as it stays local.
Typically that is the case as I'd imagine most people doing home automation stuff with MQTT needs would already have a MQTT server running. I have one kicking with about 80 or 90 clients off of it already.
I see I see...
Oh well It was fun to make the server at least and learned lots doing it.
Give me a day or two and I'll swap it over a client. I'll call you again for a retest?
@digiblur I have a client based version up and running
Please download the latest version from same place as before.
Your config toml should look like this
[[cameras]]
name = "Cammy02"
username = "UserForCamera"
password = "passForCam"
address = "127.0.0.3:9000"
[cameras.mqtt]
server = "127.0.0.1"
port = 1883
[[cameras]]
name = "Cammy03"
username = "AnotherUserForCamera"
password = "passForAnotherCam"
address = "127.0.0.2:9000"
[cameras.mqtt]
server = "127.0.0.1"
port = 1883
credentials = ["User4MQTT", "Pass4MQTT"]
Each [[cameras]] must have a [cameras.mqtt] to post a message. Each camera runs its own client with the cilient id Neolink-CameraName.
The [cameras.mqtt] table supports the following options
[cameras.mqtt]
# The address of the mqtt server
server = "127.0.0.1"
# The port of the mqtt server
port = 1883
# Optional: User name and password for mqtt
credentials = ["User4MQTT", "Pass4MQTT"]
# Optional: The servers encryption certificate
# Enabling this WILL turn on encryption
# You cannot use both ca and client_auth
ca = "/path/to/cert"
# Optional: The clients encryption certificate and key
# Enabling this WILL turn on encryption
# You cannot use both ca and client_auth
client_auth = ["/Path/to/client/cert", "/Path/to/client/key"]
It also send the message
topic: /neolink/CameraName/start
message: start
When neolink starts the client
If there any other message you think we should send please let me know.
Cool, I'll check it out. Is the start message configured as a LWT message to MQTT?
The start message is not a command just a report as a normal publish at the start.
P.s. it's getting late where I am so if you have issues I wont be able too help until morning. Hope it goes well :)
No worries. Basically when the client logs in it posts a message as last will and testament with the retained flag set. If the MQTT server detects the client is gone for whatever reason it will post the unavailable payload of the LWT. It's great for knowing if a service, device, etc fell off the network or crashed etc. https://www.hivemq.com/blog/mqtt-essentials-part-9-last-will-and-testament/
I think I can maintain two messages, one for neolink online/offline and one for camera connected/disconnected this should be a handy features thanks :)
Perhaps a single message with
- Startup
- Disconnected
- Camera connected
- Connected
- Camera disconnected
- Disconnected
- Neolink shutdown/terminate
- Offline
It does need to be specified as the LWT though so the broker can flip it to offline if the device/service no longer responds.
Should be a matter of
- download
- extract
- chmod 755 ./neolink
- ./neolink --config config_file_name
You could try my docker
docker pull quantumentangledandy/neolink:mqtt
But I have not tested it
Be aware that setting the mqtt server to 127.0.0.1 or localhost will not work from inside a docker as it has its own seperate network. You have to use host.docker.internal instead
Cool I see the MQTT client logging in... drop the initial / of the topic as it creates a null initial entry on MQTT explorer and others. Motion events seem to send across just fine. Very cool! I'm not seeing the LWT topic working though. It should come across as a retain and LWT option. I brought the container down and it didn't post the offline message after getting a MQTT timeout.
I'd also like to see a motion event retain message as an option. For myself I'd combine with the use of LWT to determine if the device was online or not then use the retained state to see the last event even during a home automation system reboot.
Definitely cool to see MQTT messages off a motion event! Nice! Is there a way to send messages to Reolink cams? Like the Lumus to turn on the white LEDs?
Yes I haven't had time to setup the retain or LWT yet been busy with other things. But I agree that we should retain the on off status of the motion event since they do represent continuous states.
Thanks for the update on the leading / will change when I can.
With regards to control messages: We are currently reverse engineering more of the protocol in #80, I think we now know enough to control aspects such as the LED. I shall add this functionality into the MQTT for you to test if you'd like, but will have to wait until I have the time.
I have a build with the LWT and the new connection status setup, will take about an hour for it to publish to docker but you can test after that :)
@digiblur Ok so I have a working LED status light change over MQTT.
Please try the latest docker
docker pull quantumentangledandy/neolink:mqtt
The control done via is:
-
topic:
neolink/CAMERANAME/control/led -
message:
onoroff
This is not perfect yet but it seems to be working... p.s. so many changes to our protocol were required!
Been a few days since I've been able to circle back due to storm issues.... So the LED control kind of works. That's the status LED, the little one. There's another flood LED on the Lumus. Maybe a LED2? But it does indeed turn the little blue LED on and off.
I'm not sure if you got the LWT going yet, I don't see that one working yet.
Do you mean the red circle ones? Those are the IR lights, I have only ever seen the official client set thoses to "auto" or "close" so auto or off (close and off are the same things in some Asian languages). I can send it an "open" but since I have never seen the offical client send that it may not work
Do you mean the red circle ones? Those are the IR lights, I have only ever seen the official client set thoses to "auto" or "close" so auto or off (close and off are the same things in some Asian languages). I can send it an "open" but since I have never seen the offical client send that it may not work
Nope. Not the IR. I guess technically the Lumus has 3 LED types. The IR LED, the Blue status light and the 2 white flood lights. Guess it would be hard to snag the commands it sends without the camera itself in hand.
I will check the LWT maybe I am missing the topic.
Well if your willing to use wireshark and a desktop reolink client you can snag the packets for me and send me the dump
Will do. Question on LWT, I was looking at the code quickly. Is the LWT topic and payload getting configured and sent at connect?
Yes its done on line 86 of mqtt.rs
// On unclean disconnect send this
mqttoptions.set_last_will(LastWill::new(
format!("neolink/{}/status", name),
"offline",
QoS::AtLeastOnce,
true,
));
Unless I have misread the docs on this
I am not familiar with that library. I will have to read up on it. I know on the Arduino code side I had to set the will options. Set the topic and death message so the MQTT broker knows what to publish if the client falls off. On connect I had to send the birth message to the same topic.
Yep yep thats how I am doing it, birth message is sent on line 204 after the client receives conack from the broker
Birth message starts off as disconnected but swaps to connected right after reolink connects to the camera (which dosen't take long) it will swap to disconnected if the camera goes offline but neolink stays online.
Ok at initial connect with the camera disconnected. I see two retains...the disconnected and motion off. That's good to see both of those agreeing.
Plug in the camera and wait for it to reconnect.
I can see the motion going on/off but the status is still disconnected.
@QuantumEntangledAndy Just wanted to check in on this, since I'd like to use the PR as a base for an ftp event. I have a Lumus I'll be using for this test. Does it look like this will be merged to main? Or needs more testing?
@brgavino Do you mean my #78 PR? I think it will need some more work really, this is why it is a draft. I am also not sure how thirtythreeforty feels about binding neolink with mqtt. I also think that #83 or something like it that restructures the message BC structures should be completed first aswell. Perhaps we should ping @thirtythreeforty for his comments.
@QuantumEntangledAndy Yeah #78 was what I was referencing. What dependencies do you see with #83 that are outstanding to close? Adding a general event forwarder interface could decouple mqtt, but the ability to send motion events is a big value add to the features here. If we have a plan for these PRs, I can pull in the right code to test ftp. Otherwise, I'll just take your last commit as a start and see what I can do.
@QuantumEntangledAndy Yeah #78 was what I was referencing. What dependencies do you see with #83 that are outstanding to close? Adding a general event forwarder interface could decouple mqtt, but the ability to send motion events is a big value add to the features here. If we have a plan for these PRs, I can pull in the right code to test ftp. Otherwise, I'll just take your last commit as a start and see what I can do.
That would be great as not everyone wants MQTT, but for the ones that do, it's a game changer.
If you have problems with compiling this branch/PR, see https://github.com/thirtythreeforty/neolink/pull/78#issuecomment-894662456
I realise there are some commands implemented but not sure of the limits so far. Can the light of e.g. Reolink Lumus be controlled using a command from Neolink? I.e. turn on motion activation of LEDs, or turn off LEDs completely, without using Reolink app.
@sumade123 I want to add this kind of LED as it was requested some time ago. However the camera I use only has a status LED and IR LEDs not this flood light LEDs. It would be really good if you (or someone else) could send me some wireshark dumps that capture the packets that the official client sends to activate the lights.
You can find a dump from my Lumus attached @QuantumEntangledAndy. Frame 99 and 435 is turning the LED on and frame 255 and 572 turned the LED off.
Looks like the messageId is 288.
I hope this helps. Let me know if you have something that needs testing. Really looking forward to you getting some of your great PRs merged.
@petergp thank you for the dump unfortunately this is using the newer AES firmware. It is AES encrypted and not easy for me to decrypt.
- Assuming it is encryption mode 02: To decrypt it I would need the NONCE that is send in the login msg_id 1 and the password for the camera.
- If it is encryption mode 03: To decrypt it.... we don't know at the moment that's on the todo list.
The easiest way to make it readable is to use an old reolink client that was released last year and record those packets. This should force it to use encryption mode 01 which is trivially breakable without any passwords.
Right, that explains why I wasn't able to make anything from the body.
I've included the dump from running an older version of the Reolink Android app in an Emulator. That version works a little different than the Reolink iOS app since the light will turn off after 180 seconds however this is configurable in the body as seen below. There must be a value for the duration that will turn it on permanently.
Turn on:
<?xml version="1.0" encoding="UTF-8" ?>
<body>
<FloodlightManual version="1.1">
<channelId>0</channelId>
<status>1</status>
<duration>180</duration>
</FloodlightManual>
</body>
Turn off:
<?xml version="1.0" encoding="UTF-8" ?>
<body>
<FloodlightManual version="1.1">
<channelId>0</channelId>
<status>0</status>
<duration>180</duration>
</FloodlightManual>
</body>
The relevant frames are 319 and 500.
Lumus_led_unencrypted.pcapng.zip
Thanks very much, I am going to finish of my PR motion based streaming then I'll try this one