neolink
neolink copied to clipboard
QuantumEntangledAndy
Help with translating to python for HomeAssistant
I am the maintainer of the build-in Reolink integration. I have just tested this neolink project and indeed I was able to turn on/off the HTTPS port on my Reolink cameras. It would be very benificial for the HomeAssistant users of the Reolink integration if they did not need to turn on this HTTPS port on the cameras manually, especially for new users. Therefore I am hoping to implement the Baichuan protocol (port 9000) in python to:
- login to the camera
- check which ports are available and which are enabled (HTTP, HTTPS, RTMP, RTSP, ONVIF)
- enable the HTTPS port
- logout
Unfortunately I am not familiar with Rust. It would be a greath help for me if you would be willing to point me in the right direction.
I don't know if you are familiar with Python? It would be amazing if you could write a (small as possible) python script that can login to the camera using the Baichuan protocol. But even if you could provide the relevant simplified Rust code used by this library to make the login that would already be very helpfull (so small as possible just the login code, so I don't need to figure out the complete workings of this project).
Would you be willing to help me out a bit?
I'm quite familiar with python, so please let me know what issues you are facing. You what this in pure python or would you be ok with some FFI binding to neolink core?
I've got the protocol documented in dissector/ have you read that?
Thank you so much, amazing you are willing to help me out a bit.
Unfortunately I will need pure python. It will be part of a build-in HomeAssistant integration (not a add-on) so will need just python.
I read the docs under dissector, that is very very helpful. I am just not so sure about the encryption part of the protocol. But honestly I have not tried it out yet, will attempt first login this weekend.
Two possible paths for encryption
- find a package for it, not sure if there are any in pure python for this but i suspect ha already has some aes package in its requirement
- during login you can limit the maximum encryption you support, 0: no encryption (some camera refuse this), 1 bc encryption (trivial to write your own pure python encoding functions), 2+: aes needs external libs
I have worked with encryption libs before, that's no issue. I was just wondering which part of the message and wich messages will be encrypted. But I will figure it out.
Especially if you would be willing to help me point in the right direction if I get stuck
Encryption starts at the 01 level (what we call bc_encrypt trivial to write (and to break)), where the nonce is negotiated then is elevated to 03 with aes.
their are three parts to a message
- binary header
- extension payload (optional)
- message payload (optional)
When extension payload is present it is always encrypted
When present message payload comes in two forms
- xml commands (always encrypted)
- binary
The binary payloads are described by the extension payload (<binary>1<\binary> set) and are present for video and some ptz. These may or may not be encrypted depending on if the flags are set in the extension xml
@QuantumEntangledAndy I have started working on my first attempt at the initial legacy login message.
your docs state:
Send legacy login message
User and pass MD5'ed
Capped at 32 bytes with a null terminator
Bytes 32 is always zero so only first 31 bytes are compared
But unfortunately that is not quite clear enough for me :(
Do I just concanate the username and password directly after each-other (nothing in-between) then cap it at 31 chars with a \0 at the end (or pad with \0 to a lenght of 32 chars). and then do the MD5-hash. results in 16 bytes
Or do I do a seperate MD5-hash of the username and MD5-hash of the password and then combine them? results in 32 bytes
When I look at a wireshark capture of your neolink program, the first message beeing send (which I am hoping is the legacy login) is actually 126 bytes long, so no idea where the other bytes are coming from.....
Moreover in the header of this captured message, between the message length ('2c070000') and the message class ('1465') there are all zeros. I would have expected a encryption_offset of '00000001' and encrypt '01' with the unknown 'dc' as in your docs. what is going on there?
Oh and after I make the MD5-hash, should I then do some encryption of that body? I do see the decrypt methods documented, but I do not see the encrypt method....
Sorry for beeing slow to pick this up...
The BCEncrypt cipher is symmetric in other words decrypt and encrypt are the same.
For the hashing see this comment in the source
https://github.com/QuantumEntangledAndy/neolink/blob/f15fdaefbcfe5cde3b0289c1e106cb85f6d3d39b/crates/core/src/bc_protocol.rs#L436-L443
The code below is
fn test_md5_string() {
// Note that these literals are only 31 characters long - see explanation above.
assert_eq!(
md5_string("admin", Truncate),
"21232F297A57A5A743894A0E4A801FC"
);
assert_eq!(
md5_string("admin", ZeroLast),
"21232F297A57A5A743894A0E4A801FC\0"
);
}
Shows how their are too different ways BC handles hashes, one simple truncates to 31 chars instead of the usual 32 and the other replaces it with a \0
fn md5_string(input: &str, trunc: Md5Trunc) -> String {
let mut md5 = format!("{:X}\0", md5::compute(input));
md5.replace_range(31.., if trunc == Truncate { "" } else { "\0" });
md5
}
For login we need both:
- Stage 1 Legacy:
ZeroLast - State 2 Modern:
Truncate
For the legacy though, I've discovered that you can actually skip the passwords and hashes and just send the header only without a body. That way we don't compromise the MD5 hashes of the passwords.
- Send legacy upgrade header only
| magic | message id | message length | encryption offset | Requested Max Encryption protocol | message class |
|---|---|---|---|---|---|
| f0 de bc 0a | 01 00 00 00 | 00 00 00 00 | 00 00 00 01 | 01 dc | 14 65 |
- Recieve XML from camera with NONCE
| magic | message id | message length | encryption offset | Final confirmed protocol | message class |
|---|---|---|---|---|---|
| f0 de bc 0a | 01 00 00 00 | 91 00 00 00 | 00 00 00 01 | 01 dd | 14 66 |
<?xml version="1.0" encoding="UTF-8" ?>
<body>
<Encryption version="1.1">
<type>md5</type>
<nonce>13BCECE33DA453DB</nonce>
</Encryption>
</body>
- Send modern login, using Truncate
| Magic | Message ID | Message Length | Encryption Offset | Status Code | Message Class | Payload Offset |
|---|---|---|---|---|---|---|
| f0 de bc 0a | 01 00 00 00 | 28 01 00 00 | 00 00 00 01 | 00 00 | 14 64 | 00 00 00 00 |
<?xml version="1.0" encoding="UTF-8" ?>
<body>
<LoginUser version="1.1">
<userName>...</userName> <!-- Hash of username with nonce -->
<password>...</password> <!-- Hash of password with nonce -->
<userVer>1</userVer>
</LoginUser>
<LoginNet version="1.1">
<type>LAN</type>
<udpPort>0</udpPort>
</LoginNet>
</body>
- Sucess (status code is 200), camera replies with details of itself
| Magic | Message ID | Message Length | Encryption Offset | Status Code | Message Class | Payload Offset |
|---|---|---|---|---|---|---|
| f0 de bc 0a | 01 00 00 00 | 2e 06 00 00 | 00 00 00 01 | c8 00 | 00 00 | 00 00 00 00 |
<?xml version="1.0" encoding="UTF-8" ?>
<body>
<DeviceInfo version="1.1">
<firmVersion>00000000000000</firmVersion>
<IOInputPortNum>0</IOInputPortNum>
<IOOutputPortNum>0</IOOutputPortNum>
<diskNum>0</diskNum>
<type>wifi_solo_ipc</type>
<channelNum>1</channelNum>
<audioNum>1</audioNum>
<ipChannel>0</ipChannel>
<analogChnNum>1</analogChnNum>
<resolution>
<resolutionName>2304*1296</resolutionName>
<width>2304</width>
<height>1296</height>
</resolution>
<language>English</language>
<sdCard>1</sdCard>
<ptzMode>pt</ptzMode>
<typeInfo>IPC</typeInfo>
<softVer>33555019</softVer>
<hardVer>0</hardVer>
<panelVer>0</panelVer>
<hdChannel1>0</hdChannel1>
<hdChannel2>0</hdChannel2>
<hdChannel3>0</hdChannel3>
<hdChannel4>0</hdChannel4>
<norm>NTSC</norm>
<osdFormat>DMY</osdFormat>
<B485>0</B485>
<supportAutoUpdate>0</supportAutoUpdate>
<userVer>1</userVer>
</DeviceInfo>
<StreamInfoList version="1.1">
<StreamInfo>
<channelBits>1</channelBits>
<encodeTable>
<type>mainStream</type>
<resolution>
<width>2304</width>
<height>1296</height>
</resolution>
<defaultFramerate>15</defaultFramerate>
<defaultBitrate>2560</defaultBitrate>
<framerateTable>15,12,10,8,6,4,2</framerateTable>
<bitrateTable>1024,1536,2048,2560,3072</bitrateTable>
</encodeTable>
<encodeTable>
<type>subStream</type>
<resolution>
<width>896</width>
<height>512</height>
</resolution>
<defaultFramerate>15</defaultFramerate>
<defaultBitrate>512</defaultBitrate>
<framerateTable>15,12,10,8,6,4,2</framerateTable>
<bitrateTable>128,256,384,512,768,1024</bitrateTable>
</encodeTable>
</StreamInfo>
</StreamInfoList>
</body>
For the Requested Max Encryption protocol protocols, this represents what the client supports
- 00 dc: Request Unencrypted
- 01 dc: Request BCEncrypt
- 02 dc: Request AES
- 03 dc: Request AES (This one encrypts more content like the binary messages too)
The camera of course might not support all these encryption methods so Final confirmed protocol is what the camera says it supports and it will be what needs to actually be used
- 00 dd: Request Unencrypted
- 01 dd: Request BCEncrypt
- 02 dd: Request AES
- 03 dd: Request AES (This one encrypts more content like the binary messages too)
If client requests 03 and camera dosent' support it will revert to 01. If client requests 01 and camera dosen't support it will return 00. Camera will never report a higher number than what client requests. So if client requests 01 but camera supports 03, camera will return 01 as well
For testing start with 00 dc that way no encryption is required (providing your camera allows that)
In your neolink config you can set max_encryption = "none" to force neolink to use unencypted if you want to test (also quite helpful for the wireshark)
Here's the BCEncrypt in rust
const XML_KEY: [u8; 8] = [0x1F, 0x2D, 0x3C, 0x4B, 0x5A, 0x69, 0x78, 0xFF];
let key_iter = XML_KEY.iter().cycle().skip(offset as usize % 8);
key_iter
.zip(buf)
.map(|(key, i)| *i ^ key ^ (offset as u8))
.collect()
The offset is stored in the header
Maybe this, but haven't tested
XML_KEY = [0x1F, 0x2D, 0x3C, 0x4B, 0x5A, 0x69, 0x78, 0xFF]
# .iter means iterate (for loop)
# .cycle means wrap around, so modulo
# .skip means well move forwards so +offset
def bccrypt(buf, offset):
crypted = []
for idx, b in enumerate(buf):
key = XML_KEY[(offset + idx) % len(XML_KEY)]
c = b ^ key ^ (offset)
crypted.append(c)
return crypted
Also its symmetric so test like this where you just feed it back to get the orignal
let zeros: [u8; 256] = [0; 256];
let decrypted = EncryptionProtocol::BCEncrypt.encrypt(0, &zeros[..]);
let encrypted = EncryptionProtocol::BCEncrypt.decrypt(0, &decrypted[..]);
assert_eq!(encrypted, &zeros[..]);
Do I just concanate the username and password directly after each-other (nothing in-between) then cap it at 31 chars with a \0 at the end (or pad with \0 to a lenght of 32 chars). and then do the MD5-hash. results in 16 bytes
Yes concat username + nonce
let concat_username = format!("{}{}", credentials.username, nonce);
let concat_password = format!("{}{}", modern_password, nonce);
let md5_username = md5_string(&concat_username, Truncate);
let md5_password = md5_string(&concat_password, Truncate);
When I look at a wireshark capture of your neolink program, the first message beeing send (which I am hoping is the legacy login) is actually 126 bytes long, so no idea where the other bytes are coming from.....
Assuming your using an older neolink which actually send the md5 hashed password rather than just the header. The legacy login message it is zero padded to 1836 bytes total
if username.len() != 32 || password.len() != 32 {
// Error handling could be improved here...
return Err(GenError::CustomError(0));
}
tuple((
slice(username),
slice(password),
// Login messages are 1836 bytes total, username/password
// take up 32 chars each, 1772 zeros follow
slice(&[0u8; 1772][..]),
))(out)
But as I said earlier it's better to just do no userpass in the legacy
Oh and after I make the MD5-hash, should I then do some encryption of that body? I do see the decrypt methods documented, but I do not see the encrypt method....
Encrypt the body in every message body after the NONCE is received
The extension XML and body XML are encrypted separately and then concatenated
ext_xml = something()
body_xml = something_else()
ext_xml_enc = encrypt(ext_xml)
body_xml_enc = encrypt(body_xml)
magic = [0xf0, 0xde, 0xbc, 0x0a]
message_id = msg_id_for_somehting()
message_length = len(ext_xml_enc) + len(body_xml_enc)
encryption_offset = 1
stream_type = 0
msg_number = get_and_increment_message_number()
status_code = 0
message_class = [0x64, 0x14]
payload_offset = len(ext_xml_enc)
bytes = (
magic
+ message_id.to_bytes(4, byteorder="little")
+ message_length.to_bytes(4, byteorder="little")
+ encryption_offset.to_bytes(1, byteorder="little")
+ stream_type.to_bytes(1, byteorder="little")
+ msg_number.to_bytes(2, byteorder="little")
+ status_code.to_bytes(2, byteorder="little")
+ message_class
+ payload_offset.to_bytes(4, byteorder="little")
+ ext_xml_enc
+ body_xml_enc
)
conn.send(bytes)
I added the more complete header listing given recent discoveries since I made the messages.md
- Magic
- Message length: len(Ext+Payload bytes)
- Encryption Offset/Channel ID
- Stream Type: 00=HD, 01=SD (not in all cameras mostly ignored, seems to be legacy)
- MsgNumber: Links a request from the client with the response from the camera, think of it as the message UID. If you set a value of 100 the camera will reply with the same value. Useful if you want to ensure concurrent messages
- Status code: 200 for success, 400 for unauthorsisd, 500 for Error (Try again later)
- Payload Offset: len(Ext Bytes), payload length is therefore (message length - payload offset)
Thank you so much for all this extra info.
I started testing with only the header for the initial legacy login.
When I use for the encryption '00dc', '01dc', '02dc', '10dc', '11dc' I do not get any response. When I use '12dc' I got my first response form the Reolink NVR !!!, however it does seem to be encrypted.
I am currently using a NVR because that was just conveniant, will take out a seperate camera for testing tonight, maybe that responds diffrently.
What does the encryption '12dc' mean and can I decrypt that withouth knowing the Nonce yet?
You can decrypt with bcencypt without the nonce. You just need the nonce for the password hash and aes
We are not sure what the 12dc means. We have also seen 21dc. I think the first part the 1 is the encryption and the 2 is some other flag.
In an nvr the channel number is more important too. It's zero based so the firs camera is at 0 and the second at 1 (despite how it's labelled as 1 and 2 in the camera view)
When possible maybe you can share your code on github and I'll see if I can test it
@QuantumEntangledAndy thank you so much for the detailed help!
As a little update: In the meantime I have been able to send the header, get back the response, decode it and extract the nonce. I am very happy with the progress.
Next up is the modern login, looking forward to also getting that working.
Once you have the nonce your mostly there, so congratulations. Let me know if you need any more help.
@QuantumEntangledAndy again, thank you so incredebly much for your assistance!
I now have the baichuan protocol fully working (including the AES encryption) in python.
You can see my implementation here: https://github.com/starkillerOG/reolink_aio/pull/80/files I already integrated it into my reolink-aio library such that it will automatically open the HTTPs port when needed. This will make the initial setup for new Home Assistant users even easier!
I will do some additional testing on diffrent camera models and will merge the PR soon.
@QuantumEntangledAndy as you can see in the PR, I added you in the Acknowledgments section of the reolink-aio Readme.
Thanks again for the help. I will close this issue now, hopefully I can reach out again in the future if I need further assistance with the baichuan protocol.
@QuantumEntangledAndy one more question:
Do you know if it is possible to get the 80: <VersionInfo> command for a IPC camera connected to a NVR instead of just the VersionInfo of the NVR itself?
Can you do it with the official app? If you look at version number in the official reolink app does it report the NVR or the camera? If so I can work it out from the wireshark packets
No I did not find this info in the mobile app or desktop client, so I am guessing it is just not there.
I did receive a command ID 145 from my NVR containing a list of the connected channels with their UIDs and some info. That would be really intresting, but I can't seem to figure out how to specifically request the cmd_id 145, it is just beeing send by the NVR upon connection, but if I try to request it I get a 405 status code: method not allowed.