slip radio not sending data to RF channel afetr AES encryption enable in 6lbr

[bbhagasra1]

is there any change in slip radio for AES encryption enable.

[msolters]

This problem still presents itself on the develop branch of 6LBR. When I configure the BR like so:

And then configure my nodes as such:

#undef LLSEC802154_CONF_ENABLED
#define LLSEC802154_CONF_ENABLED           1
#undef NETSTACK_CONF_FRAMER
#define NETSTACK_CONF_FRAMER              noncoresec_framer
#undef NETSTACK_CONF_LLSEC
#define NETSTACK_CONF_LLSEC               noncoresec_driver
#undef NONCORESEC_CONF_SEC_LVL
#define NONCORESEC_CONF_SEC_LVL           7

#define NONCORESEC_CONF_KEY { 0x00 , 0x01 , 0x02 , 0x03 , \
                              0x04 , 0x05 , 0x06 , 0x07 , \
                              0x08 , 0x09 , 0x0A , 0x0B , \
                              0x0C , 0x0D , 0x0E , 0x0F }

I stop seeing any wireless_input traffic in /etc/6lbr/6lbr.log, and the nodes lose their route to the internet. I also get a lot of br-rdc: send failed, slip ack timeout messages.

On the node side, I have started exploring different PRs such as https://github.com/contiki-os/contiki/pull/1425. Should I be recompiling 6LBR against these PRs as well??

[laurentderu]

The inclusion of LLSEC into 6LBR had unforeseen consequences, the previous fix needed modification in the slip-radio, otherwise the slip-radio would drop the packets.

I pushed today a better fix that completely isolate the slip-radio from the LLSEC modifications, could you give a try ? Btw, modifications in #1425 are conflicting with some of the modifications in 6LBR that we made in the past to achieve the same result. It should be unified again in the next merge from Contiki.

[msolters]

I am compiling a fresh develop branch now. Can you please clarify: should I use the slip-radio from the official Contiki master, or should I compile the slip-radio from 6LBR?

[laurentderu]

You can recompile it from 6LBR tree, but a standard slip-radio from Contiki should work too with this fix.

[msolters]

Yes! It works, using the following configuration!

• Raspberry Pi: using a freshly compiled develop branch with your latest commits.
• Slip radio: CC2650 compiled against Contiki master branch
• Node: CC2650, compiled against Contiki master & including https://github.com/contiki-os/contiki/pull/1425 (see above project-conf.h)

A million thanks.

[msolters]

Actually, I have discovered a (in my case severe) limitation. It seems this only works for UDP datagrams up to (about) 47 bytes.

I have a node that is simply emitting UDP datagrams with strings:

char msg[47] = "some fixed text...";
 uip_udp_packet_send(udp_client_conn, msg, strlen(msg));

In this case udp_client_conn corresponds to a UDP connection with the host of 6LBR (a UDP server on the raspberry pi).

• Without encryption, this works fine: even when msg is many hundreds of bytes long.
• When I enable encryption, any msg up to about 46 bytes long reliably is seen by 6LBR and forwarded on as appropriate. However, for any message 47 bytes or longer it becomes highly unreliable -- over 99% loss. It seems that only one UDP packet is encrypted and sent, then decrypted and read by 6LBR, and then no more.

I don't know if this is a 6LBR issue or an issue with Contiki on the devices. All I know is that is encryption-specific.

[laurentderu]

If you don't have slip-radio errors, this looks like more a 6LoWPAN fragmentation issue. The default parameter in Contiki is to drop fragmented packet when another packet, not related to the message being reassembled is received. So, if you have some traffic on your network this could cause all fragmented packet to be dropped. There is a pending PR on Contiki to solve this properly.

A workaround is to change the default policy : in sicslowpan.c, change PRIORITIZE_NEW_PACKETS to 0

[msolters]

I don't think that's what's causing this. If you mean this PR (https://github.com/contiki-os/contiki/pull/1258) it's been merged already.

There is no instance of PRIORITIZE_NEW_PACKETS anywhere in Contiki's master branch.

I wonder if altering the packet's attributes, as was done to fix this issue, somehow caused further problems for the fragmentation assembly process?

[msolters]

So I've made this reproducible. I am using

• CC2650 for the slip-radio (built against Contiki master)
• CC2650 node, running this simple contiki example: https://github.com/msolters/encrypted-udp-test
• Latest develop 6LBR

I also set DEBUG to DEBUG_FULL inside sicslowpan.c in the node, so that I could see what I (think) is actually being sent. I found something very interesting.

Encryption off

In the case of encryption being off for both 6LBR and the node, I see full output from sicslowpan.c debugging, and I see the traffic in the 6LBR log.

Encryption on

Once I turn on encryption (for both) however, I stop seeing the traffic in 6LBR. And, regardless of the length of the message I am sending, sicslowpan.c debug output becomes like this:

Client sending to: [bbbb:0000:0000:0000:0000:0000:0000:0001] (msg: aaaaaaaaaaaaaaaaaaa ) [20]
Client sending to: [bbbb:0000:0000:0000:0000:0000:0000:0001] (msg: aaaaaaaaaaaaaaaaaaa ) [20]
Client sending to: [bbbb:0000:0000:0000:0000:0000:0000:0001] (msg: aaaaaaaaaaaaaaaaaaa ) [20]
Client sending to: [bbbb:0000:0000:0000:0000:0000:0000:0001] (msg: aaaaaaaaaaaaaaaaaaa ) [20]
Client sending to: [bbbb:0000:0000:0000:0000:0000:0000:0001] (msg: aaaaaaaaaaaaaaaaaaa ) [20]
sicslowpan output: sending packet len 46
sicslowpan output: header of len 41
Client sending to: [bbbb:0000:0000:0000:0000:0000:0000:0001] (msg: aaaaaaaaaaaaaaaaaaa ) [20]
Client sending to: [bbbb:0000:0000:0000:0000:0000:0000:0001] (msg: aaaaaaaaaaaaaaaaaaa ) [20]
Client sending to: [bbbb:0000:0000:0000:0000:0000:0000:0001] (msg: aaaaaaaaaaaaaaaaaaa ) [20]
Client sending to: [bbbb:0000:0000:0000:0000:0000:0000:0001] (msg: aaaaaaaaaaaaaaaaaaa ) [20]
Client sending to: [bbbb:0000:0000:0000:0000:0000:0000:0001] (msg: aaaaaaaaaaaaaaaaaaa ) [20]
Client sending to: [bbbb:0000:0000:0000:0000:0000:0000:0001] (msg: aaaaaaaaaaaaaaaaaaa ) [20]

(Client sending to... is my application printing out that it is calling the UDP transmission method, in this case with a msg 20 bytes long.)

So, with encryption, the node keeps claiming it is sending UDP packets (uip_udp_packet_send()), but sicslowpan.c only claims an IP packet is being sent after very many such calls, and it reports that the packet being sent is 46 chars long, regardless of msg length! It does not indicate any fragmentation (which does occur without encryption), and I don't see the traffic on 6LBR.

[laurentderu]

I tried using your code and latest Contiki, I have the same issue, the few frames coming out of the node are wrong (e.g. the frame type is set to beacon iso data) and incoming data is corrupted either when entering 6LoWPAN layer (wrong dispatch byte).

When I recompile your code in 6LBR tree, it works without issue using encryption (You need an additional define : #define LLSEC802154_CONF_SECURITY 1) and the frames are correctly decrypted by Wireshark, so I believe they are correct.

So I think there is a bug in the latest llsec framer.

[msolters]

I'm having a hard time getting that code to run by compiling under 6LBR as you asserted.

~~I can compile by changing the BOARD=sensortag (without /cc2650), and it flashes just fine to a sensortag, but I get no response. No UART output via debugger pack, nor any apparent radio traffic.~~

~~Same goes for compiling for BOARD=srf06. Neither of these boards give any UART output.~~

~~This is true across all the examples. The one I provide here, ipv6/slip-radio, cc26xx-demo, etc. If I compile under 6LBR develop the cc2650 just does nothing.~~

~~Have you made any changes to your CC26XX source under your tree? Did you add or change any additional configuration data?~~

UPDATE: I can get UART working on the srf06 board. So my results are:

• security disabled: I get radio traffic in the 6LBR router
• security enabled: no radio traffic detected by 6LBR

[msolters]

@laurentderu, do you think you could confirm that those packets you sniffed with Wireshark are actually showing up in 6LBR's log?

[laurentderu]

I don't think you can see them in 6LBR log, as the frame are non standard, I think the framer drops them. (Or maybe wireshark displayed garbage, that's always a possibility). I'll try again in Cooja to see what's going on

[msolters]

Yes the problem here is that compiling this test application under 6LBR causes total packet failure on my side.

And worse, even if compiling the node applications under 6LBR worked, it would not be a viable solution because the Contiki master branch contains many, many hardware & driver improvements such as energy usage that are not present in 6LBR.

Turning on encryption in the 6LBR router causes me to miss

• 100% of packets send from a 6LBR-compiled app
• 100% of packets 46 bytes and greater in a Contiki-compiled app.

[laurentderu]

I'm missing something obvious because, pinging using 200 bytes payload is working with encryption as well as CoAP in the 6LBR non regression tests

[msolters]

Ok;

1. would you mind showing me the exact project-conf.h you are using?
2. you are using a fresh clone of the latest develop branch of 6LBR for this testing?
3. what tree was your slip-radio compiled under?

Pursuant to the third point here I would like to point out that under no conditions can I get any application compiled under 6LBR tree with BOARD=sensortag to work with UART, which means I cannot compile slip-radio under 6LBR for the sensortag.

[laurentderu]

It's indeed the latest develop from github, and the project-conf.h is available in examples/6lbr-demo. I modified it to enable llsec and use level 7.

[msolters]

I really don't understand. I built 6lbr-demo as you suggest, WITH_LLSEC=1 (and security level changed to 7), and it does not function with encryption at all. There must be some difference in our configurations!

Could you answer the third question? Were you using a slip-radio built under Contiki or 6LBR? I have tried both with no success. As soon as encryption is turned on in the 6LBR web control panel, I stop receiving both encrypted and non-encrypted traffic.

[msolters]

So I've just recompiled 6LBR itself from a fresh clone, on a different PC. As soon as encryption is enabled on the client node, the slip radio stops reporting any wireless input, regardless of whether encryption is enabled in the 6LBR web console.

I'm becoming fairly confident this is a slip radio localized problem, which brings me back to my question as to what you are using and under what branch you are compiling the slip radio software to get a working configuration.

[laurentderu]

The error was actually quite trivial, in your project-conf.h, you don't define the NETSTACK_FRAMER as noncoresec_framer, and so your frame are not encrypted/decrypted at all.

If you add :

#undef NETSTACK_CONF_FRAMER
#define NETSTACK_CONF_FRAMER noncoresec_framer

it works properly using your example and latest contiki master.

To answer your third question, I tested with a pure contiki slip-radio, the purpose of the original fix was to be able to use again standard slip-radio even with LLSEC enabled.

[msolters]

Yes, you are absolutely correct -- that line is critical! Unfortunately, that does not fix the actual problem here. (Notice even that even in my first comment here, https://github.com/cetic/6lbr/issues/115#issuecomment-188511267, I was using noncoresec_framer.)

Ensuring I do not forget that that line again, the behaviour is as follows:

• I can see in sicslowpan.c that my encrypted packets are now being fragmented and sent! Before I wasn't even getting those messages.
• However, messages 47 characters and longer are still "ignored" by 6LBR. They just don't show up as wireless_input, where all messages less than 47 chars are received.

[laurentderu]

Indeed, I didn't no check and so I kept the default packet size of 20 byte, hence I did not test the same setup as you (It's easy to lose track of thing in a lengthy discussion). I will retest and come back to you

[laurentderu]

Actually, it's a Contiki bug in LLSEC (see https://github.com/contiki-os/contiki/issues/1537)

A quick workaround is to comment the condition in add_security_header() in noncoresec.c

[msolters]

Almost! Commenting out the condition as suggested does fix the issue insofar as now I can transmit UDP datagrams many hundreds of bytes.

BUT... It requires me to restart the slip-radio! Here's an example test:

1. I am running the encrypted-udp-test program with MSG_LENGTH=50 or some other number
2. I am receiving the encrypted, fragmented packets correctly in 6LBR
3. I recompile encrypted-udp-test program with a different MSG_LENGTH. Longer or shorter, it does not matter. Let's just say 200.
4. All of a sudden, I stop receiving the new packets.
5. If I restart 6LBR, I start receiving the packets again!

I wonder if the modifications we've made here to 6LBR cause it to hang if we receive some of the fragments of a packet but not all of them? Which is to say, when I flash the modified binary to my sensortag, it might interrupt the uC mid-packet transmission, and so the slip-radio is waiting for another fragment. But that fragment never comes, so we must force-restart it by restarting 6LBR?

[laurentderu]

Well, that looks like the normal behaviour of 'noncoresec'. The anti replay mechanism of 'noncoresec' does not support node reboot, the whole network has to be restarted. There is a workaround in the develop branch of 6LBR that should help, but it's not the best. (What's strange is that it should have worked in your case) A new llsec layer, adaptivesec, that will solve this issue correctly, should be included soon in Contiki.

Btw, the slip-radio, rdc, csma and sicslowpan are all protected by timeouts, so the BR or the slip-radio should never get stuck.

[msolters]

Aha! Well then, it seems we just have this one bug in Contiki to contend with to get noncoresec functional with 6LBR and we can close this issue.

However, I do have one final question regarding adaptivesec. I have looked at that source over at https://github.com/kkrentz/contiki. My question is, would not that layer have to be merged into 6LBR itself as well? It seems fairly trivial to pull this into Contiki but I'd imagine 6LBR will require a bit more configuration?

[msolters]

I've been playing around with the two new settings in develop -- disable anti-replay & anti-replay reboot workaround.

Turning on both features, I can reboot a node once and it will then rejoin the mesh! Awesome!

~~But if I reboot it again, it does not work. It stops showing up as wireless_input in the 6LBR log.~~

It seems the device will sometimes or perhaps eventually rejoin the mesh, but it can take many minutes sometimes. I waited for at least 10 minutes once with no luck, but in other cases the second reboot takes about 5 minutes for the device to be recognized again.

Is this perhaps the result of some sort of "backing off" time-dependent factor?

[laurentderu]

Actually you have to also disable the anti-replay, or copy the workaround, in the mote too. Otherwise the mote will reject the 6LBR until the counter reach the previous value.

[laurentderu]

Integrating adaptivesec would solve the issue properly, but it's not a trivial merge. So it has to wait the release of 1.4.0 and the following merge.

[msolters]

OK, I understand. Well, how would I approach implementing the anti-replay fix in the motes (other than compiling under the 6LBR tree)? There's a lot of commits so I'm not too sure what specific changes I should be folding in.