Arduino Uno Serial read seems to be dropping bytes

[bhansconnect]

I am new to embedded rust in general and have just been messing around with the Arduino Uno examples. I noticed something off with the examples that use serial communication (uno-usart.rs and uno-millis.rs). They seem to only be able to read a 4 bytes per line sent.

So with the uno-usart.rs example run, I get:

Hello from Arduino!
a
Got 97!
Got 10!
abc
Got 97!
Got 98!
Got 99!
Got 10!
abcd
Got 97!
Got 98!
Got 99!
Got 10!
abcdefg
Got 97!
Got 98!
Got 99!
Got 10!

If you look at the above examples, you can see that the answers are wrong starting with abcd. The values returned are for a, b, c, and \n. Nothing after abc every prints a value. This is the case for the later examples as well. Any ideas what is going on here? Can you reproduce?

Note: My environment was created by using the generator at https://github.com/Rahix/avr-hal-template.git and then copying over the serial example code. The code is running in release.

[Rahix]

Hmm, I'm not sure if this is really a problem with the serial implementation or rather a problem with the way it is used in the example. The relevant code is

https://github.com/Rahix/avr-hal/blob/92d540c4716a1360c8b513f067d75c318318693a/examples/arduino-uno/src/bin/uno-usart.rs#L20-L26

I think there is a big problem here: After receiving a single character, this loop blocks on the echoing write (which takes ~7x longer than a single byte read) before reading the next byte. Thus, we probably loose input characters in the meantime when more than one char is sent at a time.

I think there are two actionable items here:

1. The example should probably be rewritten to read in an entire line and then echo it once instead of this character-wise approach. This will hopefully nudge users in the right direction when copying the example into their own projects.
2. The serial driver needs to report an overrun error. I think it currently does not check for such conditions at all...

[bhansconnect]

So just for testing, I changed to using a ring buffer. I am able to successfully deal with more charters. The way I implemented things, it still is very easy to overflow the buffer, but with some smarter prioritization about when to read vs write, I think it would work quite well. I guess I should have realized that unlike arduino, rust does not automatically deal with buffer and such.

[stappersg]

On Tue, Feb 08, 2022 at 10:33:17PM -0800, Brendan Hansknecht wrote:

... I am able to successfully deal with more charters. ...

Consider to share that with the community you came to consult.

[bhansconnect]

Sure, though I don't think it is particularly useful for other. It is just a simple ring buffer for serial. The buffer needs to be dealt with in a smart way for it to be useful. Currently if you overflow, the program is essentially setup to halt. For a practical program, in the case of an overflow, an interrupt probably should be triggered that pauses execution until a character is drained from the buffer. I am not fully sure the proper solution, but I am sure it could be built by analyzing what arduino or others do.

Anyway, here is a gist of the version with a buffer: https://gist.github.com/bhansconnect/2dcc87ee2db96abb1317f5839aaaf3dc

[bhansconnect]

An extra follow up. This is roughly what Arduino does:

They have 1 interrupt for receiving data. It will receive 1 character of data and put it into their receive buffer. If the buffer is full, it will be ignored. This code is only ever triggered by the hardware interrupt. They have 1 interrupt for sending data. It will send a character but stay enabled if the buffer still has more data in it (Should trigger again once the character has sent and usart is ready to send, iiuc). This code can be called from other locations in the code, like write and flush. In those cases, extra checks need to be made to not get messed up by interrupt handling. Write will only call it if the output buffer is full. It will block the user code until it has sent a character and then proceed writing.

This has a bit of a description of the design: https://arduino.stackexchange.com/questions/77647/what-is-the-logic-behind-arduino-inlining-hardwareserial-rx-complete-irq-f#77650

I am new to embedded hals. Do they have a standard way to deal with serial interrupts and such? I would assume that they wouldn't expect every user to define serial interrupts and ring buffers, but maybe that was a design/tradeoff decision.

[bhansconnect]

One more follow up that at least address some of what can be done in userland. This gist is a bit more proper. It sets up an rx recieve interrupt. This way, data isn't dropped while the processor is blocked sending out bytes. The interrupt just fills a buffer. If the buffer is filled, I currently have it flashing an led instead of just ignoring the byte.

The code is definitely not robust and to rust standards (use mutable static and unsafe instead of mutex/cell/etc), but it at least shows half of what is need to match what arduino does. This also acts equivalent to how arduino would act when the tx buffer is full. Serial write call become blocking, but the receiving can still happen via interrupt and buffer.

https://gist.github.com/bhansconnect/35f0a93c4c6c009728a8419d2701d615

For this bug, I am not sure specifically how you want it solved (interrupts, just buffer a full line, library changes, some mix), but with a bit of pointers on the proper goal, I may be able to work on a fix. Generally I think that at least the rx interrupt is pretty important, otherwise you may lose data while spending too much time on other tasks. Thoughts?

[devcexx]

I've found myself in this situation when I tried to read full lines in a non-blocking way while writing data synchronously to a LED Strip. It doesn't take too much time to do so (less than 0.5ms probably?), but it is enough to make the serial port start loosing data when using a "high" baud rate (in my case, 57600). Trying to do something like this:

fn main() {
  // initialization
  loop {
    read_all_serial_data();
    do_stuff_with_serial_data();
    write_led_strips();
  }
}

I think that for those kinds of workloads when your Arduino is just doing stuff until receives a command from a serial that interrupts it would be beneficial to have an implementation that uses a small ring buffer for temporarily storing received data, as the original implementation that Arduino does (ref: https://github.com/arduino/ArduinoCore-avr/blob/master/cores/arduino/HardwareSerial.h#L113).