UsbSerial icon indicating copy to clipboard operation
UsbSerial copied to clipboard

Problem with the read callback from Arduino

Open OdysseusU opened this issue 8 years ago • 49 comments

Hi,

I have an issue in the read callback when connecting the arduino. It seems that I receive my bytes twice and sometime the callback is triggered but the package received is empty.

I only send a byte array with the arduino every 2 seconds and I just print what we receive with android.

Also I don't understand how the package is received. Sometimes I can get the whole array in one simple array and sometimes it's received truncated and the callback is called several time to get the whole array.

Thank you

OdysseusU avatar Mar 22 '16 17:03 OdysseusU

Hi, In some devices data is split into more than callback because of the usb to serial chipset. You can onvercome this using two approaches:

  • Append the received data into a buffer and use it when it makes sense.
  • Use the sync vdrsion of the library (added in the last version). Methods are called like syncOpen, syncRead, syncWrite...With those methods you should be able to specify how many bytes you want to receive.

felHR85 avatar Mar 22 '16 17:03 felHR85

Thank you for your fast reply! I had already been buffering but it was troublesome. I'll see with the sync version.

OdysseusU avatar Mar 22 '16 18:03 OdysseusU

It is the last thing I added so it would be nice to receive your feedback about if the sync version works correctly. What USB-Serial converter does your arduino use?

felHR85 avatar Mar 22 '16 19:03 felHR85

I will work on that today and get to you after that. I am using the last Arduino UNO which uses their own firmware for USB-Serial converter on their Atmega8U2 chip but it should be like a FTDI chip.

OdysseusU avatar Mar 23 '16 09:03 OdysseusU

So after doing some tests with the sync version, it seems to be working but I get a lot of 0 in between my bytes when I (sync)read. I suppose that's because i limited my byte array to 13 bytes and i just receive a certain amount below 13. The thing is that I send from the arduino exactly 13 bytes. So I still get a truncated packaged received from the Arduino.

I did a simple test where I write 13 bytes from android every 2 seconds and on the arduino it sends s13 bytes whenever it receives something (13 bytes). And everytime android sends something it reads in a while loop (with a timeout) and prints what it received.

in summary : Android --> Arduino --> Android .........13 bytes.....13 bytes.........

I will look further to see how I can improve my code.

OdysseusU avatar Mar 23 '16 11:03 OdysseusU

Java arrays are set to 0 by default so probably you are receiving less than 13 bytes when the timeout raises.

Are you using UsbSerial 4.1.1 because that release solved an issue with the synchronous read in FTDI devices.

I would check the arduino code too.

felHR85 avatar Mar 23 '16 13:03 felHR85

Yes I am using the final release.

So yes i suppose that it was because i was receiving less than what i expected that I had the 0s.

But the biggest problem comes from receiving multiple package when sending only one. And this is the case for both methods (sync and callback). How would you suggest using the sync version the best way? Having a timeout seems to be the problem. Because the data could arrive at the end of the timeout, so it would cut the received data. And I can't use flow control.

The code on Arduino seemed fine. I went back with the callback version because it was simpler to work with and improved my buffering to get what I want. But it could be great to have the received data in a whole.

I suppose the only solution is to buffer the received package.

OdysseusU avatar Mar 23 '16 14:03 OdysseusU

Set timeout to 0 :)

felHR85 avatar Mar 23 '16 14:03 felHR85

OdysseusU, did you get the synchronous api to work? i wanted to attempt using it since I only need to keep reading a string of 13 characters from the Arduino.

jstonis avatar Apr 04 '16 04:04 jstonis

Yes do you have any problem with it? I didn't really have any. The main problem was that my frames were cut but that is device related.

If you want to use synchronous api just put a while in there with a syncRead and changing everything open by syncOpen and close by syncClose. This should be good.

OdysseusU avatar Apr 04 '16 12:04 OdysseusU

Seems i do..so i only get one reading and thats it..i put the syncRead in the broadcastReceiver...not in a while loop just in there and then used tvappend to append to display..1st error is decoding ..not converting to string, 2nd error is getting just 1 read and thats it

jstonis avatar Apr 04 '16 12:04 jstonis

The decoding is a bit strange with android. Look into other methods to convert into string (don't use the method .toString()) rather create your String and put your encoding ( new String(bytes, encoding UTF8 for instance). I don't really remember how to catch the string again but I think the .toString() method isn't good.

And the reading problem, are you sure your Arduino sends the data multiple times? Did you put the timeout to 0? this caused me some problem since it waits until the byte arrives and sometimes stops completely the application if nothing is sent (even if it was). Put a short timeout to be sure.

But generally the library works as it was intended to. So the problems you encounter comes from java or your arduino.

OdysseusU avatar Apr 04 '16 14:04 OdysseusU

So.i did that and it cuts my string..it produces the string but only a quarter of it...atrange..its still splitting up the data

jstonis avatar Apr 04 '16 14:04 jstonis

Idk why it still splits it up..but it does. I set the size of the array to 14, because the string im expecting is A: 000 B: 000...so idk what to do

jstonis avatar Apr 04 '16 14:04 jstonis

Yup i've got the same problem. Your solutions are :

  • Set timeout to 0 (or longer to catch all the bytes) and wait to have all your bytes
  • Set a buffer which adds up every bytes you have until you get your 13 bytes.

I took the second solution because i couldn't afford to wait.

This is not library related as every library that does the serial with arduino does the same. It's device related (or Android related, don't know).

OdysseusU avatar Apr 04 '16 16:04 OdysseusU

So you really could use the async..but i.noticed i was losiglng data..so you read until you have 13 bytes..what if you receive extra bytes so its the next set of data..

jstonis avatar Apr 04 '16 16:04 jstonis

Well you have to know when to stop the reading and do buffering right. But I never lose the data (only 1 every minute or so when sending every 50 - 100 ms) But yeah I went with the async version.

OdysseusU avatar Apr 04 '16 16:04 OdysseusU

So you do lose a byte every min? Can i see how you did it

jstonis avatar Apr 04 '16 16:04 jstonis

I won't put the whole code here it would be useless since it's quite specific what I do. But here is a sample in the read callback, b being the bytes received : `

    if(b != null){
        if(b.length > 0){
            if (isStartByte(b[0])&& !startbytefound) { //look if its a new frame
                startbytefound = true;
                clearBytes(); // clears my buffer
                iNBbyte = /* formula specific to my use, just put the number of bytes you need */;
            }
            appendBytes(b):
            if(bufferSize >= iNBbyte && startbytefound){

                bufferSize = iNBbyte;
                byte[] buf = new byte[iNBbyte];
                System.arraycopy(buffer, 0, buf, 0, bufferSize);
                clearBytes();
                appendBytes(buf); // append to my buffer

                checkData(); //process the data
                startbytefound = false;
                isAck = false;
            }
        }
    }`

What I do is quite simple. I check if i have the start data (in your case 'A') then i begin to append the data in the buffer. If i have what i need (bufferSize >= iNBbyte) then i process it. I do all these checks (b.length > 0 or b != null) because sometimes i get empty bytes or bytes of length too short, and even sometimes i get the data 2 times.

OdysseusU avatar Apr 04 '16 17:04 OdysseusU

okay i tried something similar to this algorithm. in your arduino code, do you do serial.println..so theres a new line in the string, or just serial.print?

jstonis avatar Apr 04 '16 17:04 jstonis

Actually i use serial.write to write byte code. But you could serial.println and find the ´\n' character to find the end of frame

OdysseusU avatar Apr 04 '16 17:04 OdysseusU

It is actually simpler since you just need to find if your received bytes contains the ´\n' character and if it does process the buffer you have. But it is more optimized to not search for the ´/n' character if you already know the size of the frame you should receive.

OdysseusU avatar Apr 04 '16 17:04 OdysseusU

So can i just stick to the way you did it with serial.println in arduino?

jstonis avatar Apr 04 '16 18:04 jstonis

Yes the serial.println all it does is serial.print +'\n'

OdysseusU avatar Apr 04 '16 18:04 OdysseusU

byte [] data=new byte[0]; byte iNBbyte;

byte[] buffer=new byte[13];
int bufferSize=13;
boolean isAck=false;

public boolean isStartByte(byte firstChar){
    if(firstChar=='A'){
        return true;
    }
    else{
        return false;
    }
}
public void clearBytes(){
    data=null;
}
public void appendBytes(byte[] b){
   byte[] one=buffer;
    byte[] two=b;
    byte[] combined=new byte[one.length + two.length];
    for(int i=0; i<combined.length; i++){
        combined[i]=i<one.length?one[i]: two[i-one.length];
    }

    buffer=combined;

}
public void checkData(){
   tvAppend(errorCheck,buffer.toString());
}



UsbSerialInterface.UsbReadCallback mCallback = new UsbSerialInterface.UsbReadCallback() { //Defining a Callback which triggers whenever data is read.
    @Override
    public void onReceivedData(byte[] arg0) {

        data = null;

           // data = new String(arg0, "UTF-8");
            data=arg0;

            //try this for now

            if(data!= null){
                if(data.length > 0){
                    if (isStartByte(data[0])&& !startbytefound) { //look if its a new frame
                        startbytefound = true;
                        clearBytes(); // clears my buffer
                        iNBbyte =13; /* formula specific to my use, just put the number of bytes you need */;
                    }
                    appendBytes(data);
                    if(bufferSize >= iNBbyte && startbytefound){

                        bufferSize = iNBbyte;
                        byte[] buf = new byte[iNBbyte];
                        System.arraycopy(buffer, 0, buf, 0, bufferSize);
                        clearBytes();
                        appendBytes(buf); // append to my buffer

                        checkData(); //process the data
                        startbytefound = false;
                        isAck = false;
                    }
                }
            }

what am i doing wrong???

jstonis avatar Apr 04 '16 20:04 jstonis

It's my fault I didn't detailed my variables. clearBytes() should clear the buffer and not data. also bufferSize should be the actual bufferSize (which would be buffer.length) and iNBbytes the threshold at which you launch the check data (iNBbytes should be constant in your case at 13). Also are you sure byte=='A' works? I don't know if it gives the same byte, but that i'm not sure. And you don't need data as it's only used as a local variable here. And finally, to copy the array you should use System.arraycopy which is faster and simpler too.

so the code would be : ` int iNBbyte = 13; byte[] buffer; int bufferSize;

public boolean isStartByte(byte firstChar){
    if(firstChar=='A'){ // check if it works
        return true;
    }
    else{
        return false;
    }
}
public void clearBytes(){
    byte[] buffer=new byte[13];
    bufferSize = 0;
}
private void appendBytes(byte[] buf){
    System.arraycopy(buf, 0, buffer, bufferSize, buf.length);
    bufferSize += buf.length;
}
public void checkData(){
   tvAppend(errorCheck,buffer.toString());
}

UsbSerialInterface.UsbReadCallback mCallback = new UsbSerialInterface.UsbReadCallback() {
//Defining a Callback which triggers whenever data is read.
@Override
public void onReceivedData(byte[] arg0) {
        if(arg0!= null){
            if(arg0.length > 0){
                if (isStartByte(data[0])&& !startbytefound) { //look if its a new frame
                    startbytefound = true;
                    clearBytes(); // clears my buffer
                }

                appendBytes(arg0);

                if(bufferSize >= iNBbyte && startbytefound){
                    bufferSize = iNBbyte;
                    byte[] buf = new byte[iNBbyte];
                    System.arraycopy(buffer, 0, buf, 0, bufferSize);

                    checkData(); //process the data
                    startbytefound = false;
                }
            }
        }`

Maybe that would work. Check every passage with a Log.i() to see if everything's ok.

OdysseusU avatar Apr 05 '16 08:04 OdysseusU

hm..its still not working.. i mean its not displaying the string!

jstonis avatar Apr 06 '16 04:04 jstonis

for clearBytes..you mean to set the global variable buffer to new byte[13] right?

jstonis avatar Apr 06 '16 14:04 jstonis

Oh it looks like youre right..it never returns true for when byte==character..hmm im not sure how to do that conversion.

jstonis avatar Apr 06 '16 15:04 jstonis

Right i let a byte[] buffer you can erase byte[] it was an error. Well you can log the bytes you receive (their values) knowing that you have an A coming just take the value and use that. if for example you see that your 'A'=42 then use firstChar==42.

OdysseusU avatar Apr 06 '16 15:04 OdysseusU