MKR1400 hangs when running the GsmWebClient sketch

[lathoub]

My MKRGSM1400 hangs after a random amount of time (sometime after a couple of seconds, sometimes after an hour).

The Code (based on the GsmWebClient example, but then 'get' is put in a loop). The Serial Monitor provides a trace (see below)

If I want to upload new code to the MKR1400, I need to reset it first using the little reset button to 'unhang' it.

I'm using a MacBook Pro (15-inch, 2017) with High Sierra (10.13.4 (17E199)), Thunderbolt 3 to USB dongle. The MKR1400 has a charged 850mAh battery attached to it (I also tried a 2000mAh battery, same result).

Used both Arduino IDE 1.8.5. and 1.9.0 Beta - same result, hangs at random places during the run.

#include <MKRGSM.h>

#include "arduino_secrets.h"
// Please enter your sensitive data in the Secret tab or arduino_secrets.h
// PIN Number
const char PINNUMBER[]     = SECRET_PINNUMBER;
// APN data
const char GPRS_APN[]      = SECRET_GPRS_APN;
const char GPRS_LOGIN[]    = SECRET_GPRS_LOGIN;
const char GPRS_PASSWORD[] = SECRET_GPRS_PASSWORD;

// initialize the library instance
GSMClient client;
GPRS gprs;
GSM gsmAccess(true);

// URL, path and port (for example: arduino.cc)
char server[] = "arduino.cc";
char path[] = "/asciilogo.txt";
int port = 80; // port 80 is the default for HTTP

void setup() {
  // initialize serial communications and wait for port to open:
  Serial.begin(9600);
  while (!Serial) {}

  Serial.println("Starting Arduino web client.");
  // connection state
  boolean connected = false;

  // After starting the modem with GSM.begin()
  // attach the shield to the GPRS network with the APN, login and password
  while (!connected) {
    if ((gsmAccess.begin(PINNUMBER) == GSM_READY) &&
        (gprs.attachGPRS(GPRS_APN, GPRS_LOGIN, GPRS_PASSWORD) == GPRS_READY)) {
      connected = true;
    } else {
      Serial.println("Not connected");
      delay(1000);
    }
  }
}

void loop() {

  Serial.println("connecting...");

  // if you get a connection, report back via serial:
  if (client.connect(server, port))
  {
    Serial.println("connected");
    // Make a HTTP request:
    client.print("GET ");
    client.print(path);
    client.println(" HTTP/1.1");
    client.print("Host: ");
    client.println(server);
    client.println("Connection: close");
    client.println();

    Serial.println("awaiting response");

    while (true) {
      // if there are incoming bytes available
      // from the server, read them and print them:
      if (client.available())
        char c = client.read();

      // if the server's disconnected, stop the client:
      if (!client.available() && !client.connected()) {
        Serial.println();
        Serial.println("disconnecting.");
        client.stop();
        break;
      }
    }
  }
  else
  {
    // if you didn't get a connection to the server:
    Serial.println("connection failed");
  }

  delay(5000);
}

The content of the serial monitor:

21:49:19.604 -> Starting Arduino web client.
21:49:23.393 -> ⸮AT

21:49:23.605 -> OK
21:49:23.605 -> AT+IPR=921600

21:49:23.605 -> OK
21:49:23.712 -> AT

21:49:23.712 -> OK
21:49:23.712 -> AT+UPSV=3

21:49:23.712 -> OK
21:49:23.817 -> AT+CPIN?

...

21:51:57.115 -> +UUSORD: 0,2
21:51:57.115 -> AT+USORD=0,1

21:51:57.115 -> +USORD: 0,1,"20"
21:51:57.115 -> 
21:51:57.115 -> OK
21:51:57.115 -> 
21:51:57.115 -> +UUSORD: 0,1
21:51:57.115 -> AT+USORD=0,1

21:51:57.115 -> +USORD: 0,1,"0A"
21:51:57.115 -> 
21:51:57.115 -> OK
21:51:57.369 -> 
21:51:57.369 -> +UUSOCL: 0
21:51:57.369 -> 
21:51:57.369 -> disconnecting.
21:52:02.352 -> connecting...
21:52:02.463 -> AT+USOCR=6

21:52:02.463 -> +USOCR: 0
21:52:02.463 -> 
21:52:02.463 -> OK
21:52:02.675 -> AT+USOCO=0,"arduino.cc",80

21:52:02.961 -> OK
21:52:02.961 -> connected
21:52:02.961 -> AT+USOWR=0,4,"47455420"

21:52:02.961 -> +USOWR: 0,4
21:52:02.961 -> 
21:52:02.961 -> OK
21:52:02.961 -> AT+USOWR=0,14,"2F61736369696C6F676F2E747874"

21:52:02.961 -> +USOWR: 0,14
21:52:02.961 -> 
21:52:02.961 -> OK
21:52:02.961 -> AT+USOWR=0,9,"20485454502F312E31"

21:52:02.997 -> +USOWR: 0,9
21:52:02.997 -> 
21:52:02.997 -> OK
21:52:02.997 -> AT+USOWR=0,2,"0D0A"

21:52:02.997 -> +USOWR: 0,2
21:52:02.997 -> 
21:52:02.997 -> OK
21:53:05.524 -> 
21:53:05.524 -> +UUSOCL: 0

A regular successful sequence would be:

connecting...
AT+USOCR=6
+USOCR: 0
OK
AT+USOCO=0,"arduino.cc",80
OK
connected
AT+USOWR=0,4,"47455420"
+USOWR: 0,4
OK
AT+USOWR=0,14,"2F61736369696C6F676F2E747874"
+USOWR: 0,14
OK
AT+USOWR=0,9,"20485454502F312E31"
+USOWR: 0,9
OK
AT+USOWR=0,2,"0D0A"
+USOWR: 0,2
OK

processing broke here and should normally continue with the messages below. Yet, +UUSOCL: 0 is the last message before the MKR1400 hangs

AT+USOWR=0,6,"486F73743A20"
+USOWR: 0,6
OK
AT+USOWR=0,10,"61726475696E6F2E6363"
+USOWR: 0,10
OK
AT+USOWR=0,2,"0D0A"
+USOWR: 0,2
OK
AT+USOWR=0,17,"436F6E6E656374696F6E3A20636C6F7365"
+USOWR: 0,17
OK
AT+USOWR=0,2,"0D0A"
+USOWR: 0,2
OK
AT+USOWR=0,2,"0D0A"
+UOWR: 0,2
OK
awaiting response

[lathoub]

Here the MKR1400 stalled during reading of the response:

+UUSORD: 0,241
AT+USORD=0,1
+USORD: 0,1,"74"
OK
+UUSORD: 0,240
AT+USORD=0,1
+USORD: 0,1,"65"
OK
+UUSORD: 0,239
AT+USORD=0,1
+USORD: 0,1,"6E"
OK
+UUSORD: 0,238
AT+USORD=0,1
+USORD: 0,1,"74"
OK
+UUSORD: 0,237

[lathoub]

I bought another brand new MKR1400 from the Arduino store, hoping the above hanging just due to hardware failure - nope! New unit hangs as well.....

What is the ideal power setup? Am I the only one experiencing hangs/freezes?

Here is the log:

OK

+UUSORD: 0,119
AT+USORD=0,1

+USORD: 0,1,"67"

OK

+UUSORD: 0,118
AT+USORD=0,1

+USORD: 0,1,"74"

OK

+UUSORD: 0,117
AT+USORD=0,1

+USORD: 0,1,"68"

OK

+UUSORD: 0,116
AT+USORD=0,1

+USORD: 0,1,"3A"

OK

+UUSORD: 0,115

[FrancMunoz]

Hi @lathoub , Please take a look at #10, I think it could help you. All in all update your libraries to last version.

Hope it helps!

[lathoub]

Thanks @FrancMunoz

When reading #10, I understand that the issue is still under investigation (with some pointers how to solve the issue). Let me try to include your alternatives.

Not sure if I understand the 'update my libraries'. In the sketch above I don't use any libraries at all (except for MKRGSM.h)

[FrancMunoz]

Hi @lathoub , with 'update your libraries' I mean make sure you use last version of MKRGSM and also SAMD core. You can update both with arduino IDE first under Libraries Manager, second under Boards Manager. More information about SAMD core:

Arduino SAMD Core

[lathoub]

Roger! I'm running 1.6.18 for SAMD core and 1.1.2 of MKRGSM.

I'm also reading AT command examples trying to understand better how the U201 socket reading works. Looks like to U201 gives an indication of the bytes still unread. I believe you suggest in #10 for a small buffer to fasten the reading. I'll look into that as well later this week.

[lathoub]

I got TinyGSM to work with the MKR GSM and is running stable for 3 hours now (doing HTTP GET) - I'll leave it overnight and see it if freezes or not.

[lathoub]

Pull request to the example: https://github.com/vshymanskyy/TinyGSM/pull/161

[Prof62]

+1 on an urgent fix for this.

I'm using the current release version of the SAMD core (1.6.18) and the MKRGSM library (1.1.2) but still experience random hangs after anywhere between a few seconds and a few hours.

Issue #10 is closed so presumably there is no official active investigation going on there (Although there are a few recent requests to reopen it) though it has been merged with issue #14 which is still open but relates to a slightly different issue and hasn't had any activity since the merge point in February.

The buffering approach described by @FrancMunoz in issue #10 does not resolve the problem but his delay solution is looking promising so far. Though, given the random nature of the problem it's going to take a few more days of testing to be certain.

Until this is fixed the MKR GSM 1400 is virtually useless to me as an IOT device which is a pity because it has great potential.

[FrancMunoz]

Hi! I've found the cause of the infinite loops (pointing to hangs). The GSMClient::connect() function aren't timed out so if something goes wrong (for example some URC like +CREG: 2 is received while connecting -> that means module is not registered) the loop never ends and that's it... a hang.

So, I've updated Modem.cpp to handle 20ms just waiting for the difference between loops, as I already suggested (that is, just subtracting to 20 the main loop time), and also GSMClient.cpp adding timeouts to all while (ready() == 0) loops, and at this time several days without a hang even disconnecting and reconnecting the antenna... I've to say that the sketch code has to be prepared for it negative connect calls... Until this last change when SARA module was registered no hangs ocurred.

I've attach my testings it would be great If you could test and post results... to improve.

MKRGSM.zip

I would like to collaborate with this libraries making PRs, but I guess my thinkings are on other way and I'm a litle bit affraig of making it.

All in all @Prof62, MKR is a nice board, we only need to improve the code and make it stable, I'm totally agree with you. Arduino team is triying to make it very nice... while we only need it to be stable.

Hope it helps!

[lathoub]

Hi @FrancMunoz I appreciate your positiveness, but to be honest: MKR 1400's are sold under the Arduino brand, so I expect quality and active software development. @Prof62 is right, currently the MKR 1400 is useless

@agdl you work for Arduino (looking at your profile), what is the roadmap to fix this show-stopper issues?

[FrancMunoz]

Hi! Yes, I told you: I'm totally agree with @Prof62 and you (I got very frustrated seeing MKR hanging). I'm just trying to solve that problem: you see I'm not working nor I'm part of Arduino team... so my interest is purely to solve that annoying issues and help if it's possible.

I've just made some changes to MKRGSM library:

MKRGSM.zip

This one is able to loose registration to network (deattaching and reattaching the antenna), it doesn't hangs for me. I guess even with this situation somebody will test it.

So, @lathoub or @Prof62 could you please test the library attached to this message, it will help me to know if I'm on the right way?

Thank you!

[Rocketct]

Hi all, i start to work on this issue, i looking into reproducing the problem.

[FrancMunoz]

Hi @Rocketct, AFAIK the issue appears randomly and it's due to SARA module does not send any response to MKR, that makes infinite loops (waiting data) in code. The library I attached above, the last one, solve this issue adding timeouts and taking control if something goes wrong (that happens too if you remove antenna while receiving or sending data) ... Changes I made are working since two days without any hang... but does NOT solve the main problem: why SARA does not send nothing.

I my case, it happens randomly, it sometimes when connecting: it dies at +CREG URC. But most of cases are when transfering or receiving data.

Hope it helps!

[Rocketct]

ok @FrancMunoz thank you, i'll try it, have you used a particular sketch to reproduce the issue?

[FrancMunoz]

@Rocketct , No simply used the same sketch that was failing changing libraries and running for a long time. The #12 is based on it.

[FrancMunoz]

@Rocketct I've noticed using the above attached library that timeouts are so low so sometimes when network is slow it ends but no error has occurred yet (but control from arduino is not lost). I've increades timeouts to 10000ms. If you test please increase these values. If you need I can send you modificacions.

[Rocketct]

ok @FrancMunoz you talk about the timeout in the waitFoResponse() in modem.h correct?

[FrancMunoz]

@Rocketct , no, I mean in:

GSMClient::connect()

int GSMClient::connect()
{
  unsigned long start;
  if (_synch) {
    start = millis();
    while (ready() == 0) {
      if(millis()-start>10000) {
      	_state = CLIENT_STATE_ERROR_TIMEOUT;
      	return 0;
      }
    }
  } else if (ready() == 0) {
    return 0;
  }

  _state = CLIENT_STATE_CREATE_SOCKET;

  if (_synch) {
  	start = millis();
    while (ready() == 0) {
      delay(100);
      if(millis()-start>10000) {
      	_state = CLIENT_STATE_ERROR_TIMEOUT;
      	return 0;
      }
    }

    if (_socket == -1) {
      return 2;
    }
  }

  return 1;
}

GSMClient::write

size_t GSMClient::write(const uint8_t* buf, size_t size)
{
  if (_writeSync) {
  	unsigned long start = millis();
    while (ready() == 0) {
      if(millis()-start>10000) {
      	_state = CLIENT_STATE_ERROR_TIMEOUT;
      	return 0;
      }
    }
  } else if (ready() == 0) {
    return 0;
  }
...

and GSMClient::available

int GSMClient::available()
{
  unsigned long start = millis();
  if (_synch) {
    while (ready() == 0) {
      if(millis()-start>10000) {
      	_state = CLIENT_STATE_ERROR_TIMEOUT;
      	return 0;
      }
    }
  } else if (ready() == 0) {
    return 0;
  }

  if (_socket == -1) {
    return 0;
  }

  MODEM.poll();

  return _available;
}

All in the modified library I uploaded above. I created the state CLIENT_STATE_ERROR_TIMEOUT to know the state of modem and respond if necessary.

[daz]

In the meantime, how about using a watchdog?

https://github.com/adafruit/Adafruit_SleepyDog/blob/master/utility/WatchdogSAMD.cpp

[Rocketct]

@FrancMunoz i have test your library and also for me it hangs generally after one day, i had experienced the freeze in writing operation and only one time in +CREG

[FrancMunoz]

Hi @Rocketct , yes... it freezes at these points I'm working to solve that (I need it make it work to finish our project). All in all I'm guessing the problem is in SAMD core, I'm not sure but have some suspicion about it.

In my case, sometimes all goes crazy all commands sent to SARA returns ERROR or there's no return from SARA, sometimes it simple get in an infinite loop, sometimes fails writing and sometimes +CREG also fails. Since I made it work with previous version of SAMD core, will try and post here.

@daz , I think a watchdog is a bad practice here, it won't help. Only will have an arduino rebooting without any control (I had it).

[FrancMunoz]

Ah! @Rocketct , I think it freezes after a day because SARA gets collapsed and all goes crazy.

[FrancMunoz]

Hi! I've updated SAMD core with hourly builds and it seems that is more stable. Then noticed SARA hangs the Arduino processor because it does not respond to it and then we get infinite while loops.

That's because my didn't the code don't take notice of SARA state and repeated commands cause SARA to stop working. So changed my code to consider SARA status before trying to do anything else.

Then noticed that Movistar (Spain operator) changes IP or forces to reconnect every 30-36 hours (maybe from DHCP lease timeout.

So changed my code:

1. To know every time the status of SARA
2. To reconnect sockets in case of disconnection (local or remote)
3. To reattach GPRS in case of IP changed (also can be done sending AT+UPSDA=0,3 command)
4. To reboot (is the same time reboot to restart GSM) in case of GSM failure.

That worked very fine, now we don't have hangs nor have to reboot. It has been working four days without any reboot (first message sent from arduino is a hello to server and server stores all hellos received)

I still using 20ms delay and downgraded modem speed to 57600.

Will update soon hope with good news.

[Rocketct]

ok @FrancMunoz i'll try your version of the core immediately, while i had try to reestablish again the connection with the module and try to manage the delay from sketch. Why you have downgrade the speed to 57600?whit 921600 had find some problems?

[FrancMunoz]

Hi! I downgrade speed because I discovered is more stable in previous versions of Core, right now when I've problems I downgrade and it seems more stable... and then try to upgrade... I think it's not necessary right now... but when desperation comes around... :)

[ardiri]

i've come across this myself today and this thread sums up my findings as well..

the sad thing is; the library has been designed to be asynchronous in nature (avoid these infinite loops); but the implementation has been half done.. for example; you can pass a false value to the constructors for GSM and GPRS - but, can you check the status? hell no..

GSM3_NetworkStatus_t GSM::begin(const char* pin, bool restart, bool synchronous) { if (!MODEM.begin(restart)) { _state = ERROR; } else { _pin = pin; _state = IDLE; _readyState = READY_STATE_CHECK_SIM; if (synchronous) { while (ready() == 0) { delay(100); } } else { return (GSM3_NetworkStatus_t)0; } } return _state; }

how does one get the state of the modem if they were to implement asynchronous calls? they cannot; i had to modify the library to add a method to the objects:

int lowPowerMode(); int noLowPowerMode(); GSM3_NetworkStatus_t getState() { return _state; };

and then i could modify my code to be as follows:

dbgUART.print(":: register GSM network: "); do { gsm.begin(PIN_NUMBER, true, true); // force reset of modem ts = millis(); while ((gsm.ready() != 0) && millis() - ts < AT_TIMEOUT); } while (gsm.getState() != GSM_READY); dbgUART.println("done");

and put the timeout handling in my sketch; with minimal changes to the library. i'm finding now that my code is hanging in the connect(), send() or available() methods - and i'll make more changes similar to what FrancMunoz has done to rectify the problem.

[ardiri]

just a small update; using the code thats half written - sorted out the client being asynchronous as well

int getState() { return _state; }

and inside the write() function comment out the _writeSync (it has a timeout on it)

// if (_writeSync)

you can then connect using the following code:

dbgUART.print(":: connecting to server:   ");
client.connect(server, port);
ts = millis(); while ((client.ready() == 0) && millis() - ts < HTTP_TIMEOUT) ;
if (client.getState() == 0)
{
  dbgUART.println("done");
  
  // make a HTTP request: 
  client.beginWrite(false);
  client.print("GET ");   client.print(path);     client.println(" HTTP/1.1");
  client.print("Host: "); client.println(server);
  client.println("Connection: close");
  client.println();
  client.endWrite(false);

of course; the other changes that FrancMunoz mentions are still an issue - but, i believe there was some design consideration to have asynchronous manipulation to avoid these horrible infinite loops as part of the design; it just hasn't been completed fully.

[Rocketct]

Hi @ardiri why you need the value of _state? is not sufficient the value returned by ready()

[ardiri]

actually; i would prefer to have written it as:

if (client.getState() == CLIENT_STATE_IDLE)

but the constant is defined in the .cpp file; ready could refer to a number of conditions:

    if (c == '\n') {
      int responseResultIndex = _buffer.lastIndexOf("OK\r\n");
      if (responseResultIndex != -1) {
        _ready = 1;
      } else {
        responseResultIndex = _buffer.lastIndexOf("ERROR\r\n");
        if (responseResultIndex != -1) {
          _ready = 2;
        } else {
          responseResultIndex = _buffer.lastIndexOf("NO CARRIER\r\n");
          if (responseResultIndex != -1) {
            _ready = 3;
          }
        }
      }

which is deeply nested within the library itself.. in fact; the code is a little badly written; you could technically deal with checking if ready is 1; but, you would have to have found the code above to do so.