[boschindego] Optimization of API calls

[jlaur]

In order to:

• Reduce risk of hitting rate limits (or being banned) by Bosch.
• Avoid unneeded wake-ups from robot sleep (and display turning on in the middle of the night).
• Possibly have faster updates.

it would be beneficial to:

• Explore additional possibilities with the API.
• Revisit previously implemented logic (#12938, #12986, #13017).
• Introduce new algorithms for smarter polling.

As an example, operating data is of most interest when the device is working or charging. Additionally, when this data is requested, the display on the device will turn on, i.e. the device will be woken up. On the other hand, only by performing this call will we actually reach the device, so if we would avoid calling it for several days, the device could have been stolen, and it would still appear online.

The state request: GET https://api.indego.iot.bosch-si.com/api/v1/alms/xxxxxxxx/state

has some parameters that might be of use:

• longpoll=true: awaits any state change
• timeout=<timeout in seconds>: timeout for long poll, when reached will return 504.
• last=<current state>: current state for longpoll. If new state is different, method will return. If omitted, any state change will be awaited.
• cached=false: not sure what it does
• force=true: not sure what it does

The longpoll is particularly interesting as this could be a way to get faster updates with fewer calls, i.e. a kind of push mechanism. If this would be implemented as asynchronous HTTP calls using Jetty, that might be a significant improvement.

#13158 recently improved update of next planned cutting time, but this could be further improved by separating last and next cutting times. This way one of the two calls could be eliminated in some cases. For example, when time of next planned cutting is hit, next planned cutting needs an update, but last cutting does not. This will only be updated after completing a session - which does not affect next cutting. The calls are inexpensive, but nevertheless.

Last, when making changes to .items files, all items will request REFRESH. Not only will this cause multiple calls, but worse than that, it will cause redundant calls as some channels are updated through the same API calls.

[jlaur]

I have now been able to verify that this request indeed does start position tracking: POST /api/v1/alms/xxxxxxx/requestPosition?count=10&interval=6

Also, longpoll mentioned above has been confirmed to work in the way that it doesn't return until state has changed. When it does, the response is a bit differently structured than usual:

{
    "state": 258,
    "map_update_available": false,
    "charge": 24,
    "operate": 30,
    "mapsvgcache_ts": 1659036798849,
    "svg_xPos": 408,
    "svg_yPos": 680
}

[jlaur]

@BillGOH - if you are interested, you can give latest version a try: https://ci.openhab.org/job/openHAB-Addons/lastSuccessfulBuild/artifact/bundles/org.openhab.binding.boschindego/target/org.openhab.binding.boschindego-3.4.0-SNAPSHOT.jar

It includes #13192 as well as #13179. The first of these has a significant fix for an issue that caused the mower to never go to sleep, and leaving the display turned on all the time.

[BillGOH]

@jlaur Thank you very much. I installed the new version and I will test it. I already saw that the display is on very often; however I was unsure whether it is a problem of the binding or the "normal" functioning of the indego.

[BillGOH]

Seems to work fine!

[jlaur]

Most of the findings were addressed in #13192, but what still remains:

• longpoll support which will dramatically reduce the number of needed calls, and at the same time provider instant state updates. I'm currently working on this, but having some issues with Jetty or thread pool stopping after exactly 5 minutes.
• Fix too many requests being made when .items file is saved due to REFRESH implementation.

[jlaur]

Small status update: I'm struggling with Jetty and longpoll. After five minutes of waiting for HTTP GET, an EOFException is thrown. What I have tried to far:

• Use custom HttpClient (not shared one).
• idleTimeout on request.
• setIdleTimeout on httpClient.
• Custom executor.
• Custom scheduler.

Example:

final QueuedThreadPool queuedThreadPool = createThreadPool(BINDING_ID, 5, 10, 3600);
httpClient.setExecutor(queuedThreadPool);
httpClient.setScheduler(new ScheduledExecutorScheduler(BINDING_ID + "-scheduler", false));
httpClient.start();

And yet:

org.openhab.binding.boschindego.internal.exceptions.IndegoException: java.util.concurrent.ExecutionException: java.io.EOFException: HttpConnectionOverHTTP@dea133::DecryptedEndPoint@32d347{l=/192.168.0.236:39190,r=api.indego.iot.bosch-si.com/139.15.214.16:443,OPEN,fill=-,flush=-,to=303481/3660000}
	at org.openhab.binding.boschindego.internal.IndegoController.getRequest(IndegoController.java:312) ~[?:?]
	at org.openhab.binding.boschindego.internal.IndegoController.getRequestWithAuthentication(IndegoController.java:223) ~[?:?]
	at org.openhab.binding.boschindego.internal.IndegoController.getState(IndegoController.java:611) ~[?:?]
	at org.openhab.binding.boschindego.internal.handler.BoschIndegoHandler.longPollJob(BoschIndegoHandler.java:229) ~[?:?]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
	at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) [?:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
	at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.util.concurrent.ExecutionException: java.io.EOFException: HttpConnectionOverHTTP@dea133::DecryptedEndPoint@32d347{l=/192.168.0.236:39190,r=api.indego.iot.bosch-si.com/139.15.214.16:443,OPEN,fill=-,flush=-,to=303481/3660000}
	at org.eclipse.jetty.client.util.FutureResponseListener.getResult(FutureResponseListener.java:118) ~[?:?]
	at org.eclipse.jetty.client.util.FutureResponseListener.get(FutureResponseListener.java:101) ~[?:?]
	at org.eclipse.jetty.client.HttpRequest.send(HttpRequest.java:730) ~[?:?]
	at org.openhab.binding.boschindego.internal.IndegoController.sendRequest(IndegoController.java:571) ~[?:?]
	at org.openhab.binding.boschindego.internal.IndegoController.getRequest(IndegoController.java:265) ~[?:?]
	... 9 more
Caused by: java.io.EOFException: HttpConnectionOverHTTP@dea133::DecryptedEndPoint@32d347{l=/192.168.0.236:39190,r=api.indego.iot.bosch-si.com/139.15.214.16:443,OPEN,fill=-,flush=-,to=303481/3660000}
	at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.earlyEOF(HttpReceiverOverHTTP.java:385) ~[?:?]
	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:1620) ~[?:?]
	at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.shutdown(HttpReceiverOverHTTP.java:269) ~[?:?]
	at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.process(HttpReceiverOverHTTP.java:185) ~[?:?]
	at org.eclipse.jetty.client.http.HttpReceiverOverHTTP.receive(HttpReceiverOverHTTP.java:80) ~[?:?]
	at org.eclipse.jetty.client.http.HttpChannelOverHTTP.receive(HttpChannelOverHTTP.java:131) ~[?:?]
	at org.eclipse.jetty.client.http.HttpConnectionOverHTTP.onFillable(HttpConnectionOverHTTP.java:172) ~[?:?]
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311) ~[?:?]
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) ~[?:?]
	at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:555) ~[?:?]
	at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:410) ~[?:?]
	at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:164) ~[?:?]
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105) ~[?:?]
	at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104) ~[?:?]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:338) ~[?:?]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:315) ~[?:?]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:173) ~[?:?]
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) ~[?:?]
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:409) ~[?:?]
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:883) ~[?:?]
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:1034) ~[?:?]
	... 1 more