ohsome-api icon indicating copy to clipboard operation
ohsome-api copied to clipboard

Send proper Error message on API OutOfMemoryError

Open SlowMo24 opened this issue 4 years ago • 9 comments

Use Case Description

Too large bpolys can overcharge the ohsome api depending on its settings (e.g. with a 3MB geojson of 900 polygons with an avg of 109 vertices). Sending this request will (after some time) return the following "error":

    "status": "200,\n  \"error\" : \"OK\""
    "message": "java.lang.OutOfMemoryError\",\n  \"path\" : \"/elements/geom"

This message is confusing (error and OK at the same time) and does not help the user how to overcome this issue or what went wrong.

Request Description

The returned message should be improved, even though it may not be cause by the ohsome-api itself but by the used framework.

ohsome-py logs: errorOnLargeBuffer.zip

SlowMo24 avatar May 03 '21 11:05 SlowMo24

error and OK at the same time

at first glance, this seems like a bug from your description. what kind of request endpoint was this?

Sending this request

could you please share the large input geometry and rest of the query parameters, so we can replicate this issue?

tyrasd avatar May 07 '21 09:05 tyrasd

could you please share the large input geometry and rest of the query parameters, so we can replicate this issue?

I updated the issue with a file link. As discussed the geometries come from a buffer of existing osm-elements. The same query works fine if the buffer radius is decreased by 10m

SlowMo24 avatar May 07 '21 10:05 SlowMo24

~~rest of query also, please :pray: (see https://github.com/GIScience/ohsome-api/blob/master/.github/ISSUE_TEMPLATE/bug_report.md#general-information)~~

//edit: ah, sorry, there's also a json file included in the zip file. :see_no_evil: sorry for the oversight

tyrasd avatar May 07 '21 10:05 tyrasd

one last question: is the snippet above the full result you get, or is there more which you omitted?

tyrasd avatar May 07 '21 10:05 tyrasd

OK, this is a data extraction request, which could be a first hint why the status code is 200 "OK": these requests use chunked/streaming data output, where there is no technical way to modify the status code from 200 to something else once data transmission has begun.

Anyway, I agree that the error message should not contain "error" = "OK" since it is highly confusing.

tyrasd avatar May 07 '21 10:05 tyrasd

one last question: is the snippet above the full result you get, or is there more which you omitted?

It is the full ohsome-py log output but I have no knowledge if anything gets omitted within ohsome-py

SlowMo24 avatar May 07 '21 10:05 SlowMo24

The raw response (see attached file) shows that data is transferred for this error just as for a timeout that occurs during transaction. Its just not valid json.

SlowMo24 avatar Jun 01 '21 11:06 SlowMo24

Its just not valid json.

yes, that's the expected behaviour for extraction endpoints, as documented:

If an error occurs during a data-extraction request, the result will be a broken GeoJSON containing the error JSON object at the end. In this case, the ohsome API will give back a 200 response status.

tyrasd avatar Jun 01 '21 11:06 tyrasd

yes, I just wanted to provide a raw response, as requested, as this was not contained in the errorOnLargeBuffer.zip above (masked by oshome-api).

SlowMo24 avatar Jun 01 '21 12:06 SlowMo24