<remark> element in Augmented Diffs

[pa5cal]

One of the latest augmented diff files contains several remark elements, which I have not seen so far.

runtime error: Way 547230203 cannot be expanded at timestamp 2018-05-08T15:48:01Z. runtime error: Way 547230203 cannot be expanded at timestamp 2018-05-08T15:48:01Z.

Link to the augmented diff file: https://overpass-api.de/api/augmented_diff?id=2972693

[mmd-osm]

Thanks for reporting! It looks like all 0.7.55 instances are affected by this issue (z.*, lz4.*, dev.*/api_mmd) while dev.*/api_drolbr on 0.7.54 seems ok.

Same error using a smaller test case:

[date:"2018-05-08T00:00:00Z"];
way(547230203);
out geom;

Please note that a db rebuild is also pending due to issues with attic data: https://github.com/drolbr/Overpass-API/issues/463#issuecomment-380664639.

[drolbr]

Thank you for reporting the issue. Have you seen further of these errors?

[pa5cal]

Yes. Sadly, I didn't save the dates.

[drolbr]

I were unable to reproduce the issue right now. I will keep this open until the next rebuild as a reminder.

[pa5cal]

Last night (Oct. 29th/30th) between 11.30pm and 1.15am the augmented diff files contain unexpected elements (uri:"", local:"remark"). Not sure if this belongs to the mentioned issue.

[mmd-osm]

Did you capture what the remark said?

[pa5cal]

No, unfortunately not.

[mmd-osm]

Ok. Any chance you could add some logging on your end to have a bit more details next time?

[pa5cal]

Not sure if this is an expected behavior, but the augment diff (http://overpass-api.de/api/augmented_diff?id=3780770) has no data. Same for the following IDs: 3780771-3780777

Reported by logging 15min ago.

[mmd-osm]

There wasn't any upstream data available at this time either:

[pa5cal]

There you go, 7 minutes ago: http://overpass-api.de/api/augmented_diff?id=3783548 Issue: unexpected element (uri:"", local:"remark"). Expected elements are <{}action>,<{}meta>,<{}note>

Content of the augmented diff file:

<?xml version="1.0" encoding="UTF-8"?>
<osm version="0.6" generator="Overpass API 0.7.55.1009 5e627b63">
<note>The data included in this document is from www.openstreetmap.org. The data is made available under ODbL.</note>
<meta osm_base="2019-11-22T19:56:02Z"/>
<remark> runtime error: Query run out of memory using about 2048 MB of RAM. </remark>
</osm>

[mmd-osm]

Thanks, this looks much better. For some reason, a few nodes or ways belonging to rather huge Flixbus relations were changed in that time period. As their geometry changed, the augmented diff now includes all of those relations along with their full details.

You can see the query being executed by appending "&debug=true" to the URL: http://overpass-api.de/api/augmented_diff?id=3783549&debug=true

Now, to overcome the size limit, you could add maxsize and timeout parameters and run this query from the command line. It will produce an XML file with almost 200MB worth of data.

[maxsize:4000000000][timeout:1800]
[adiff:"2019-11-22T18:03:00Z","2019-11-22T18:04:00Z"];(node(changed:"2019-11-22T18:03:00Z","2019-11-22T18:04:00Z");way(changed:"2019-11-22T18:03:00Z","2019-11-22T18:04:00Z");rel(changed:"2019-11-22T18:03:00Z","2019-11-22T18:04:00Z"););out meta geom;

Reasons for Out of memory need to be investigated in more detail.