Feature Tag Indexing seems to be incomplete since December 23

[Langlaeufer]

I'm submitting a (bug report/feature request/ support or question)

Brief Description

I have been monitoring several regions for over a year. Since the problems with the Overpass interface at the end of last year, it seems to me that tag indexing is still not working correctly.

What is the motivation / use case for this feature?

Select all recent changes affecting highway in an area

What is the current behaviour, (attach relevant screenshots) ?

incomplete listing - not all edited highways are listed - (when I remove the "OSM feature tag" highway from the filter I get the complete result.

When does this occur ?

since dec. 2023

[Langlaeufer]

it is not the spacial index, it's the tag index Does the meanwhile the way how tags are indexed change?

Example:

All my Edits since Jan (mostly affecting highways) https://osmcha.org/changesets/147069026?filters=%7B%22date__gte%22%3A%5B%7B%22label%22%3A%222024-01-01%22%2C%22value%22%3A%222024-01-01%22%7D%5D%2C%22users%22%3A%5B%7B%22label%22%3A%22Langlaeufer%22%2C%22value%22%3A%22Langlaeufer%22%7D%5D%7D

Edits found with filter "highway=*" https://osmcha.org/?filters=%7B%22date__gte%22%3A%5B%7B%22label%22%3A%222024-01-01%22%2C%22value%22%3A%222024-01-01%22%7D%5D%2C%22users%22%3A%5B%7B%22label%22%3A%22Langlaeufer%22%2C%22value%22%3A%22Langlaeufer%22%7D%5D%2C%22tag_changes%22%3A%5B%7B%22label%22%3A%22highway%3D%2A%22%2C%22value%22%3A%22highway%3D%2A%22%7D%5D%7D

[Langlaeufer]

Indexing is still not working correctly. No canges of highway indexed since 4 days. This is not correct!

[Langlaeufer]

Since 20 day feature-tag indexing not working at all.

[Langlaeufer]

Again, the indexing of the tags (e.g highway=cycleway, bicycle=*) is not working since one week.

[Langlaeufer]

Ok. It's working again, but some one week delayed

[Langlaeufer]

Feature Tag Indexing is again not working since one week . Last entries are now 17 days old. https://osmcha.org/?filters=%7B%22date__gte%22%3A%5B%7B%22label%22%3A%22%22%2C%22value%22%3A%22%22%7D%5D%2C%22tag_changes%22%3A%5B%7B%22label%22%3A%22cycleway%3D%2A%22%2C%22value%22%3A%22cycleway%3D%2A%22%7D%5D%7D

[jake-low]

Hi @Langlaeufer, thanks for reporting these issues. Indexing of tag changes tends to fall behind after an outage, because the backend needs to "catch up" on changes that it missed. Processing the data is slow (we can only process new changesets slightly faster than the rate that mappers are creating them) so even after brief downtime of the backend it can take a long time to backfill the missing data. I know that this is frustrating and am working on improving the performance of the data processing pipeline in order to alleviate this.

[Langlaeufer]

Thanks for your answer. From an outside perspective, the indexing process appears to keep stalling because there hasn't been any progress in the last 10 days (a regression from an 8 to 17-day delay). However, there are now new entries again (still 17 days behind).