seq-tickets
seq-tickets copied to clipboard
datalust
Global signal administration
With the advent of signal indexing, creating a signal impacts the server resource utilization. It would be nice to have a dashboard (or at least API), that can query all (including personal) signals there are and their space (and possibly cpu time) overhead. One concrete issue i've run into is that i've noticed a
The expression [...] associated with index ea515532-3dfb-4b12-9af5-97290efdb8ba triggered a regular expression match timeout; the index will be ineffective {}
System.Text.RegularExpressions.RegexMatchTimeoutException: The RegEx engine has timed out while trying to match a pattern to an input string. This can occur for many reasons, including very large inputs or excessive backtracking caused by nested quantifiers, back-references and other factors.
but i was unable to locate which signal caused this, i downloaded all shared signals as a json dump via seqcli and searched for parts of the expression but couldn't find anything. My assumption is that this is comming from a personal signal but the log neither reports the name of the signal nor the user who created it so tracking this down is really hard and involves talking to every user of seq and have them fix it manually. Being able to globally administrate this would be much easier.
This failure to index is also not surfaced anywhere i could spot it except in the full diagnostic log i happened to take a look at out of curiosity.
Hey @Suchiman :wave:
Indexing is still a mostly background process that doesn't accept much user interaction or surface many direct details. That's mostly to keep it from becoming an endless tuning task for users to manage, since there isn't a clear point (like the creation of a new table or additional column in a SQL database) to suggest when indexes need to be considered. We are looking at leaning more heavily on indexing over in-memory caching for future releases though, which is going to have to come with some more tools for working with them.
@Suchiman coincidentally, 2022.1 includes this fix, which reduces the impact of regex evaluation timeouts like the one you mention. Well worth updating, as soon as you have the opportunity :-)
With the indexing dashboard in 2024.3, most of the concerns here have been solved so i think this can be closed 👍