Transaction Filter Definition

[fracek]

In #33 we started discussing how a transaction filter should look and work. I think the first step is discuss how you see users using this feature.

It's a good idea to start by defining what we want to achieve with the filter and only then define its synatx. If we get the core idea right we can avoid a ridiculously complex parser and make it easy to extend in the future.

The language should filter on:

• Schema by name
• Tables by name
• Columns by value, it should have equality and inequality for sure, I'm not sure about comparison. For example, it doesn't make much sense to say buffer1 > buffer2 (i.e. comparing two byte columns), so we either type check the filters or allow some weird behaviour.

It should maybe implement:

• Logical conditions, e.g. user.id=4 or user.id=10, user.role in ['admin', 'super-admin']

It should not filter:

• Dynamically, e.g. with conditions like column1 value equals column2 value
• Using data from other tables

Nice to have:

• Warn user if we are filtering on a column that does not exist. This would help catch spelling mistakes and make it more user friendly in general.
• Warn about malformed filter definitions as soon as possible, i.e. before the filter is used.

Prior Art

Most logging services provide a similar filtering language. We should look at a couple of examples to see how they solve this problem.

[chasers]

Feel free to take a look at our parser: https://github.com/Logflare/logflare/blob/master/lib/logflare/logs/lql/lql_parser.ex

We actually compare the path with a schema and alert if the path is not found ... what you're talking about here.

And we do some type check so you can do like metadata.metrics.response_time:>100

If you're doing this in Elixir NimbleParsec is what you're looking for.

[chasers]

We have this now, and will expand on it in the docs shortly.

Added two new issues for the warnings you suggested which we should definitely do.