feat(flink): Support pattern recognition

[mfatihaktas]

trafficstars

Is your feature request related to a problem?

Pattern recognition in Flink enables searching for a set of event patterns in data streams. Flink comes with a complex event processing (CEP) library which allows for pattern detection in event streams. Flink consolidates CEP and SQL API using the MATCH_RECOGNIZE clause for complex event processing in SQL.

More information is available on the Flink doc.

Describe the solution you'd like

Ibis would support pattern recognition for Flink backend with an API similar to the following

table = con.create_table(...)

expr = table.match_recognize(
    measures= ...,
    partition_by= ...,
    order_by= ...,
    pattern= ...,
    ...,
)

Note: As can be seen in the doc, Flink provides a rich set of pattern matching functionality through MATCH_RECOGNIZE. It is likely that this issue will need to be addressed in multiple iterations.

Other backends

Supporting MATCH_RECOGNIZE:

• Snowflake
• Trino
• Oracle

Not supporting MATCH_RECOGNIZE:

• Clickhouse
• Dask
• Datafusion
• Druid
• Duckdb
• Exasol
• Impala
• Mssql
• MySQL
• Pandas
• Postgresql
• Spark
• Sqlite

What version of ibis are you running?

7.2.0

What backend(s) are you using, if any?

Flink

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[cpcloud]

Generally to introduce new APIs, we like to do a survey of other systems to get an idea of whether it's something that is supportable across backends.

I think other systems support match_recognize, can you look into whether some of our non-Flink backends support this functionality?

[mfatihaktas]

I think other systems support match_recognize, can you look into whether some of our non-Flink backends support this functionality?

Sure, will do and share my findings here.

Edit: Updated the description with Other backends.