Query events by subscription filter

[nguyer]

Subscriptions have a filter on them, which is applied to events as they appear in the event stream to determine if they match the subscription.

This filter is spelt with a JSON semantic that is completely unique to subscriptions: https://hyperledger.github.io/firefly/head/reference/types/subscription.html#subscriptionfilter

So if you wanted to browse the history of messages that would be delivered to your subscription, and look at where the offset of your subscription is up to in that stream, it's a really hard thing to do!

You have to construct an API query from scratch, that you believe is accurate compared to what FireFly is constructing internally in Go code.

So, this enhancement request is to instead provide a simple API GET endpoint underneath a subscription to ask FireFly to do that query-building for you.

e.g. you get a collection API that's pre-filtered by all the filtering that's part of your SubscriptionFilter, and when you do queries on top of it (like id=xyz or created=>...) you're adding an additional set of filters on top of the filters already in place.

[peterbroadhurst]

The wrinkle here, is that SubscriptionFilter applies filters as a Regular Expression, and currently we don't have support for regular expression filters in our DB semantics.

So I think the most straight forward and determinsitic way to build this feature is:

• Run the DB query without skip and limit from the API call (use our own internal skip + limit)
• Use the existing go code from the filtering that is done in the subscription to post-filter these
• Loop on the skip + limit in our internal loop, until we've fulfilled the skip + limit on the external

[SamMayWork]

PR for this now open at https://github.com/hyperledger/firefly/pull/1452

[EnriqueL8]

This has been merged and completed