go icon indicating copy to clipboard operation
go copied to clipboard

Ingestion Filtering, advanced rule, include activity of sponsored accounts of allowlisted account

Open sreuland opened this issue 2 years ago • 0 comments

What problem does your feature solve?

Capturing sponsored account related tx/ops data during filtering which is normally one-hop away and not dynamically captured during normal account whitelist rule processing.

What would you like to see?

As a service provider with my own Horizon installation, I want to configure a new filter rule to trigger tracking transactions and operations related to any accounts that have active sponsorship from given whitelisted account.

I don't want my horizon historical database to contain more data related to other accounts not related to sponsorships, so, I expect to see my horizon db size on disk still remain relatively small with a slower growth rate compared to no-filtering(i.e. data for entire ledger was saved)

Acceptance Criteria:

  • Horizon provides additional endpoint support on the Filter Configuration API on the admin port to perform CRUD on the 'include sponsored accounts' filter rule model that allows defining whitelisted account ids that should trigger whether filtering includes tx's/ops related to accounts that have active sponsorship from whitelisted accounts.
  • Design the config rule model, it could be a new rule such as account_filter_by_sponsorship that uses the standard whitelist model of a list of strings(account ids), or it could be a new model such as a flag account_filter_traverse_sponsorship=[true|false], and the filter applies the flag in conjunction with existing account_filter_by_whitelist=["acct1,"acct2",...] rule from #4221 .
  • All Horizon ingestion processes(current-forward/reingest) will run the same filtering that applies this rule when persisting historical data.
  • The filtering on this new rule, effectively compiles the state of accounts that have an active sponsorship from the whitelisted account as of present ledger state, and detects when a sponsored account is added or removed from a whitelisted account, at which point the filtering effectively includes/removes the Account from its internal compiled 'sponsored account whitelist' ruleset for filtering to apply.
  • This advanced filter feature should be provided as early/alpha drop builds to some willing early-adopter partners for trial usage.
  • This advanced filter feature is layered on top of existing filter infrastructure established on prior #4221 and #4222.

Pre-requisites:

This feature is dependent on #4221 and #4222 being implemented first to establish the initial filtration framework within ingestion.

sreuland avatar Feb 11 '22 21:02 sreuland