Hydra backlog

[dzhelezov]

Overview

This issue covers future directions and nice-to-haves for Hydra v2, which refers to Hydra post-Babylon release (Hydra v0) and post-refactoring (Hydra v1).

Grading

Impact: 1 to 5

• 1 -> nice to have but no subsential improvement over the feature set / long-term maintanance
• 5 -> will enable genuinely new use-case/onboard users

Complexity: 1 to 5

• 1 -> Will fit at most in a week-long sprint, fully clear how to implement
• 5 -> Requires substantial "grooming" to estimate costs and figure out implementation details, likely > 1 month

Features to consider

Indexing

• Add Prometheus metrics Complexity: 1 Impact: 2
• Event and block subscriptions https://github.com/Joystream/hydra/issues/25 Complexity: 2 Impact: 1
• Index transactions, blocks, and meta information https://github.com/Joystream/hydra/issues/119 Complexity: 2 Impact: 3
• Handle runtime upgrades Complexity: 4 Impact: 3

Processing

• Manifest file https://github.com/Joystream/hydra/issues/5 Complexity: 2 Impact: 3
• Events filtering https://github.com/Joystream/hydra/issues/105 Complexity: 2 Impact: 2
• Type-safe mappings https://github.com/Joystream/hydra/issues/13 Complexity: 4 Impact: 4
• Processing middleware (= plugin-based pipelining) https://github.com/Joystream/hydra/issues/61 Complexity: 5 Impact: 5
• Pipelining plugins (such as decoding sudo calls, sub-calls (https://github.com/Joystream/hydra/issues/6), batch calls): Complexity: 3 Impact: 4 <-- depends on https://github.com/Joystream/hydra/issues/110
• Block arrival hooks https://github.com/Joystream/hydra/issues/14 Complexity: 3 Impact: 1
• Transactions support https://github.com/Joystream/hydra/issues/120 Complexity: 4 Impact: 4

CLI/Shema generation

• @Field type https://github.com/Joystream/hydra/issues/64 Complexity: 2 Impact: 1
• Revisit Full-text-search queries (multiple language support) Complexity: 2 Impact: 2
• Independent schema generation (https://github.com/Joystream/hydra/issues/108) Complexity: 3 Impact: 2

Maintenance and tech debt

• Improve test coverage to at least 80% for the core components: hydra-indexer, hydra-processor, hydra-cli Complexity: 2 Impact: 2
• More (integration) tests for invalid schemas (https://github.com/Joystream/hydra/issues/15) Complexity: 2 Impact: 3

DevOps

• Setup a delivery pipeline, currently all the releases (to NPM and Dockerhub) are done manually in an ad-hoc fashion Complexity: 3 Impact: 2
• Graphana dashboards for the processor and indexer metrics (https://github.com/Joystream/hydra/issues/107) Complexity: 2 Impact: 1
• Deploy prometheus server + graphana + alerts (https://github.com/Joystream/hydra/issues/3) Complexity: 2 Impact: 3