pathfinder icon indicating copy to clipboard operation
pathfinder copied to clipboard

Move `PendingData` into storage module

Open Mirko-von-Leipzig opened this issue 1 year ago • 0 comments

In contrast to other state data, pending data is ephemeral - it is not persisted to disk. It is only used by RPC methods.

The current flow for pending is roughly:

  1. Sync emits new pending data as it is received
  2. RPC methods read pending data - but are only allowed access to it once it has been validated against the latest actual state in storage. This prevents race conditions from pending being in memory, and an actual db transaction being opened.

Since we already use a db transaction to access pending data, we should move it to storage.

Rationale

  • having all state encapsulated in storage makes sense
  • allows us to unify pathfinder_storage::BlockId and pathfinder_common::BlockId
  • already requires a db transaction
  • opens the door for testing other database abstractions
    • e.g. having a postgres network db feeding a cluster of rpc nodes - this would require either:
      • pending queries can only be done on the master node, or
      • pending data must be persisted (which would be easy to try out once storage abstracts it)

Warning ⚠️

This may be quite convoluted to do correctly. Each RPC method currently handles pending itself - all of this would now need to be done in storage somehow. Could end up being simple or difficult.

Mirko-von-Leipzig avatar Jan 23 '24 08:01 Mirko-von-Leipzig