nearcore icon indicating copy to clipboard operation
nearcore copied to clipboard

Querying historical transaction status should be compatible with re-sharding

Open frol opened this issue 2 years ago • 3 comments

Describe the problem

While this issue roots down to view-client, I will describe it from JSON RPC point of view.

Currently, the tx endpoint [expects transaction hash and a signer account id](https://docs.near.org/docs/api/rpc/transactions#transaction-status), which is not sufficient given that we use signer account id to map to a shard id which might be different given re-sharding will split the state differently and the same account can move between shards from one epoch to another. @bowenwang1996 suggested to also require to specify epoch id (and assume it is the current epoch if not specified).

@bowenwang1996 Ideally, I would love to ask the chain team to implement the optional epoch_id support on view-client level, and the node interfaces team to take it from there and upgrade JSON RPC implementation.

To Reproduce

This command does not use valid transaction hash and signer id, so it is just for demo purposes:

 $ http post [https://rpc.testnet.near.org](https://rpc.testnet.near.org) jsonrpc=2.0 id=dontcare method=tx params:='[ "6zgh2u9DqHHiXzdy9ouTP7oGky2T4nugqzqt9wJZwNFm", "sender.testnet"]'

Would you need to find some transaction from a signer account that got re-sharded to some other shard later, the node that is processing the request will route the request to a wrong shard.

Expected behavior

Historical transaction status should be available no matter what.

frol avatar Oct 20 '21 20:10 frol

@frol I looked again at the code and noticed the following:

  • Querying transactions from archival nodes is not actually affected. While the calculation of target_account_id will result in a different shard id after resharding, cares_about_shard will still return true because archival nodes track all shards and as a result, it will try to call get_final_transaction_result, which will succeed because it indeed has that data on the node since the node is archival.
  • It will become a problem, however, if in the future we have archival nodes that do not track all shards. Then the change in shard id may cause cares_about_shard to return false, thereby causing the query to fail. @frol do you think we will ever have archival nodes that do not track all shards? If so, we need to rethink the design here since the current design is catered to regular rpc nodes, not necessarily archival nodes.

bowenwang1996 avatar Oct 20 '21 22:10 bowenwang1996

@bowenwang1996 Indeed, it does not affect nodes that track all shards, and that is the current state of things, but when we move on from it, both RPC and archival nodes will be affected. Thus, it is not pressing at the moment, but it is good to keep it in mind.

frol avatar Oct 21 '21 12:10 frol

This issue has been automatically marked as stale because it has not had recent activity in the last 2 months. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

stale[bot] avatar Jan 19 '22 12:01 stale[bot]

As discussed in this thread, we don't think there's an impact while we are tracking all shards. For now, I've added a nayduck test to check this scenario and we can get back to the problem of resharding while nodes track a subset of shards at a future timestamp.

shreyan-gupta avatar Dec 05 '23 13:12 shreyan-gupta

@shreyan-gupta I agree that we're good to go with resharding. However would this issue still be relevant for the state sync rollout when chunk only producers stop tracking all shards? cc state sync team @nikurt @VanBarbascu @telezhnaya I'll leave it to you to either re-open this issue and take ownership, track it separately or leave it as is if irrelevant

wacban avatar Dec 05 '23 13:12 wacban