Show pending transactions

[flexsurfer]

trafficstars

Currently, we create "fake" pending transaction when we send a transaction in status, but after relogin it disappears, an incoming pending transaction isn't showed at all

Status go should scan for pending transactions

related issue: https://github.com/status-im/status-react/issues/9976

[Samyoul]

For out bound

Best to use RPC call eth_getTransactionReceipt

Example call:

curl -X POST --data '{"jsonrpc":"2.0","method":"eth_getTransactionReceipt","params":["0xb903239f8543d04b5dc1ba6579132b143087c68db1b2168786408fcbce568238"],"id":1}' https://nodeaddress.eth:8585

For a failed transaction status field will be 0x0, see example for tx 0x3e73d46538a6e9118d3753d8d44559fb45db451f7ce16777a5106776962d91d2

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "blockHash": "0x91e51c7f01a47682bc5a891fe4d4c489da30e813ab970771fade3b23429402e9",
    "blockNumber": "0x9a6267",
    "contractAddress": null,
    "cumulativeGasUsed": "0x438b18",
    "from": "0x6d6d388bd3d80485991aa4300633b8281d3468b0",
    "gasUsed": "0x5909",
    "logs": [],
    "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
    "status": "0x0",
    "to": "0x5acc84a3e955bdd76467d3348077d003f00ffb97",
    "transactionHash": "0x3e73d46538a6e9118d3753d8d44559fb45db451f7ce16777a5106776962d91d2",
    "transactionIndex": "0x4a"
  }
}

Result for a successful transaction, 0xb0215d2b6d2f975074027f7b4df5179607addd5a559c66e1fde6c6cb13a1d763

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "blockHash": "0x91e51c7f01a47682bc5a891fe4d4c489da30e813ab970771fade3b23429402e9",
    "blockNumber": "0x9a6267",
    "contractAddress": null,
    "cumulativeGasUsed": "0x234412",
    "from": "0xe411cd4a47bd986aed679d85e6a5d28885780d2c",
    "gasUsed": "0x628a",
    "logs": [],
    "logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
    "status": "0x1",
    "to": "0xde6c5273c0c3277829e1f856012fc8807a79f521",
    "transactionHash": "0xb0215d2b6d2f975074027f7b4df5179607addd5a559c66e1fde6c6cb13a1d763",
    "transactionIndex": "0x36"
  }
}

For a non-existent or pending tx 0xb0215d2b6d2f975074027f7b4df5178607addd5a559c66e1fde6c6cb13a1d763

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": null
}

[Samyoul]

Problem

https://specs.status.im/spec/13#3rd-party-apis-used-by-current-status-app

How Status use it?

Status works on mobile devices and therefore can’t rely on local node. So all communication to Ethereum network happens via Infura.

Concerns

Making http request means that a user leaks metadata, which can be used in various attacks if the service is hacked. Infura hosts on centralized providers. If these fail or the provider cuts off service, then Status features requiring Ethereum calls will.

Status is 100% dependant on Infura for our RPC calls. Infura doesn't provide eth_pendingTransactions RPC method.

[Samyoul]

Additionally

It seems likely that there is no answer to filtering the pending transactions.

• https://ethereum.stackexchange.com/questions/83742/is-it-possible-to-filter-pendingtransactions
• https://stackoverflow.com/questions/62031380/is-it-possible-to-filter-ethereum-pendingtransactions

This makes it likely that we will need to implement our own node layer, at least for eth_pendingTransactions

[Samyoul]

Proposal

In order to facilitate efficient pending transaction querying Status needs to guarantee mobile nodes access to the pending transaction pool. I've drafted the following:

eth_pendingTransactions infrastructure

Elements

• Ethereum Network
• Status Network
  • Raw Ethereum Nodes (Status Ethereum Node)
  • Services
    • Cleanup Service
    • pendingTransaction Service
    • pendingTransaction Query Service
  • Storage Group
    • Redis Storage
    • Redis Cache
• Status Mobile

Ethereum Network

The wider Ethereum network.

Status Network

The umbrella of server infrastructure directly under the control of a Status IM legal entity.

Raw Ethereum Nodes (Status Ethereum Node)

A full Ethereum node physically located in a geographic location of strategic value.

Services

Application services created for the purpose of reading, parsing, storing, maintaining and querying pendingTransaction data.

Cleanup Service

Service for ensuring that transactions that have left the pending pool are marked as successful or failed. Ensuring that the pending transactions held in storage always represents as faithfully as possible the current state.

pendingTransaction Service

Service for collecting and parsing pending transactions from Status Ethereum Nodes in variously geographic locations.

Note: The main reason for having multiple nodes spread out geographically is to ensure that the Status Network is aware of as many pending transactions as possible, as pending transaction precipitate through the network but are not guaranteed to be readable by any one node in the Ethereum Network.

pendingTransaction Query Service

Service for handling client requests for eth_pendingTransaction filtered by the result.to field.

Storage Group

A group of functionalities and services dedicated to storing, indexing and caching eth_pendingTransaction data.

Redis Storage

Functionality for the efficient indexing and query-ability of eth_pendingTransaction data.

Redis Cache

Functionality for the efficient caching of eth_pendingTransaction queries and related data.

Status Mobile Node

A device running the status-go mobile node.

---

EDIT

Thanks @jakubgs for your feedback, I've updated the diagram to replace ElasticSearch with Redis.

[jakubgs]

That's more or less what me an @corpetty were thinking about last time we discussed a self-hosted alternative to Infura.

My only complaint is that I don't see why we need a freakin' ElasticSearch cluster for data like this. All we need is to map transactions by their hashes and to and maybe from addresses. You can do the same with something much simpler and faster like Redis.

[Samyoul]

OpenEthereum parity_pendingTransactions

After further research it is possible to achieve the sort of queries with OpenEthereum

https://openethereum.github.io/wiki/JSONRPC-parity-module.html#parity_pendingtransactions

The Parity/OpenEthereum JSON_RPC method parity_pendingTransactions gives the kind of filtering required natively to the node.

Example:

{
  "jsonrpc":"2.0",
  "method":"parity_pendingTransactions",
  "params":[
    null,
    {
      "to": { "eq": "0xe8b2d01ffa0a15736b2370b6e5064f9702c891b6" }
    }
  ],
  "id":1
}

This could remove the need for a Redis store.

[Samyoul]

Current Pending Transaction Pool

According to Etherscan there are currently approximately 93,000 pending transactions. See https://etherscan.io/txsPending

To run a solution decentralised on a mobile node would require that node to be aware of all 93,000 transactions and be able to parse and filter them, and keep the list of pending transactions updated. Granted the mobile node could watch for any transaction with a select list of to addresses and discard all others, but the mobile node would still need to receive and parse every pending transaction propagated to it.

However Status mobile nodes currently don't have the ability to broadcast directly to the Ethereum network and require connection to an intermediary node, meaning that they are not currently informed of any data related to the pending transaction pool.

[Samyoul]

Update - Blocked

• In house infra
  • Require approval on the initial proposal
    • Approval is dependant on the costing of archive node infrastructure and maintenance
    • corpetty has some provisional information about costing.
    • @jakubgs initial estimate of provisional costing for solving this specific issue https://notes.status.im/pending-tx-infra-costs
    • Reach out to team and community for input on - Full Status Archive Nodes - A path to independence and true decentralisation
• Decentralised
  • Require a method of performing queries on the pending transaction pool in a decentralised manner.
    • Currently light node solutions basically boil down to a client / server relationship, with no incentive mechanism they are no different to self hosted archive nodes.
      • https://github.com/ethereum/devp2p/blob/master/caps/les.md
      • https://openethereum.github.io/wiki/Light-Ethereum-Subprotocol-(LES)
      • https://discuss.status.im/search?q=ultra%20light%20client
• Infura
  • A third way of implementing incoming pending transactions is to get a feature request fulfilled from Infura. Implementing something similar to parity_pendingTransactions
    • 2020-06-10 - I've sent an email request awaiting a response
    • 2020-06-11 - Infura have responded, they are working this kind of functionality and have requested more details on our requirements.
      • Requirements sent, awaiting response.

[3esmit]

I propose the use of Status Network infrastructure to do this.

For every transaction that happens, when it detects a transaction, it message to the recipient wallet the txid using Status Network. When a user wants to read their pending transaction list, they simply sync a public chat:

When a user wants to read their pending transaction details, the requests are made in private chats:

Status Service only answer to messages signed by the wallet (or by a representative chatid which can authenticate in Server using a signed message from wallet at install).

For example:

Additionally, Status Server might do it only for users that subscribe to the service (which is free).

This way Status can protect privacy of that information, and keep decentralized, while reusing most of Status current infrastructure, being Status Service almost like a chatbot.

Mermaid graphs sources at https://hackmd.io/NP6-uBumRqqTRi1WA_xp4A?both

[Samyoul]

Minor update.

A decentralised alternative to static infrastructure is https://www.pokt.network/ . We'd have to ensure that the nodes have the required RPC methods. Either parity_pendingTransactions or eth_pendingTransactions.

[status-github-bot[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.