zebra icon indicating copy to clipboard operation
zebra copied to clipboard

Published 20 hours ago • verified publisher icon ZcashFoundation

Add support for `getblocktemplate` RPC call

Open mpguerra opened this issue 1 year ago • 2 comments

Motivation

Mining pools use the getblocktemplate RPC method.

getblocktemplate

  • getblocktemplate
    • parameters:
      • rules: segwit - not in Zcash
    • fields:
      • height
      • version
      • curtime
      • bits
        • new state request
      • previousblockhash
      • target
        • same state request as bits
      • transactions - get from mempool, needs consistency fix
        • data
        • hash
        • txid
      • coinbaseaux - undocumented constant
      • coinbasevalue - calculation based on consensus rules
      • default_witness_commitment - not in Zcash (required by SegWit)

Transaction Selection

We want to select transactions based on ZIP-317.

In-flight ZIP PRs:

  • https://github.com/zcash/zips/pull/631
  • https://github.com/zcash/zips/pull/637

Error Handling

We might need to return an error until Zebra is at the tip

Testing

  • [ ] Snapshot the output of the RPC
  • [ ] Test calling the RPC from the mining pool software
  • [ ] Manually compare the zcashd and zebrad RPC output using https://github.com/ZcashFoundation/zebra/blob/main/zebra-utils/zcash-rpc-diff

mpguerra avatar Sep 22 '22 09:09 mpguerra

Hey team! Please add your planning poker estimate with Zenhub @arya2 @conradoplg @dconnolly @oxarbitrage @teor2345 @upbqdn

mpguerra avatar Oct 06 '22 10:10 mpguerra

Some potentially useful documentation from bitcoin: https://en.bitcoin.it/wiki/Getblocktemplate

mpguerra avatar Oct 10 '22 10:10 mpguerra

This is the default testnet getblocktemplate response from zcashd.

It contains the following undocumented fields:

  • capabilities: fixed value proposal
  • transactions.*.authdigest: the authdigest value for the block or transaction
  • coinbasetxn: format is undocumented, appears to be the same as a single entry in transactions

This makes it more likely that other parts of the documentation are out of date.

$ zcash-cli -testnet getblocktemplate

{
  "capabilities": [
    "proposal"
  ],
  "version": 4,
  "previousblockhash": "065f6320634754f3e5fbc93ffbde1d4100b96d6a3f3dd552d5b56a015e5b78dd",
  "blockcommitmentshash": "95c502ddc997e79266943e3fcef3f83d496dc2716c3c762f706f73455e754014",
  "lightclientroothash": "95c502ddc997e79266943e3fcef3f83d496dc2716c3c762f706f73455e754014",
  "finalsaplingroothash": "95c502ddc997e79266943e3fcef3f83d496dc2716c3c762f706f73455e754014",
  "defaultroots": {
    "merkleroot": "8cd693cd20e239426327ea06a64558afbddc7637e67884ee17f4ee1d53b63c63",
    "chainhistoryroot": "cdb17ded03160eeed581a7c5f0378d98f820a218856a0251cb0b6ac921c960d8",
    "authdataroot": "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
    "blockcommitmentshash": "95c502ddc997e79266943e3fcef3f83d496dc2716c3c762f706f73455e754014"
  },
  "transactions": [
  ],
  "coinbasetxn": {
    "data": "0400008085202f89010000000000000000000000000000000000000000000000000000000000000000ffffffff050385d21f00ffffffff0480b2e60e000000001976a914054d6009cb52b13b812bfa6fcf4bc7495b65a89c88ac286bee000000000017a9140c0bcca02f3cba01a5d7423ac3903d40586399eb8738c94d010000000017a9145d8b6cb94fb7d99363ae0e7114be5208f47924508740787d010000000017a91471e1df05024288a00802de81e08c437859586c87870000000085d21f000000000000000000000000",
    "hash": "8cd693cd20e239426327ea06a64558afbddc7637e67884ee17f4ee1d53b63c63",
    "authdigest": "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
    "depends": [
    ],
    "fee": 0,
    "sigops": 1,
    "required": true
  },
  "longpollid": "065f6320634754f3e5fbc93ffbde1d4100b96d6a3f3dd552d5b56a015e5b78dd77311",
  "target": "029b220000000000000000000000000000000000000000000000000000000000",
  "mintime": 1666251905,
  "mutable": [
    "time",
    "transactions",
    "prevblock"
  ],
  "noncerange": "00000000ffffffff",
  "sigoplimit": 20000,
  "sizelimit": 2000000,
  "curtime": 1666254605,
  "bits": "20029b22",
  "height": 2085509
}

teor2345 avatar Oct 20 '22 08:10 teor2345

This is the default mainnet getblocktemplate response from zcashd, with no mempool transactions.

$ zcash-cli getblocktemplate

{
  "capabilities": [
    "proposal"
  ],
  "version": 4,
  "previousblockhash": "00000000005a92b9c568345fbb04e80d1d8f6bbcbd5c623ed1163e58806d28ab",
  "blockcommitmentshash": "f086a03f0ab071e8d20c2f9515253b14cfcb76cfbba34e0cb5314c99ffc8a311",
  "lightclientroothash": "f086a03f0ab071e8d20c2f9515253b14cfcb76cfbba34e0cb5314c99ffc8a311",
  "finalsaplingroothash": "f086a03f0ab071e8d20c2f9515253b14cfcb76cfbba34e0cb5314c99ffc8a311",
  "defaultroots": {
    "merkleroot": "3a8f3244e15670a623a08d9b95db69d29ba48d1e8f6bdee970fe1ab39b3f4f9a",
    "chainhistoryroot": "6adb3762fbefc6d49c6a472d71d87fe7e0c4d1adc1a4a97a3a165474d65e1cf8",
    "authdataroot": "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
    "blockcommitmentshash": "f086a03f0ab071e8d20c2f9515253b14cfcb76cfbba34e0cb5314c99ffc8a311"
  },
  "transactions": [
  ],
  "coinbasetxn": {
    "data": "0400008085202f89010000000000000000000000000000000000000000000000000000000000000000ffffffff050364381c00ffffffff0480b2e60e000000001976a914dc38751362cd527d2295064f1a6fac6685e7a14488ac40787d010000000017a914931fec54c1fea86e574462cc32013f5400b891298738c94d010000000017a914a409be939aede71e3e1fa5e2d594161101e7147687286bee000000000017a914d45cb1adffb5215a42720532a076f02c7c778c90870000000064381c000000000000000000000000",
    "hash": "3a8f3244e15670a623a08d9b95db69d29ba48d1e8f6bdee970fe1ab39b3f4f9a",
    "authdigest": "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
    "depends": [
    ],
    "fee": 0,
    "sigops": 1,
    "required": true
  },
  "longpollid": "00000000005a92b9c568345fbb04e80d1d8f6bbcbd5c623ed1163e58806d28ab10",
  "target": "000000000135fb00000000000000000000000000000000000000000000000000",
  "mintime": 1666255449,
  "mutable": [
    "time",
    "transactions",
    "prevblock"
  ],
  "noncerange": "00000000ffffffff",
  "sigoplimit": 20000,
  "sizelimit": 2000000,
  "curtime": 1666256072,
  "bits": "1c0135fb",
  "height": 1849444
}

teor2345 avatar Oct 20 '22 08:10 teor2345

This is the default mainnet getblocktemplate response from zcashd, with some mempool transactions.

I replaced 3.6 MB of hex encoded transaction data with ....

$ zcash-cli getblocktemplate | jq '(.transactions[].data) |= "..." '

{
  "capabilities": [
    "proposal"
  ],
  "version": 4,
  "previousblockhash": "00000000001be30d82561462528b56a61c0a5e921960760edd6e325557ecdcbb",
  "blockcommitmentshash": "18d46472afb5d45985c23b0fb1404ae3f5cc002a463b9f44b11091483bd0cc09",
  "lightclientroothash": "18d46472afb5d45985c23b0fb1404ae3f5cc002a463b9f44b11091483bd0cc09",
  "finalsaplingroothash": "18d46472afb5d45985c23b0fb1404ae3f5cc002a463b9f44b11091483bd0cc09",
  "defaultroots": {
    "merkleroot": "7742ba54ab1a130de3a60020b4e56a45d1a25d0b6776ae9d49091c43aad176d0",
    "chainhistoryroot": "b03c4bea6583d9c35bdbea4de9435fd487f6c8a253c119481f15ee0c2bb7fbe6",
    "authdataroot": "7f5d36e82fcc84267c4cca93a2800cdcfe26b6f0f03566907739c07cda00ef14",
    "blockcommitmentshash": "18d46472afb5d45985c23b0fb1404ae3f5cc002a463b9f44b11091483bd0cc09"
  },
  "transactions": [
    {
      "data": "...",
      "hash": "610dbcbb639dd02a0958a9a7f2cec4a65bdcbe96b96c0f99c74382b7b1f39e35",
      "authdigest": "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
      "depends": [],
      "fee": 245,
      "sigops": 2
    },
    {
      "data": "...",
      "hash": "07ebe4606ced7a3ad952115ceb999f73fdaeb147c4c5aa00084d4f1856fe4806",
      "authdigest": "e2f90dec876601b63412f966b6544a8a7392cfc50ddd421d076ad85e9445ec84",
      "depends": [],
      "fee": 1000,
      "sigops": 0
    }
  ],
  "coinbasetxn": {
    "data": "0400008085202f89010000000000000000000000000000000000000000000000000000000000000000ffffffff050366381c00ffffffff045db7e60e000000001976a914f72070b750eba1b327a268a8f68ab5c05834ef6788ac40787d010000000017a914931fec54c1fea86e574462cc32013f5400b891298738c94d010000000017a914a409be939aede71e3e1fa5e2d594161101e7147687286bee000000000017a914d45cb1adffb5215a42720532a076f02c7c778c90870000000066381c000000000000000000000000",
    "hash": "201d369050ab59cfdc9cd016ea642997943827bc9633de16522ce32469836308",
    "authdigest": "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff",
    "depends": [],
    "fee": -1245,
    "sigops": 1,
    "required": true
  },
  "longpollid": "00000000001be30d82561462528b56a61c0a5e921960760edd6e325557ecdcbb22",
  "target": "00000000013bf900000000000000000000000000000000000000000000000000",
  "mintime": 1666255680,
  "mutable": [
    "time",
    "transactions",
    "prevblock"
  ],
  "noncerange": "00000000ffffffff",
  "sigoplimit": 20000,
  "sizelimit": 2000000,
  "curtime": 1666256289,
  "bits": "1c013bf9",
  "height": 1849446
}

teor2345 avatar Oct 20 '22 09:10 teor2345

Draft of the split we want to do to implement this method:

Goal : Create a default getblocktemplate RPC implementation.

6 tickets total.

This split was chosen because after we introduce a very basic getblocktemplate RPC call with an empty response (1-) then most of the tickets can be worked in parallel.

1- Introduce getblocktemplate

Tasks:

  • Add a getblocktemplate RPC method behind the getblocktemplate-rpcs rust feature.
  • The method should have no arguments.
  • Create the response structure BlockTemplate : https://github.com/ZcashFoundation/zebra/issues/5235#issuecomment-1285174731
  • Always return an empty BlockTemplate
  • Test the rpc method is callable and returns a Blocktemplate type.

No dependencies.

2- Populate Blocktemplate responses with fixed values data.

Several BlockTemplate fields introduced in 1- are fixed values.

Tasks:

  • Identify fixed values fields and add them to the response of the getblocktemplate RPC call.
  • Update the test created at 1- so it can also test for fixed values in the responses.

Depends on 1.

3- Populate BlockTemplate fields with functional values.

Whatever is not fixed values(1-), coinbase transaction(4-) or mempool transaction(5-) will be functional values.

Tasks:

  • Identify functional values.
  • Return functional values in the getblocktemplate RPC method response.

Depends on 1. Can depend on 2 for identification.

4- Populate the coinbase transaction data of GetBlockTemplate.

The coinbasetxn field of the BlockTemplate is a struct:(https://github.com/ZcashFoundation/zebra/issues/5235#issuecomment-1285174731) and can be complex enough to implement it in a separated ticket.

Tasks:

  • Create CoinBaseTxn struct.
  • Populate on getblocktemplate response.
  • Test: TODO

Depends on 1.

5- Populate transactions of BlockTemplate.

The transactions field of the BlockTemplate response is an array of BlockTemplateTx structure with the transaction data and other fields.

Tasks:

  • Create BlockTemplateTx
  • bring transactions from the mempool and populate the transactions.data fields. https://github.com/ZcashFoundation/zebra/issues/5235#issuecomment-1285189453
  • Populate the additional fields of BlockTemplateTx

Depends on 1.

6- Add a miner address config option.

Tasks:

  • Add a config containing the miner address to zebrad.toml, probably in a new [mining] section.
  • Hide the whole mining section in the getblocktemplate-rpcs feature if this is possible?

No dependencies.

oxarbitrage avatar Oct 21 '22 13:10 oxarbitrage

Based on https://github.com/ZcashFoundation/zebra/issues/5235#issuecomment-1286981170 i created 6 tickets:

  • https://github.com/ZcashFoundation/zebra/issues/5451
  • https://github.com/ZcashFoundation/zebra/issues/5452
  • https://github.com/ZcashFoundation/zebra/issues/5455
  • https://github.com/ZcashFoundation/zebra/issues/5453
  • https://github.com/ZcashFoundation/zebra/issues/5454
  • https://github.com/ZcashFoundation/zebra/issues/5456

Feel free to edit the descriptions, they are not all complete yet.

oxarbitrage avatar Oct 21 '22 14:10 oxarbitrage

Based on #5235 (comment) i created 6 tickets:

I just checked the s-nomp code, getblocktemplate also needs to return an error if Zebra isn't synced yet:

  • #5466

teor2345 avatar Oct 24 '22 05:10 teor2345

Removing estimate from issue as it is now a tracking issue, estimates of sub-tasks to be taken into account instead

mpguerra avatar Oct 26 '22 10:10 mpguerra