Wrapping unconfirmed transactions

[kushti]

Currently, only ErgoTransaction instances are stored in the mempool. This limits mempool (and candidate generation maybe) a lot. For example, transaction's computational cost could not be used.

So introduce UnconfirmedTransaction class in the mempool and around, which is having following fields:

• ergo transaction
• (optional) transaction cost during the last check
• timestamp when transaction entered the node
• (most recent) timestamp when transaction was checked
• (optional) transaction bytes - as transaction usually entered via the network, store bytes to avoid serialization when the node is propagating the transaction. Will be None (for some time) when transaction is submitted via API

[ApexTheory]

I can take this one.

[nirvanush]

@kushti @ApexTheory Just saw this issue, it seems related, not sure I should open a new one or not.

A problem

Unconfirmed transactions are getting stuck in mempool for way too long. I've run a scanner for a few day, see the statistics from yesterday below, age is calculated in minutes.

It significantly slows down the response of the transactions/unconfirmed endpoint. Sometimes requests for 50 txs are timeouts after 60 seconds. (Not sure if it is related to my machine, I'm using my own node on 8GM RAM VPS).

Possible solution

It would be great rather have a way to filter the API response based on the enter timestamp (after it is wrapped with UnconfirmedTransaction) to filter out old transactions. Or maybe have a smaller TTL for the unconfirmed tx, that will encourage devs to do optimizations: box value/ add a fee.

In general, as we grow bigger I think the endpoint POST transaction of ergo node and the equivalent of it for ergo explorer should have more strict validation for what is being submitted to the pool and also should share exact same validation rules.

8513d7b14d9432396cbdaff7f4add45f454b06c02f39470e72c7cf9255b2fb4e age: 165.0 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
1e8813f2e608920f537fd266583ebd9b585d8cdfccdec0ac129519db79279dcc age: 165.0 fee: 0.01 size: 843 rate: 11862 inputs: 5 outputs: 3
ada25395d82484d5e3265c9aa77d188258e6e85d1046c57046e85c872cf7286d age: 162.2 fee: 0.01 size: 2193 rate: 4560 inputs: 20 outputs: 3
8442e514c9ea8e7136bb1489e1371b4463710c071fa91e52213c5ded99066bb9 age: 162.2 fee: 0.01 size: 1113 rate: 8985 inputs: 8 outputs: 3
998f8a1616ce341f64f0b11ee45d99f94e94cc2e6a663009ec99f19962eef18b age: 162.0 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
fc98117dcb7dbb4d90124c85c095395526ae58f70711af7653120408d820bb98 age: 162.0 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
5feb6d1ca9588deb997fb950ba3fb44de3b5ba685c9616b90f4f1c4121ea8654 age: 161.0 fee: 0.01 size: 2193 rate: 4560 inputs: 20 outputs: 3
f95f8e4e6d98519de829742ab7c1d0057abe47b01f850bd2da8ccf4824b36cd3 age: 144.6 fee: 0.01 size: 1473 rate: 6789 inputs: 12 outputs: 3
8918cc944b1d5f05c59ff4c236a5d7968f7fc4ce2c18665568eace85b303f30e age: 135.1 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
968f37aa68dc399168b0ee4bc93e52c00016da764deed6bbf206e68fd69c3543 age: 135.1 fee: 0.01 size: 843 rate: 11862 inputs: 5 outputs: 3
e10b816063bdead62ed6d03341c5a67b6ab63822b1a90f9fa6c47a0da247f61f age: 118.5 fee: 0.01 size: 1293 rate: 7734 inputs: 10 outputs: 3
0b26b81bc7ecc4df79a680330724d4e37c7b04f0413ddea507ad132931868b65 age: 116.9 fee: 0.01 size: 2012 rate: 4970 inputs: 20 outputs: 3
5523a02d5bba93e626d93a01763a43a1966c0b40c2c43fdb9733856031bd8fb4 age: 116.9 fee: 0.01 size: 2012 rate: 4970 inputs: 20 outputs: 3
c1ef1f9c9bcfab664618442bfaddef39afb8c6f79bbefa4e11d2346463d8c4b7 age: 116.9 fee: 0.01 size: 1293 rate: 7734 inputs: 10 outputs: 3
c858e47ff8750cce32033cee7b106728b49a82c3ae4e4adb43fb36437e731f14 age: 116.3 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
8d72bbaceed695024727033dd29285e9784c18b44cbd61a3259567a8ee7bde75 age: 116.3 fee: 0.01 size: 1653 rate: 6050 inputs: 14 outputs: 3
1adc23f1587a7691eaffa9b6286c4ac3ad39d96cab51fa63e5baa5d05f4283d1 age: 116.3 fee: 0.01 size: 843 rate: 11862 inputs: 5 outputs: 3
69e73afc8b510b15db35798e72197fc305d8c727294e6150eab7f35a7a7c41eb age: 116.3 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
aad3530ce923767216e716ab1adc3c3fe8e9a27f677d32423a677703be06d55c age: 116.3 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
076c759530e1c1df5de88dd0fb75123716ffabafed6087d3727d269665855bd9 age: 116.3 fee: 0.01 size: 2373 rate: 4214 inputs: 22 outputs: 3
08a0afe173930d914cd79d0f8b30ed3ad9f44717a2f806d74bcfbf00c5b35a8f age: 116.3 fee: 0.01 size: 2193 rate: 4560 inputs: 20 outputs: 3
adcae53f8e4814d7aa0d15783913550a32935442042d219abeeb02f769133207 age: 116.3 fee: 0.01 size: 663 rate: 15083 inputs: 3 outputs: 3
59d4823de5158160623bf44edeb82d686501510a4c3149e1a92d77be26200da4 age: 116.3 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
c5e77cde9b253265331767e69cd88ba5980f5a348721a1b5188489d48831b278 age: 112.9 fee: 0.01 size: 1473 rate: 6789 inputs: 12 outputs: 3
f76d1ec883359287f6612b577b8bf2eacfc887cd3fa3850281a984e451a2d0dc age: 110.3 fee: 0.01 size: 1023 rate: 9775 inputs: 7 outputs: 3
8fe0680f285b336ee646dd0cd5dc532feb0ddf4564c5bd7e70375e565a1bb744 age: 110.3 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
12cc134aa42a29e7d9ea7ab820769fa9a870b8ac8487f433ebb17a788770cdca age: 110.3 fee: 0.01 size: 3903 rate: 2562 inputs: 39 outputs: 3
583846a4c2e8d0d0c3fa9ddddd2f86964a1642691d8d5e607d1bf02a1426f4d1 age: 110.2 fee: 0.01 size: 1743 rate: 5737 inputs: 15 outputs: 3
8e3794609c05fc06654030064c9e0a968f966a3cfa79a284820eb4c994fde322 age: 107.9 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
413c5e806d4a9a7974ccd497ae43d5df41a50ebe682293b2ca0d2de6f4a6b20a age: 106.6 fee: 0.01 size: 573 rate: 17452 inputs: 2 outputs: 3
be86aefdcb1cf24e0e738ccd5613b703a325a4e352aee9295e42b7d51e32239e age: 92.4 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
8e188fd4801dc4b345d8143538a386e24ed3ce48961bbf9226eb0dc919976289 age: 87.1 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
e0f663c9601f516b256eb7b1452b90eec339af084a5c52a795142dc0a55296ce age: 85.6 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
73c8ce954399cc7648b5c53f6e785a70bf9f3473de0cde051ba265cd0be1e45a age: 85.6 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
36c9952fcdd2d3f533e3e930a0609c07522093b5408f1b2c2a19208d74958656 age: 85.6 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
e100039da98f95d37d107566f1bbc350e002958f9a19c1411d419f360d3dc070 age: 84.5 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
8dbcaf6f8c7d0b090e37a8817dbcee8035dac1f69a08452002bf5381c4bf967f age: 83.5 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
9fab2fc445b0b3cfd67b0a0544a88e20699c71c940e5ff7a7a82d2217bcb3ca5 age: 83.5 fee: 0.01 size: 573 rate: 17452 inputs: 2 outputs: 3
913d02062092ff4656efc8c8d9a7d0f66c646fd132f18599d6067a8a727293bc age: 83.5 fee: 0.01 size: 1923 rate: 5200 inputs: 17 outputs: 3
68ebc9d6e1592003a18ef97d1b13b0c265a2179eb7e80e9a17e14af4a03f08f0 age: 83.5 fee: 0.01 size: 933 rate: 10718 inputs: 6 outputs: 3
d05163771dc3a79707382f5da88fb5d3aa17fdb55cd03d0633e1ea251c140630 age: 83.5 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
c0d7109f24bc79caf130f3a6535cadc51a084b1d37ab40ffc1043bab2276a3f6 age: 83.5 fee: 0.01 size: 573 rate: 17452 inputs: 2 outputs: 3
6e822e931c470f77a517019674e89b296b5e279546cf3b98964ede65f1165c75 age: 83.5 fee: 0.01 size: 2373 rate: 4214 inputs: 22 outputs: 3
851273e5ff49dae67ffbfccc1ebbb19678c205795607822f11a4fb5ffd47079f age: 83.5 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
5d6e8cc5d70bb4c512e3a78c7d1b3cd1dd306a0216d77c57fced982c1537c688 age: 83.5 fee: 0.01 size: 843 rate: 11862 inputs: 5 outputs: 3
885f978f5d4a60111b6db5ca46c2aa5f28cf8c7c8c36e1593634aeb862070a0c age: 82.2 fee: 0.01 size: 2193 rate: 4560 inputs: 20 outputs: 3
40125997c9954855140f05e4ff83a84610443a4e2c828a578df9eda4c93c7223 age: 82.2 fee: 0.01 size: 2193 rate: 4560 inputs: 20 outputs: 3
8612a3341273751acd4f33a774b6bf25e1119201cff90ca187039973ff325f2f age: 82.2 fee: 0.01 size: 1833 rate: 5456 inputs: 16 outputs: 3
8e56e4d052f47a887c6cdd598e7128d8b60269002b5d49ab254e42c70116e1e6 age: 81.2 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
f2e53fef9ac1d46ea9415ab198e758d82ad06e6520d46be93652779cc8e7c8e7 age: 80.6 fee: 0.01 size: 1293 rate: 7734 inputs: 10 outputs: 3
d3177c51690b1386c5494b520216793d6abd5a6b4ce281bc21d8ebfe02891e04 age: 80.6 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
86b48a0273554b4967094ebc5c87a1e50c4157240e172d619e54def4b9ed3b18 age: 80.6 fee: 0.01 size: 573 rate: 17452 inputs: 2 outputs: 3
31987de79e7fbeb64e68de9f4b89d9fdf83702e5bd1542ec2434d24dd89a7b10 age: 80.6 fee: 0.01 size: 2193 rate: 4560 inputs: 20 outputs: 3
b7e0e3e3eddd28174d45fff83f0ea7b0012f7b5f9c88ec3891cbafd291e3fa4f age: 76.0 fee: 0.01 size: 1473 rate: 6789 inputs: 12 outputs: 3
bbd507e3297e8c09ed3dfdc21b32e6479c1c466ea25fad5ab35a1c93988768e4 age: 75.0 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
87a6d9a2bf3d87b4411723c630ac8a993c7718621c5e1d6f868c0b69e11fd80d age: 75.0 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
d5d28865826b3f4ce617b610074f55b1cbd6ea71cb8b268dc90c6f73f2943d8c age: 71.5 fee: 0.01 size: 2013 rate: 4968 inputs: 18 outputs: 3
7e3b3819450d40c629ebe39a206e7349bf00c379b99bc45ab962a6df01564926 age: 71.4 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
dbefb3ac07ea4c01ca658d7d3eced12be5b6eeadf3db07552476e6c14aad8e88 age: 71.4 fee: 0.01 size: 933 rate: 10718 inputs: 6 outputs: 3
3d7efcf361eee8851050ce3196e41daecc8592dedb09973da718a8f730568ba9 age: 71.4 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
708209bc852142b36acb4aab793cbe0a00bb319b4bed3edcfe62b5bf50b61baa age: 71.4 fee: 0.01 size: 2373 rate: 4214 inputs: 22 outputs: 3
799efa160b7710cab4c99d2f458eb77e4cca7dcb69feaed92127aec696918738 age: 71.4 fee: 0.01 size: 663 rate: 15083 inputs: 3 outputs: 3
a8f9e8437038aa216d2c52f5c8bcb2c7a087ca97748e21db0110a1f249e0eca7 age: 64.3 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
6652fed298ebd44947fee1cc6708460ceb5f22ce42c06d46ff2c4d45e1f76a6e age: 64.3 fee: 0.01 size: 1293 rate: 7734 inputs: 10 outputs: 3
d6b4fcd0af11d1dd7470fd28d1161feb4beb906de4007d5ceb43c41978c78e28 age: 64.3 fee: 0.01 size: 573 rate: 17452 inputs: 2 outputs: 3
0aae7bcecaba87f5c88c775c52231bbaf04c55d6daeff3ce384319afc8d87f17 age: 60.3 fee: 0.001 size: 1799 rate: 556 inputs: 3 outputs: 31
9396943a782e4a782ffd847ddb19c11036fe0dff20b3b1d631b36e3f3392f472 age: 60.3 fee: 0.01 size: 2012 rate: 4970 inputs: 20 outputs: 3
0b1829d981b08f9d2271ac5b7347e7ef678c1f266e7c1cfbb018982be6a37cd1 age: 60.3 fee: 0.01 size: 2012 rate: 4970 inputs: 20 outputs: 3
0032404093fe058dcddc5cd9dc928aaf8073bf5f74ab8e43f4bec9d51c44dc3d age: 54.7 fee: 0.01 size: 1743 rate: 5737 inputs: 15 outputs: 3
425c6da0c048b47f58f141ee31e69bf1beeaeeb84e7e1aeb1c7f3bbfb11d228e age: 54.7 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
fd8d79f31781d877c28685fe7409a31c97424c6351971812188381df6b0e4ab4 age: 52.1 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
39845d7cd57b49ec397812caaa8cfaebbf081cd0703076e45a305a6898058bd1 age: 48.6 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
490de5bd8b3a3f68f9e45ffb6e2c16d68ada26bff1504cde7ecdcab017b0d12f age: 48.6 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
1abff3d823c1f9a2be6131ed8f5103a71c2e999c290a0837afcf5e30d3d127cb age: 46.8 fee: 0.01 size: 2193 rate: 4560 inputs: 20 outputs: 3
dd047e24ffdfc99b56f9be651580ad5ea3dff51fa012090ccecd46c48d4468c4 age: 46.8 fee: 0.01 size: 663 rate: 15083 inputs: 3 outputs: 3
2c1d9ccc67bc38162865110665b409517d4cca60c94763b76ca5964b0d70ab7e age: 37.3 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
dd1f949e1230a2dc2c885b498d44b76747eadfff3b314721a93974f7fea30b8d age: 33.1 fee: 0.01 size: 1293 rate: 7734 inputs: 10 outputs: 3
90809a9622f7e4a892a2246457a215b6893c840950a5264b5eb47d429da95b15 age: 32.0 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
7698b1a6a486971ede4321129f7bf90fa36123854421fc43c79e3273eedb7aa9 age: 32.0 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
d0410db9d5e7c760375b5342aec0a359bcf746ffae99f39709683924054037d2 age: 28.9 fee: 0.01 size: 2193 rate: 4560 inputs: 20 outputs: 3
bb1c4b25304a1ecdbc939fe349cfc8605a70801c9879e25120ea4e3c91a22a95 age: 28.9 fee: 0.01 size: 2103 rate: 4755 inputs: 19 outputs: 3
b1f13b10bedf5f0125b9a3c829944b1054de0ee4699f2995132a5b5298e0c1e7 age: 28.9 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
cea9f3232cd9dcf6b98fedee8b5c7492cf0af31e94bb0326c17ab21c4457dbd1 age: 28.6 fee: 0.01 size: 933 rate: 10718 inputs: 6 outputs: 3
098b92a815ced11f1b9bb9c538a7504c6fdb5d00d3d89ea1824296bb339e425f age: 28.6 fee: 0.01 size: 1923 rate: 5200 inputs: 17 outputs: 3
85a274be1bef0edf82057aa760a8123a83726d2feee37462a86c4318c5717a7c age: 28.6 fee: 0.01 size: 3993 rate: 2504 inputs: 40 outputs: 3
a767864e551bf41a6ca730cfef5d9e8f19e002b1b2164d5f3404cd2da91c6f46 age: 28.6 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
61b048830d80765356119c160c33e05a06edcfde9e6c75f28a59632997add05b age: 24.2 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
553cda79b93cbfecbe11fdab37b74e246b3a7c62533d8320b73df7e57c42b967 age: 24.2 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
02e8365b89c03db3bb23d8a7ab4068ea34dea6a4ad0e20e1f89f3f1db22e11f6 age: 24.2 fee: 0.01 size: 843 rate: 11862 inputs: 5 outputs: 3
e6e9ea3649aa25d19cb35e4e076aa9eaef3b0e7f0d42615110226ae2b861600e age: 24.2 fee: 0.01 size: 573 rate: 17452 inputs: 2 outputs: 3
72b39f4f3a49daa3713da5b335b607959c1f72ffbe30db1636eae4e866af2790 age: 16.7 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
43c04f3de239946cb36af6571d6632d095738b6f7dac5653a71e7a1b55a6851c age: 15.0 fee: 0.01 size: 1473 rate: 6789 inputs: 12 outputs: 3
2266e2262d868b6e001b2a79e370e9abb76ac6744d75268e5c07093b30d5e3d1 age: 15.0 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
b3d92e3659e73f1fa343458cafb42f813d91a6d3aae68e5ee6a4591e5b2677e7 age: 13.5 fee: 0.01 size: 2193 rate: 4560 inputs: 20 outputs: 3
e8ee9d944df08aefbffe4ab68cec262f31fbf6c7fabddb18eac30c642f9dab4f age: 13.5 fee: 0.01 size: 753 rate: 13280 inputs: 4 outputs: 3
1b92a3a9c418770850c8078801d384eea893a74c2bc9ec156dc8c48fa828d522 age: 4.8 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
82d708620ae2633d0a03ebefc117518a602460b9811d10729b055248f30aeac4 age: 4.8 fee: 0.01 size: 1383 rate: 7231 inputs: 11 outputs: 3
5a8afc83228b3ecb90a69539e4aeb1c29de83b1875cc7fe9d00f86f9323a43ab age: 4.8 fee: 0.01 size: 933 rate: 10718 inputs: 6 outputs: 3
3f3b4e17e10479468356ed5fabf694513d9cad710794c7024020d8fdfee364ea age: 4.8 fee: 0.01 size: 483 rate: 20704 inputs: 1 outputs: 3
a9651dc7be24c184801b53ef476eba83ebaa01d8c9db3f01998b677e612e6c80 age: 4.8 fee: 0.01 size: 573 rate: 17452 inputs: 2 outputs: 3