ironfish icon indicating copy to clipboard operation
ironfish copied to clipboard

[Decreasing mem pool value]

Open Artyomcheg opened this issue 2 years ago • 4 comments

What happened?

At such a low mem pool value, transactions were not executed. Rebooting the node did not work and I had to kill the node process, after which the node rebooted and the mem pool value began to increase, as well as transactions began to be executed and displayed in the leaderboard.

Version

0.1.38

Debug output

Iron Fish version            0.1.38 @ src
Iron Fish library            0.0.16 @ src
Operating system             Linux x64
CPU model(s)                 AMD Ryzen 7 3700X 8-Core Processor
CPU threads                  16
RAM total                    62.78 GiB
Node version                 v16.16.0
ironfish in PATH             true
Telemetry enabled            true
Accounts head hash           00000000000694f2b585ed710644db57011cb3bb2f7f062c0d8b4d349d398465
Accounts head in chain       true
Accounts head sequence       102287

Relevant log output

Jul 20 12:25:30 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...b4612 (102054) for 00000...f879b (102054) on prev 00000...783ef (102053)
Jul 20 12:25:30 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...b4612 (102054), new: 00000...783ef (102053), fork: 00000...783ef (102053)
Jul 20 12:29:40 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102058, head-seq: 102059, hash: 00000...a9bef, head-hash: 00000...ef626, work: 2276364117863944799, head-work: 2276393076185132699, work-diff: 28958321187900
Jul 20 12:30:24 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block seq: 102060, hash: 00000...60f16, txs: 4, progress: 100.00%, time: 91.2ms
Jul 20 12:32:26 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102061, head-seq: 102061, hash: 00000...f5ee6, head-hash: 00000...705a2, work: 2276450599297401273, head-work: 2276450627398579713, work-diff: 28101178440
Jul 20 12:39:23 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...fb022 (102069) for 00000...870f6 (102069) on prev 00000...f356e (102068)
Jul 20 12:39:23 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...fb022 (102069), new: 00000...f356e (102068), fork: 00000...f356e (102068)
Jul 20 12:41:07 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...5c637 (102071) for 00000...4767b (102071) on prev 00000...74a47 (102070)
Jul 20 12:41:07 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...5c637 (102071), new: 00000...74a47 (102070), fork: 00000...74a47 (102070)
Jul 20 12:42:13 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...ab06e (102072) for 00000...7e3f4 (102072) on prev 00000...4767b (102071)
Jul 20 12:42:13 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...ab06e (102072), new: 00000...4767b (102071), fork: 00000...4767b (102071)
Jul 20 12:50:35 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...59c20 (102077) for 00000...ec568 (102077) on prev 00000...c287f (102076)
Jul 20 12:50:36 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...59c20 (102077), new: 00000...c287f (102076), fork: 00000...c287f (102076)
Jul 20 12:53:35 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102078, head-seq: 102078, hash: 00000...b9a81, head-hash: 00000...f04ed, work: 2276939643574774010, head-work: 2276939643574774010, work-diff: 0
Jul 20 12:54:23 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block seq: 102080, hash: 00000...5af2b, txs: 9, progress: 100.00%, time: 1268.6ms
Jul 20 12:57:45 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...874f6 (102082) for 00000...9eaee (102082) on prev 00000...76aa7 (102081)
Jul 20 12:57:45 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...874f6 (102082), new: 00000...76aa7 (102081), fork: 00000...76aa7 (102081)
Jul 20 12:58:39 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102082, head-seq: 102082, hash: 00000...6c4f5, head-hash: 00000...9eaee, work: 2277053293690679310, head-work: 2277053432745949000, work-diff: 139055269690
Jul 20 13:03:15 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...e7d2e (102087) for 00000...91646 (102087) on prev 00000...1ae63 (102086)
Jul 20 13:03:15 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...e7d2e (102087), new: 00000...1ae63 (102086), fork: 00000...1ae63 (102086)
Jul 20 13:03:32 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102087, head-seq: 102087, hash: 00000...8da33, head-hash: 00000...91646, work: 2277195810471732240, head-work: 2277195894190105152, work-diff: 83718372912
Jul 20 13:08:19 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102090, head-seq: 102090, hash: 00000...ff2f6, head-hash: 00000...c4929, work: 2277280662492095535, head-work: 2277280772759093175, work-diff: 110266997640
Jul 20 13:17:09 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block seq: 102100, hash: 00000...4bf28, txs: 96, progress: 100.00%, time: 3559.7ms
Jul 20 13:18:12 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102101, head-seq: 102101, hash: 00000...d996c, head-hash: 00000...91b43, work: 2277592023417085629, head-work: 2277592106356987203, work-diff: 82939901574
Jul 20 13:19:58 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102102, head-seq: 102102, hash: 00000...43154, head-hash: 00000...da3a6, work: 2277620319746839585, head-work: 2277620444156691946, work-diff: 124409852361
Jul 20 13:19:59 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...a9c2f (102103) for 00000...d2dfb (102103) on prev 00000...da3a6 (102102)
Jul 20 13:19:59 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...a9c2f (102103), new: 00000...da3a6 (102102), fork: 00000...da3a6 (102102)
Jul 20 13:27:46 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102111, head-seq: 102111, hash: 00000...7dc50, head-hash: 00000...b72e3, work: 2277875536166093853, head-work: 2277875563811988743, work-diff: 27645894890
Jul 20 13:27:59 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102111, head-seq: 102111, hash: 00000...7d13e, head-hash: 00000...b72e3, work: 2277875536166093853, head-work: 2277875563811988743, work-diff: 27645894890
Jul 20 13:29:06 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...13f49 (102112) for 00000...fdf9a (102112) on prev 00000...b72e3 (102111)
Jul 20 13:29:06 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...13f49 (102112), new: 00000...b72e3 (102111), fork: 00000...b72e3 (102111)
Jul 20 13:35:03 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...bf303 (102115) for 00000...ba9ac (102115) on prev 00000...4c852 (102114)
Jul 20 13:35:03 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...bf303 (102115), new: 00000...4c852 (102114), fork: 00000...4c852 (102114)
Jul 20 13:35:28 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102115, head-seq: 102115, hash: 00000...75455, head-hash: 00000...ba9ac, work: 2277988179685074144, head-work: 2277988220850082659, work-diff: 41165008515
Jul 20 13:40:04 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...536bb (102119) for 00000...bbc85 (102119) on prev 00000...80b4d (102118)
Jul 20 13:40:04 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...536bb (102119), new: 00000...80b4d (102118), fork: 00000...80b4d (102118)
Jul 20 13:40:17 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block seq: 102120, hash: 00000...b6506, txs: 4, progress: 100.00%, time: 265.2ms
Jul 20 13:45:25 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...0e6a8 (102121) for 00000...d8533 (102121) on prev 00000...b6506 (102120)
Jul 20 13:45:25 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...0e6a8 (102121), new: 00000...b6506 (102120), fork: 00000...b6506 (102120)
Jul 20 13:48:46 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...9a477 (102125) for 00000...c473a (102125) on prev 00000...3e89f (102124)
Jul 20 13:48:46 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...9a477 (102125), new: 00000...3e89f (102124), fork: 00000...3e89f (102124)
Jul 20 13:48:57 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...c473a (102125) for 00000...52f8b (102125) on prev 00000...3e89f (102124)
Jul 20 13:48:57 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...c473a (102125), new: 00000...3e89f (102124), fork: 00000...3e89f (102124)
Jul 20 13:53:39 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102127, head-seq: 102128, hash: 00000...5d932, head-hash: 00000...03af1, work: 2278322510551523485, head-work: 2278350146847218090, work-diff: 27636295694605
Jul 20 13:55:44 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...3eb71 (102132) for 00000...06303 (102132) on prev 00000...66192 (102131)
Jul 20 13:55:44 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...3eb71 (102132), new: 00000...66192 (102131), fork: 00000...66192 (102131)
Jul 20 13:56:59 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102134, head-seq: 102134, hash: 00000...9786c, head-hash: 00000...cc074, work: 2278516830803280173, head-work: 2278516871484311183, work-diff: 40681031010
Jul 20 14:00:53 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102138, head-seq: 102138, hash: 00000...e0320, head-hash: 00000...c4b23, work: 2278628146968107281, head-work: 2278628174154845189, work-diff: 27186737908
Jul 20 14:01:09 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block seq: 102140, hash: 00000...8996b, txs: 7, progress: 100.00%, time: 310.9ms
Jul 20 14:04:22 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...44315 (102141) for 00000...670a5 (102141) on prev 00000...8996b (102140)
Jul 20 14:04:22 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...44315 (102141), new: 00000...8996b (102140), fork: 00000...8996b (102140)
Jul 20 14:05:27 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...821fe (102142) for 00000...e668e (102142) on prev 00000...670a5 (102141)
Jul 20 14:05:27 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...821fe (102142), new: 00000...670a5 (102141), fork: 00000...670a5 (102141)
Jul 20 14:05:45 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102142, head-seq: 102142, hash: 00000...eff02, head-hash: 00000...e668e, work: 2278739557954859021, head-work: 2278739666754364765, work-diff: 108799505744
Jul 20 14:06:01 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102142, head-seq: 102143, hash: 00000...73433, head-hash: 00000...61716, work: 2278739544354920803, head-work: 2278767573827588371, work-diff: 28029472667568
Jul 20 14:12:59 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102147, head-seq: 102147, hash: 00000...6220e, head-hash: 00000...1f6a6, work: 2278878454656512464, head-work: 2278878468176776817, work-diff: 13520264353
Jul 20 14:20:37 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102156, head-seq: 102156, hash: 00000...9b199, head-hash: 00000...4e157, work: 2279127533634507524, head-work: 2279127641899122836, work-diff: 108264615312
Jul 20 14:24:33 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block seq: 102160, hash: 00000...fe2e4, txs: 300, progress: 100.00%, time: 11185.4ms
Jul 20 14:24:52 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...5a652 (102161) for 00000...884a9 (102161) on prev 00000...fe2e4 (102160)
Jul 20 14:24:52 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...5a652 (102161), new: 00000...fe2e4 (102160), fork: 00000...fe2e4 (102160)
Jul 20 14:27:15 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...4a38a (102164) for 00000...f9a6d (102164) on prev 00000...2f6f8 (102163)
Jul 20 14:27:15 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...4a38a (102164), new: 00000...2f6f8 (102163), fork: 00000...2f6f8 (102163)
Jul 20 14:27:26 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...f9a6d (102164) for 00000...466ed (102165) on prev 00000...4a38a (102164)
Jul 20 14:27:26 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 2, old: 00000...f9a6d (102164), new: 00000...4a38a (102164), fork: 00000...2f6f8 (102163)
Jul 20 14:29:20 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...ba4a9 (102167) for 00000...f0ea9 (102167) on prev 00000...1f774 (102166)
Jul 20 14:29:20 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...ba4a9 (102167), new: 00000...1f774 (102166), fork: 00000...1f774 (102166)
Jul 20 14:32:31 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...6c2d4 (102172) for 00000...85d5a (102172) on prev 00000...8bd51 (102171)
Jul 20 14:32:31 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...6c2d4 (102172), new: 00000...8bd51 (102171), fork: 00000...8bd51 (102171)
Jul 20 14:34:48 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...9c371 (102174) for 00000...e48ff (102174) on prev 00000...531ed (102173)
Jul 20 14:34:48 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...9c371 (102174), new: 00000...531ed (102173), fork: 00000...531ed (102173)
Jul 20 14:38:32 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block seq: 102180, hash: 00000...863f4, txs: 19, progress: 100.00%, time: 413.2ms
Jul 20 14:38:49 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102179, head-seq: 102180, hash: 00000...b5dac, head-hash: 00000...863f4, work: 2279770553817994312, head-work: 2279798649472480703, work-diff: 28095654486391
Jul 20 14:41:18 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...2f3db (102183) for 00000...79306 (102183) on prev 00000...ae30c (102182)
Jul 20 14:41:18 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...2f3db (102183), new: 00000...ae30c (102182), fork: 00000...ae30c (102182)
Jul 20 14:42:00 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...b7a50 (102185) for 00000...a09e2 (102185) on prev 00000...fcb17 (102184)
Jul 20 14:42:00 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...b7a50 (102185), new: 00000...fcb17 (102184), fork: 00000...fcb17 (102184)
Jul 20 14:45:10 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102181, head-seq: 102186, hash: 00000...6dc8c, head-hash: 00000...26b54, work: 2279826347047635066, head-work: 2279968157547679237, work-diff: 141810500044171
Jul 20 14:45:42 Ubuntu-2004-focal-64-minimal yarn[1389738]: Added block to fork seq: 102182, head-seq: 102187, hash: 00000...6fb5e, head-hash: 00000...6d82a, work: 2279854112243822519, head-work: 2279996279713073659, work-diff: 142167469251140
Jul 20 14:47:18 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganizing chain from 00000...7e737 (102189) for 00000...68236 (102189) on prev 00000...e7b09 (102188)
Jul 20 14:47:18 Ubuntu-2004-focal-64-minimal yarn[1389738]: Reorganized chain. blocks: 1, old: 00000...7e737 (102189), new: 00000...e7b09 (102188), fork: 00000...e7b09 (102188)
Jul 20 14:50:00 Ubuntu-2004-focal-64-minimal systemd[1]: Stopping IronFish Node...
Jul 20 14:50:00 Ubuntu-2004-focal-64-minimal systemd[1]: ironfishd.service: Main process exited, code=exited, status=1/FAILURE
Jul 20 14:50:00 Ubuntu-2004-focal-64-minimal yarn[1389738]: Shutting down node after SIGTERM
Jul 20 14:50:00 Ubuntu-2004-focal-64-minimal yarn[1389738]: Not connected to the Iron Fish network
Jul 20 14:50:03 Ubuntu-2004-focal-64-minimal yarn[1389738]: Force closing after 3 seconds
Jul 20 14:50:05 Ubuntu-2004-focal-64-minimal systemd[1]: ironfishd.service: Failed with result 'exit-code'.
Jul 20 14:50:05 Ubuntu-2004-focal-64-minimal systemd[1]: Stopped IronFish Node.
Jul 20 14:50:05 Ubuntu-2004-focal-64-minimal systemd[1]: Started IronFish Node.
Jul 20 14:50:05 Ubuntu-2004-focal-64-minimal yarn[1750410]: yarn run v1.22.19
Jul 20 14:50:05 Ubuntu-2004-focal-64-minimal yarn[1750410]: $ yarn build && yarn start:js start
Jul 20 14:50:05 Ubuntu-2004-focal-64-minimal yarn[1750444]: $ tsc -b
Jul 20 14:50:05 Ubuntu-2004-focal-64-minimal yarn[1750486]: $ cross-env OCLIF_TS_NODE=0 IRONFISH_DEBUG=1 node bin/run start
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]:                                            
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]: ::::::::::          :::::::::::::::::
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]: ::::::::::::       :::::::::::::::::::
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]: :::::::::::::     :::::::::::::::::::::
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]: ::::::::::::::   ::::::::::::::::::::::::
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]:  ::::::::::::: ::::::::::        :::::::::
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]:    :::::::::::::::::::::          ::::::::::
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]:    :::::::::::::::::::::          :::::::::
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]:  ::::::::::::: ::::::::::        :::::::::
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]: ::::::::::::::   ::::::::::::::::::::::::
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]: :::::::::::::     :::::::::::::::::::::
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]: ::::::::::::       :::::::::::::::::::
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]: ::::::::::           ::::::::::::::::
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]: Version       0.1.38 @ src
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]: Node Name     BidjOid
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]: Graffiti      BidjOid
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]: Peer Identity UW9N6Kxxs5PJ4wYT1ji+bPkWCO0MRb4pfM9JtBx8jXw=
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]: Peer Agent    ironfish/0.1.38/src
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]: Peer Port     9033
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]: Bootstrap     test.bn1.ironfish.network
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]:  
Jul 20 14:50:06 Ubuntu-2004-focal-64-minimal yarn[1750507]: Another node is using the database, waiting for that node to close.
Jul 20 14:50:55 Ubuntu-2004-focal-64-minimal systemd[1]: Stopping IronFish Node...
Jul 20 14:50:55 Ubuntu-2004-focal-64-minimal systemd[1]: ironfishd.service: Main process exited, code=exited, status=1/FAILURE
Jul 20 14:50:55 Ubuntu-2004-focal-64-minimal systemd[1]: ironfishd.service: Failed with result 'exit-code'.
Jul 20 14:50:55 Ubuntu-2004-focal-64-minimal systemd[1]: Stopped IronFish Node.
Jul 20 14:50:55 Ubuntu-2004-focal-64-minimal systemd[1]: Started IronFish Node.
Jul 20 14:50:55 Ubuntu-2004-focal-64-minimal yarn[1751676]: yarn run v1.22.19
Jul 20 14:50:55 Ubuntu-2004-focal-64-minimal yarn[1751676]: $ yarn build && yarn start:js start
Jul 20 14:50:55 Ubuntu-2004-focal-64-minimal yarn[1751699]: $ tsc -b
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751718]: $ cross-env OCLIF_TS_NODE=0 IRONFISH_DEBUG=1 node bin/run start
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]:                                            
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]: ::::::::::          :::::::::::::::::
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]: ::::::::::::       :::::::::::::::::::
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]: :::::::::::::     :::::::::::::::::::::
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]: ::::::::::::::   ::::::::::::::::::::::::
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]:  ::::::::::::: ::::::::::        :::::::::
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]:    :::::::::::::::::::::          ::::::::::
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]:    :::::::::::::::::::::          :::::::::
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]:  ::::::::::::: ::::::::::        :::::::::
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]: ::::::::::::::   ::::::::::::::::::::::::
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]: :::::::::::::     :::::::::::::::::::::
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]: ::::::::::::       :::::::::::::::::::
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]: ::::::::::           ::::::::::::::::
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]: Version       0.1.38 @ src
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]: Node Name     BidjOid
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]: Graffiti      BidjOid
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]: Peer Identity UW9N6Kxxs5PJ4wYT1ji+bPkWCO0MRb4pfM9JtBx8jXw=
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]: Peer Agent    ironfish/0.1.38/src
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]: Peer Port     9033
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]: Bootstrap     test.bn1.ironfish.network
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]:  
Jul 20 14:50:56 Ubuntu-2004-focal-64-minimal yarn[1751737]: Another node is using the database, waiting for that node to close.
Jul 20 14:53:55 Ubuntu-2004-focal-64-minimal yarn[1751737]: WebSocket server started at :::9033
Jul 20 14:54:01 Ubuntu-2004-focal-64-minimal yarn[1751737]: Connected to the Iron Fish network
Jul 20 14:54:11 Ubuntu-2004-focal-64-minimal yarn[1751737]: Starting sync from lLzyjMk (UxaxaNode). work: +112734175399520, ours: 102191, theirs: 102195
Jul 20 14:54:11 Ubuntu-2004-focal-64-minimal yarn[1751737]: Finding ancestor using linear search on last 3 blocks starting at 00000...00171 (102191) from peer lLzyjMk (UxaxaNode) at 102195
Jul 20 14:54:11 Ubuntu-2004-focal-64-minimal yarn[1751737]: Found peer lLzyjMk (UxaxaNode) ancestor 00000...00171, syncing from 102191 -> 102195 (4) after 1 requests
Jul 20 14:54:11 Ubuntu-2004-focal-64-minimal yarn[1751737]: Requesting 20 blocks starting at 00000...00171 (102191) from lLzyjMk (UxaxaNode)
Jul 20 14:54:14 Ubuntu-2004-focal-64-minimal yarn[1751737]: Peer 5dNKAml (Azat-web) sent orphan 00000...2aa86 (102196)
Jul 20 14:54:14 Ubuntu-2004-focal-64-minimal yarn[1751737]: Sync already in progress from lLzyjMk (UxaxaNode)
Jul 20 14:54:17 Ubuntu-2004-focal-64-minimal yarn[1751737]: Peer 5dNKAml (Azat-web) sent orphan 00000...41705 (102197)
Jul 20 14:54:17 Ubuntu-2004-focal-64-minimal yarn[1751737]: Sync already in progress from lLzyjMk (UxaxaNode)
Jul 20 14:54:23 Ubuntu-2004-focal-64-minimal yarn[1751737]: Finished syncing 4 blocks from lLzyjMk (UxaxaNode)
Jul 20 14:54:31 Ubuntu-2004-focal-64-minimal yarn[1751737]: Starting sync from nnGyVqy (oksyl). work: +56448920965086, ours: 102195, theirs: 102197
Jul 20 14:54:31 Ubuntu-2004-focal-64-minimal yarn[1751737]: Finding ancestor using linear search on last 3 blocks starting at 00000...871dc (102195) from peer nnGyVqy (oksyl) at 102197
Jul 20 14:54:31 Ubuntu-2004-focal-64-minimal yarn[1751737]: Found peer nnGyVqy (oksyl) ancestor 00000...871dc, syncing from 102195 -> 102197 (2) after 1 requests
Jul 20 14:54:31 Ubuntu-2004-focal-64-minimal yarn[1751737]: Requesting 20 blocks starting at 00000...871dc (102195) from nnGyVqy (oksyl)
Jul 20 14:54:34 Ubuntu-2004-focal-64-minimal yarn[1751737]: Finished syncing 2 blocks from nnGyVqy (oksyl)
Jul 20 14:57:32 Ubuntu-2004-focal-64-minimal yarn[1751737]: Added block seq: 102200, hash: 00000...e4ca2, txs: 170, progress: 100.00%, time: 2040.4ms
Jul 20 15:04:02 Ubuntu-2004-focal-64-minimal yarn[1751737]: Added block to fork seq: 102209, head-seq: 102209, hash: 00000...55aea, head-hash: 00000...618c4, work: 2280619265112651413, head-work: 2280619279018788893, work-diff: 13906137480
Jul 20 15:05:25 Ubuntu-2004-focal-64-minimal yarn[1751737]: Added block to fork seq: 102193, head-seq: 102209, hash: 00000...d5832, head-hash: 00000...618c4, work: 2280164568235603517, head-work: 2280619279018788893, work-diff: 454710783185376
Jul 20 15:10:35 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganizing chain from 00000...acb2f (102213) for 00000...101f5 (102213) on prev 00000...f06c4 (102212)
Jul 20 15:10:35 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganized chain. blocks: 1, old: 00000...acb2f (102213), new: 00000...f06c4 (102212), fork: 00000...f06c4 (102212)
Jul 20 15:15:39 Ubuntu-2004-focal-64-minimal yarn[1751737]: Added block to fork seq: 102216, head-seq: 102216, hash: 00000...e3e4b, head-hash: 00000...20672, work: 2280817235362810505, head-work: 2280817249153731823, work-diff: 13790921318
Jul 20 15:16:13 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganizing chain from 00000...3779b (102217) for 00000...4dd3e (102217) on prev 00000...20672 (102216)
Jul 20 15:16:13 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganized chain. blocks: 1, old: 00000...3779b (102217), new: 00000...20672 (102216), fork: 00000...20672 (102216)
Jul 20 15:16:20 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganizing chain from 00000...4dd3e (102217) for 00000...81e54 (102218) on prev 00000...3779b (102217)
Jul 20 15:16:20 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganized chain. blocks: 2, old: 00000...4dd3e (102217), new: 00000...3779b (102217), fork: 00000...20672 (102216)
Jul 20 15:17:22 Ubuntu-2004-focal-64-minimal yarn[1751737]: Added block seq: 102220, hash: 00000...1a931, txs: 15, progress: 100.00%, time: 602.8ms
Jul 20 15:18:20 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganizing chain from 00000...9a2a0 (102222) for 00000...c9d89 (102222) on prev 00000...6dc15 (102221)
Jul 20 15:18:20 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganized chain. blocks: 1, old: 00000...9a2a0 (102222), new: 00000...6dc15 (102221), fork: 00000...6dc15 (102221)
Jul 20 15:18:27 Ubuntu-2004-focal-64-minimal yarn[1751737]: Added block to fork seq: 102221, head-seq: 102222, hash: 00000...de11a, head-hash: 00000...c9d89, work: 2280958356179573679, head-work: 2280986668528283100, work-diff: 28312348709421
Jul 20 15:18:34 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganizing chain from 00000...c9d89 (102222) for 00000...9336b (102223) on prev 00000...9a2a0 (102222)
Jul 20 15:18:34 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganized chain. blocks: 2, old: 00000...c9d89 (102222), new: 00000...9a2a0 (102222), fork: 00000...6dc15 (102221)
Jul 20 15:18:50 Ubuntu-2004-focal-64-minimal yarn[1751737]: Added block to fork seq: 102223, head-seq: 102223, hash: 00000...48fbb, head-hash: 00000...9336b, work: 2281015063836767745, head-work: 2281015077674637376, work-diff: 13837869631
Jul 20 15:26:44 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganizing chain from 00000...48ad4 (102233) for 00000...593fe (102233) on prev 00000...94003 (102232)
Jul 20 15:26:44 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganized chain. blocks: 1, old: 00000...48ad4 (102233), new: 00000...94003 (102232), fork: 00000...94003 (102232)
Jul 20 15:29:20 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganizing chain from 00000...2f08e (102238) for 00000...ebc86 (102238) on prev 00000...36043 (102237)
Jul 20 15:29:20 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganized chain. blocks: 1, old: 00000...2f08e (102238), new: 00000...36043 (102237), fork: 00000...36043 (102237)
Jul 20 15:31:20 Ubuntu-2004-focal-64-minimal yarn[1751737]: Added block seq: 102240, hash: 00000...757b0, txs: 78, progress: 100.00%, time: 2334.9ms
Jul 20 15:37:35 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganizing chain from 00000...749b2 (102244) for 00000...b8688 (102244) on prev 00000...a3089 (102243)
Jul 20 15:37:35 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganized chain. blocks: 1, old: 00000...749b2 (102244), new: 00000...a3089 (102243), fork: 00000...a3089 (102243)
Jul 20 15:52:42 Ubuntu-2004-focal-64-minimal yarn[1751737]: Added block to fork seq: 102259, head-seq: 102259, hash: 00000...9e3e1, head-hash: 00000...fe472, work: 2282044793848558568, head-work: 2282044821765301456, work-diff: 27916742888
Jul 20 15:53:41 Ubuntu-2004-focal-64-minimal yarn[1751737]: Added block seq: 102260, hash: 00000...8b411, txs: 159, progress: 100.00%, time: 3644.7ms
Jul 20 15:55:11 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganizing chain from 00000...77a83 (102261) for 00000...34e9c (102261) on prev 00000...8b411 (102260)
Jul 20 15:55:11 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganized chain. blocks: 1, old: 00000...77a83 (102261), new: 00000...8b411 (102260), fork: 00000...8b411 (102260)
Jul 20 15:57:37 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganizing chain from 00000...0f924 (102264) for 00000...ee1ab (102264) on prev 00000...6330c (102263)
Jul 20 15:57:37 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganized chain. blocks: 1, old: 00000...0f924 (102264), new: 00000...6330c (102263), fork: 00000...6330c (102263)
Jul 20 15:57:47 Ubuntu-2004-focal-64-minimal yarn[1751737]: Added block to fork seq: 102264, head-seq: 102264, hash: 00000...fc09b, head-hash: 00000...ee1ab, work: 2282187949849541260, head-work: 2282188047795754345, work-diff: 97946213085
Jul 20 15:59:09 Ubuntu-2004-focal-64-minimal yarn[1751737]: Added block to fork seq: 102267, head-seq: 102267, hash: 00000...84e73, head-hash: 00000...4950b, work: 2282274100403277378, head-work: 2282274142380105666, work-diff: 41976828288
Jul 20 16:01:20 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganizing chain from 00000...15fa1 (102269) for 00000...277ff (102269) on prev 00000...8c6c6 (102268)
Jul 20 16:01:20 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganized chain. blocks: 1, old: 00000...15fa1 (102269), new: 00000...8c6c6 (102268), fork: 00000...8c6c6 (102268)
Jul 20 16:01:30 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganizing chain from 00000...277ff (102269) for 00000...f3785 (102270) on prev 00000...15fa1 (102269)
Jul 20 16:01:30 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganized chain. blocks: 2, old: 00000...277ff (102269), new: 00000...15fa1 (102269), fork: 00000...8c6c6 (102268)
Jul 20 16:11:52 Ubuntu-2004-focal-64-minimal yarn[1751737]: Added block to fork seq: 102269, head-seq: 102276, hash: 00000...360ba, head-hash: 00000...5d81a, work: 2282330736621765657, head-work: 2282531845591103641, work-diff: 201108969337984
Jul 20 16:13:00 Ubuntu-2004-focal-64-minimal yarn[1751737]: Added block to fork seq: 102278, head-seq: 102278, hash: 00000...455f5, head-hash: 00000...d7f78, work: 2282588778159477702, head-work: 2282588792062424801, work-diff: 13902947099
Jul 20 16:13:41 Ubuntu-2004-focal-64-minimal yarn[1751737]: Added block to fork seq: 102279, head-seq: 102279, hash: 00000...be371, head-hash: 00000...99715, work: 2282617293103979579, head-work: 2282617334812820876, work-diff: 41708841297
Jul 20 16:13:49 Ubuntu-2004-focal-64-minimal yarn[1751737]: Added block to fork seq: 102279, head-seq: 102279, hash: 00000...09ba5, head-hash: 00000...99715, work: 2282617279201032480, head-work: 2282617334812820876, work-diff: 55611788396
Jul 20 16:13:49 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganizing chain from 00000...99715 (102279) for 00000...9ecdc (102280) on prev 00000...be371 (102279)
Jul 20 16:13:49 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganized chain. blocks: 2, old: 00000...99715 (102279), new: 00000...be371 (102279), fork: 00000...d7f78 (102278)
Jul 20 16:13:51 Ubuntu-2004-focal-64-minimal yarn[1751737]: Added block seq: 102280, hash: 00000...9ecdc, txs: 3, progress: 100.00%, time: 1277.2ms
Jul 20 16:14:53 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganizing chain from 00000...3f294 (102282) for 00000...26671 (102282) on prev 00000...0e5d3 (102281)
Jul 20 16:14:53 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganized chain. blocks: 1, old: 00000...3f294 (102282), new: 00000...0e5d3 (102281), fork: 00000...0e5d3 (102281)
Jul 20 16:15:07 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganizing chain from 00000...26671 (102282) for 00000...427c3 (102283) on prev 00000...3f294 (102282)
Jul 20 16:15:07 Ubuntu-2004-focal-64-minimal yarn[1751737]: Reorganized chain. blocks: 2, old: 00000...26671 (102282), new: 00000...3f294 (102282), fork: 00000...0e5d3 (102281)
Jul 20 16:17:32 Ubuntu-2004-focal-64-minimal yarn[1751737]: Added block to fork seq: 102270, head-seq: 102286, hash: 00000...c9968, head-hash: 00000...e4c74, work: 2282358181684541127, head-work: 2282817986020145225, work-diff: 459804335604098
Jul 20 16:19:51 Ubuntu-2004-focal-64-minimal yarn[1751737]: Added block to fork seq: 102287, head-seq: 102287, hash: 00000...57270, head-hash: 00000...98465, work: 2282846611913119613, head-work: 2282846640005261883, work-diff: 28092142270
Jul 20 16:20:28 Ubuntu-2004-focal-64-minimal systemd[1]: Stopping IronFish Node...
Jul 20 16:20:28 Ubuntu-2004-focal-64-minimal yarn[1751737]: Shutting down node after SIGTERM
Jul 20 16:20:28 Ubuntu-2004-focal-64-minimal systemd[1]: ironfishd.service: Main process exited, code=exited, status=1/FAILURE
Jul 20 16:20:28 Ubuntu-2004-focal-64-minimal systemd[1]: ironfishd.service: Failed with result 'exit-code'.
Jul 20 16:20:28 Ubuntu-2004-focal-64-minimal systemd[1]: Stopped IronFish Node.

Graffiti

BidjOid

Artyomcheg avatar Jul 20 '22 15:07 Artyomcheg

How did you know the state of the mempool? It's not in the logs listed here as far as I can tell.

lwisne avatar Aug 03 '22 16:08 lwisne

I was monitoring the node using the ironfish status -f command. I have saved screenshots that I uploaded to the #🐛report-bugs discord channel.

Artyomcheg avatar Aug 03 '22 16:08 Artyomcheg

would you mind attaching those screenshots to this issue as well?

lwisne avatar Aug 03 '22 22:08 lwisne

With such mem pool values, transactions were not sent and not be displayed in the leaderboard: 87686786

After I killed the node process using the kill command, the value of the mem pool changed and transactions began to leave and be displayed in the leaderboard: 67567567567657567

Artyomcheg avatar Aug 04 '22 06:08 Artyomcheg

Closing due to staleness. There have been a variety of mempool improvements since then, please re-file a ticket if the issue persists.

holahula avatar Apr 20 '23 22:04 holahula