trueblocks-core icon indicating copy to clipboard operation
trueblocks-core copied to clipboard

Blocks missing from 13,960,000 to 14,500,000?

Open UliGall opened this issue 3 years ago • 27 comments

Hi there,

I am exporting addresses so far with Trueblocks and I am seeing a discrepancy between Etherscan Tx-count and the tx-count that I get from Trueblocks.

Following setup:

  • Exporting via CLI chifra export <address>
  • Filtering for transactions where the address is sender or receiver

I am looking at following two addresses (just random ones I found on etherscan):

  • areubad.eth (0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4)
  • seb40.eth (0xb2fc88e66b8875f0aba69c00a95a08a3c01c881c)

When I compare results from Trueblocks and Etherscan, I notice that for seb40 transactions from block 13962292 to 14499502 are missing in Trueblocks export, for areubad it's from 13972162 to 14499502.

For reference, I've attached the respective export files for Etherscan / Trueblocks. I also extracted the block numbers for each export into a separate txt file, such that it's easier to compare them in an online tool such as https://text-compare.com.

Is this an issue related to the issue that @tjayrush mentioned in discord? However, he mentioned the range to be between 13,300,000 and 13,400,000.

Any thoughts?

Thanks so much!

Files:

es-areubad-blocks.txt es-seb40.csv es-seb40-blocks.txt tb-areubad.csv tb-areubad-blocks.txt tb-seb40.csv tb-seb40-blocks.txt es-areubad.csv

UliGall avatar Jun 06 '22 21:06 UliGall

Thanks so much for reporting this. You’re absolutely correct. There are missing data in blocks after around 13,100,000 (although, it’s spotty).

We are aware of this and working hard to correct the issue which we trace to an issue with an Erigon release from a five or six weeks ago. We noticed it about three weeks ago and are working hard to correct.

I just ran these commands:

➤ chifra list 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 | wc (against the damaged index)
154 records
➤ chifra list 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 | wc (against the corrected index)
163 records

We want to be careful so that we don’t create worse problems with a poorly done migration. We’re working on a solid solution now.

On Jun 6, 2022, at 5:52 PM, Uli Gallersdörfer @.***> wrote: Hi there,

I am exporting addresses so far with Trueblocks and I am seeing a discrepancy between Etherscan Tx-count and the tx-count that I get from Trueblocks.

Following setup:

Exporting via CLI chifra export

Filtering for transactions where the address is sender or receiver I am looking at following two addresses (just random ones I found on etherscan):

areubad.eth (0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4) seb40.eth (0xb2fc88e66b8875f0aba69c00a95a08a3c01c881c) When I compare results from Trueblocks and Etherscan, I notice that for seb40 transactions from block 13962292 to 14499502 are missing in Trueblocks export, for areubad it's from 13972162 to 14499502.

For reference, I've attached the respective export files for Etherscan / Trueblocks. I also extracted the block numbers for each export into a separate txt file, such that it's easier to compare them in an online tool such as https://text-compare.com https://text-compare.com/.

Is this an issue related to the issue that @tjayrush https://github.com/tjayrush mentioned in discord?

Yes — it’s exactly this issue

However, he mentioned the range to be between 13,300,000 and 13,400,000.

It appears to be spotty.

Any thoughts?

Thanks so much!

Files:

es-areubad-blocks.txt https://github.com/TrueBlocks/trueblocks-core/files/8848315/es-areubad-blocks.txt es-seb40.csv https://github.com/TrueBlocks/trueblocks-core/files/8848316/es-seb40.csv es-seb40-blocks.txt https://github.com/TrueBlocks/trueblocks-core/files/8848317/es-seb40-blocks.txt tb-areubad.csv https://github.com/TrueBlocks/trueblocks-core/files/8848318/tb-areubad.csv tb-areubad-blocks.txt https://github.com/TrueBlocks/trueblocks-core/files/8848319/tb-areubad-blocks.txt tb-seb40.csv https://github.com/TrueBlocks/trueblocks-core/files/8848320/tb-seb40.csv tb-seb40-blocks.txt https://github.com/TrueBlocks/trueblocks-core/files/8848321/tb-seb40-blocks.txt es-areubad.csv https://github.com/TrueBlocks/trueblocks-core/files/8848322/es-areubad.csv — Reply to this email directly, view it on GitHub https://github.com/TrueBlocks/trueblocks-core/issues/2155, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJKXPVHL2MQIUAN53BUUZDVNZXIVANCNFSM5YA2B6LQ. You are receiving this because you were mentioned.

tjayrush avatar Jun 06 '22 22:06 tjayrush

Thanks so much!

Okay, then it appears that this actually might not be (at least) my issue.

chifra list 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 | wc yields 163 records on my node, but doing a chifra export 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 -x csv > records.csv only yields 80 records.

Manually asking chifra for the transactions, e.g., chifra transactions 13971718.273 -x csv > tx.csv works. Any idea?

UliGall avatar Jun 07 '22 07:06 UliGall

Something is weird with your setup. I get this:

chifra list 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 | wc
163

chifra export 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 | wc
163

chifra export 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 -x csv | wc
163

chifra export 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 -x csv >results.txt && wc results.txt
163

which makes sense.

Are you two results (list and export) on the same machine?

tjayrush avatar Jun 07 '22 11:06 tjayrush

One thing you should do is this:

chifra monitors --delete --remove 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4

That may not solve any issues, but it will reset the monitor for 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 and start from a clean slate. Then do list and export. Let me know what happens if you do that.

tjayrush avatar Jun 07 '22 11:06 tjayrush

chifra monitors --delete --remove 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4
INFO[07-06|13:44:56.559] Monitor 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 was deleted but not removed.
INFO[07-06|13:44:56.559] Monitor for 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 was permanently removed.

Didn't do anything; all commands you posted lead to the same output, which is 78. That is indeed very weird.

UliGall avatar Jun 07 '22 11:06 UliGall

I thought your first command (list) returns 163. Did that change?

tjayrush avatar Jun 07 '22 11:06 tjayrush

Sorry, the list command did not change and still outputs 163. The export commands do return 78.

UliGall avatar Jun 07 '22 11:06 UliGall

Do you mind experimenting some more?

If no (you don't mind), try this:

chifra status --terse

and notice the $cachePath. Then, remove (or move if you want to save it), $cachePath/mainnet/txs (assuming you're running against mainnet, adjust for a different chain).

This will clear the transactions cache, which is the only thing I can think might be messed up.

It will slow down your processing a bit (at least the next time you try to read that transaction), but it might 'unclog' things. (That's why I said save if you wish).

tjayrush avatar Jun 07 '22 12:06 tjayrush

I moved txs to txs_old and redid the list and export commands, however, does not affect the output. list still returns 163 elements, the export 78. Any other cache I could kill?

UliGall avatar Jun 07 '22 12:06 UliGall

Try the same thing with $cachePath/traces, $cachePath/recons, $cachePath/blocks, and $cachePath/abis as well. (One at a time, in that order).

I'm trying to see if there's a corrupted file in your cache.

In the worst case, empty the entire cache folder (leave the folder). It will be re-created.

Thanks so much for your help by the way.

tjayrush avatar Jun 07 '22 13:06 tjayrush

Quick question -- this is on mainnet. Yes?

Also, can you report the results of chifra status --terse (remove anything you don't want to share).

tjayrush avatar Jun 07 '22 13:06 tjayrush

Yes, its mainnet.

~$ chifra status --terse
2022/06/07 15:26:46 Client:       erigon/2022.99.99/linux-amd64/go1.18.1 (archive, tracing)
2022/06/07 15:26:46 TrueBlocks:   GHC-TrueBlocks//0.36.0-beta-c0fbc744d-20220604 (eskey, no pinkey)
2022/06/07 15:26:46 RPC Provider: http://localhost:8545 - mainnet (1,1)
2022/06/07 15:26:46 Config Path:  /home/<username>/.local/share/trueblocks/
2022/06/07 15:26:46 Cache Path:   /home/<username>/.local/share/trueblocks/cache/mainnet/
2022/06/07 15:26:46 Index Path:   /home/<username>/.local/share/trueblocks/unchained/mainnet/
2022/06/07 15:26:46 Progress:     14921052, 14919829, 14921024, 14921051

That is the chifra status --terse output; I'll try removing the cache and get back to you with the results.

FYI: I built the index with init --all and then contiuned with scrape indexer on my own; the missing block info seems to be from the time with init --all.

No reason to thank me; thank you for the awesome work and helping me out here!

UliGall avatar Jun 07 '22 13:06 UliGall

Renaming / moving the folders did nothing, output still the same. I even moved the complete mainnet folder, but that didn't help as well.

One thing I noticed is somehow that the export is only starting at a certain transaction, e.g. it looks like this

$ chifra export 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 -x csv | wc
00019795 (  19795)- <PROG>  : Extracting        85 of      162 txs for address 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf

Notice that the command is starting with extracting 85 of 162 which explains the result of 78 tx (162-84).

UliGall avatar Jun 07 '22 14:06 UliGall

Update from my side: Complete removal and reinstall of trueblocks did not change a thing, problem is still there. I am setting now TB on an alternative machine to check whether its an issue with Ubuntu 21.

UliGall avatar Jun 08 '22 11:06 UliGall

Alternative machine comes up with the same result, updating erigon now...

UliGall avatar Jun 08 '22 13:06 UliGall

Yeah, updating Erigon also didn't have the intended effect. Next step entirely new setup?

UliGall avatar Jun 08 '22 14:06 UliGall

Entirely new setup; fresh server, fresh ubuntu, set up Erigon, set up Trueblocks. I don't get it.

chifra list 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 | wc
188     564   10441
chifra export 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 -x csv >results.txt && wc results.txt
103   1719 572567 results.txt
chifra export 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 | wc
103    2955  569889
uli@eth:~/$ chifra status --terse
2022/07/01 10:16:21 Client:       erigon/2022.99.99/linux-amd64/go1.18.3 (archive, tracing)
2022/07/01 10:16:21 TrueBlocks:   GHC-TrueBlocks//0.37.0-beta-b84f2c12e-20220627 (eskey, no pinkey)
2022/07/01 10:16:21 RPC Provider: http://localhost:8545 - mainnet (1,1)
2022/07/01 10:16:21 Config Path:  /home/uli/.local/share/trueblocks/
2022/07/01 10:16:21 Cache Path:   /home/uli/.local/share/trueblocks/cache/mainnet/
2022/07/01 10:16:21 Index Path:   /home/uli/.local/share/trueblocks/unchained/mainnet/
2022/07/01 10:16:21 Progress:     15052736, 15050267, 15052708, 15052735

Anyone got any idea?

UliGall avatar Jul 01 '22 08:07 UliGall

Okay, it seems that Erigon might be the problem:

The RPC-daemon is outputting some info:

[WARN] [07-01|06:37:24.317] Served                                   conn=172.20.0.1:44926 method=erigon_getHeaderByNumber reqid=565013 t=977.31µs err="block header not found: 14499944"

UliGall avatar Jul 01 '22 08:07 UliGall

Have you update Erigon lately?

Entirely new setup; fresh server, fresh ubuntu, set up Erigon, set up Trueblocks. I don't get it.

chifra list 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 | wc
188     564   10441
chifra export 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 -x csv >results.txt && wc results.txt
103   1719 572567 results.txt
chifra export 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 | wc
103    2955  569889
uli@eth:~/$ chifra status --terse
2022/07/01 10:16:21 Client:       erigon/2022.99.99/linux-amd64/go1.18.3 (archive, tracing)
2022/07/01 10:16:21 TrueBlocks:   GHC-TrueBlocks//0.37.0-beta-b84f2c12e-20220627 (eskey, no pinkey)
2022/07/01 10:16:21 RPC Provider: http://localhost:8545 - mainnet (1,1)
2022/07/01 10:16:21 Config Path:  /home/uli/.local/share/trueblocks/
2022/07/01 10:16:21 Cache Path:   /home/uli/.local/share/trueblocks/cache/mainnet/
2022/07/01 10:16:21 Index Path:   /home/uli/.local/share/trueblocks/unchained/mainnet/
2022/07/01 10:16:21 Progress:     15052736, 15050267, 15052708, 15052735

Anyone got any idea?

If you do chifra list <address> | tail -2 and chifra export <address> | cut -f1,2.

I get

0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4	15035349	110
0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4	15035650	231

and

15035349	110
15035650	231

which agrees. I'm thinking you must get something different. And, perhaps, the export stops at 14499944.

I'm on a machine with a TON of memory, but there's a noticeable slowdown in the export routine. Perhaps--just conjecture here--Erigon crashes on that block on your machine but not my machine (it just takes a really long time).

Let's try to reproduce it with just curl (exclude TrueBlocks) and we can write a bug against Erigon.

tjayrush avatar Jul 01 '22 12:07 tjayrush

Have you update Erigon lately?

The erigon version is fresh from Tuesday; direct setup and synced, no updates.

uli@eth:~$ chifra list 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 | tail -2
0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4      15046162        318
0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4      15047044        28
uli@eth:~$ chifra export 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 | cut -f1,2
00067833 (    746)- <PROG>  : Extracting       185 of      187 txs for address 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf
15046162        318
00068466 (    633)- <PROG>  : Extracting       186 of      187 txs for address 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf
15047044        28
00068590 (    124)- <PROG>  : Completed        187 of      187 txs for address 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf
4

I actually have no idea was the 4 in the last line is.

which agrees. I'm thinking you must get something different. And, perhaps, the export stops at 14499944.

Seems to work beyond the 15 mio. block number mark.

I'm on a machine with a TON of memory, but there's a noticeable slowdown in the export routine. Perhaps--just conjecture here--Erigon crashes on that block on your machine but not my machine (it just takes a really long time).

My machine has 128Gb memory; Erigon does not seem to crash (at least nothing shows up in the logs as well as in docker ps. Any other way to do this?

I'll check with curl and see how that goes.

UliGall avatar Jul 01 '22 13:07 UliGall

Redirect stderr to /dev/null:

chifra export <address> 2>/dev/null | cut -f1,2

You have a huge machine, it's not a memory problem.

I am seeing a weird message when I do this:

DEV_DEBUG_CURL=true chifra export 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 2>shit --first_block 14549339 --last_block 14549340

are you seeing that? If yes, then this might be a TrueBlocks issue.

But, I thought you said Erigon was displaying an warning above? (Looks more like an error than a warning -- can't find a block.) What do you get with this curl command?

curl --data '{"method":"erigon_getHeaderByNumber","params":["0xdd4068"],"id":"31","jsonrpc":"2.0"}' -H "Content-Type: application/json" -X POST http://localhost:8545

tjayrush avatar Jul 01 '22 14:07 tjayrush

Apprantley Erigon did not access snapshot data and was therefore unable to answer requests with erigon_getHeaderByNumber when the blocks were not regularly synced but received via snapshots. That's also the reason why a freshly synced Erigon had even worse chances as almost all block data comes from snapshots.

My requests now work properly and I am able to access all the relevant data.

Nonetheless, the error you mentioned is also an issue that I saw coming up in my work:

chifra export 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 --first_block 14549339 --last_block 14549340

outputs some weird stub transaction:

blockNumber     transactionIndex        date    timestamp       from    to      ether   ethGasPrice     gasUsed hash    isError encoding        compressedTx
14549339        102     2022-04-09 04:14:50 UTC 1649477690      0xca756a3397b60a8ba8ad9421c9a0db9f1830e377      0x6c5b92aef5373a27a6c0c1850c40fa124a71fe70      0.000000000000000000    0.000000028230145942    8970959 0x4e75db59f7d4472e76eb42ccb1124c75d2c9630d764b5b7d036352758b0fe985      0       0x512d7cfd      0x512d7cfd(stub:000000000000000000000000000000000000000000000000000000000000004000000000000000000000000000000000000000000000012d231c7c593eec00000000000000000000000000000000000000000000000000000000000000000c47000000000000000000000000f4b8d86a8778bd9d6535a47014b3d84f5f4cf3580000000000000000000000006427895b9c2c787a05debe15d421d85ac19f81020000000000000000000000008914c82505bec98ab619, stub:8800fa57677cb0ef4c2c000000000000000000000000fd42c1e887e82c339a4aba06114e6de7394a31e700000000000000000000000057f09fbe4f0987e4df00ecb5019a5ce63803036b00000000000000000000000040155849eff8ec855c94f9d286df75b10d21520b000000000000000000000000c3b107e354526278792d4c0b8d2c054f70644977000000000000000000000000136645221f7b1ef8885728f8d31dbe1b9cf51ca8000000000000000000000000, stub:38ea1987427e47697572f20141a2bdc66bd834b6000000000000000000000000cb20f444d35b232a301c856be1c9c444fe3decda0000000000000000000000000dd41955c2f5980442ab8941510d7c2e26358a79000000000000000000000000a25606da9cf1ff827ac4c6289891cffbb9f03720000000000000000000000000bc1874fd5e67f1f76f32d280572dd4cc21de23430000000000000000000000000919c8cfbf1d8810f65959c51dbd4dfc65b5f14c0000, stub:00000000000000000000e5ab0435d604b71bea803c4f5afe1615a00574d00000000000000000000000003175935ac6d2eed57922fd5ea29c4cd180265d32000000000000000000000000d6d9eb36536a837336472f7392ea544306b73fb8000000000000000000000000c9f28ffe9ec07762d3235e0191d7fcc00339901f000000000000000000000000e08562b59b53640c142eeea474538f966e64e5820000000000000000000000006fda2b13a299db06ddf41b2f, stub:4e43078a9a315c740000000000000000000000006e9b973c2f2d04873dbf50b2fbe6e8c0345c1255000000000000000000000000a750da0b3f05431ac22df2b01d2885f7b0cc24cf00000000000000000000000054330c69ac5fc215998c10dcee58c9f0364a7bf900000000000000000000000001b3c7202c680271d7e4091f06b31366d68680c700000000000000000000000014df5677aa90ec0d64c631621dad80b44f9deaff0000000000000000000000005012, stub:6c13902f3b1cee2b7ebe4f55eff0a493056200000000000000000000000049123a65a643564778cb45ca8bb781629a1b9ffb000000000000000000000000d514604b0137e8cc385bd321d836bf268fd0ef110000000000000000000000003153acfab20f3ed7bff38058819a450a30ae4a360000000000000000000000006eb16877d08381dcd5df425805e97a8ca964fa53000000000000000000000000108d0f4dd578e0abd6f9d1b6f0ad23e17ca09f7100000000, stub:00000000000000007c51a97733c7006c054282fc537048600ad36f28000000000000000000000000e74a36c87c2a3e2310e9aa65cdf9187d786099cc0000000000000000000000008a7f7c5b556b1298a74c0e89df46eba117a2f6c100000000000000000000000020ffd3a8126eb53d67ac900c2ee18335b7dc989f00000000000000000000000030dcd1d4fd0094020cb022a09fb0d2d4ed574c9c000000000000000000000000e6a9b465c1535f397bf699c38d11, stub:c18f7bcaec8200000000000000000000000065c1f3fa4bf6c978311f26135c1244502539a7ff0000000000000000000000008e35488739fa0c55f9b7faee62991edb0e5b2faf000000000000000000000000b97644c87166c9edbffd5546b1a9f07@@@ 

Also, DEV_DEBUG_CURL=true chifra export 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 2>shit --first_block 14549339 --last_block 14549340 returns

blockNumber     transactionIndex        date    timestamp       from    to      ether   ethGasPrice     gasUsed hash   isError  encoding        compressedTx
TB: Error while reading stderr --  bufio.Scanner: token too long

UliGall avatar Jul 04 '22 11:07 UliGall

@dszlachta Can you spend a little bit of time looking at this: https://stackoverflow.com/questions/21124327/how-to-read-a-text-file-line-by-line-in-go-when-some-lines-are-long-enough-to-ca. I think this may be a problem for us but only for the above block (and probably others, but this one in particular).

tjayrush avatar Jul 04 '22 20:07 tjayrush

I think we can replace Scanner with this: https://pkg.go.dev/bufio#Reader.ReadString It reads characters until the given delimiter (\n in our case) and returns string.

dszlachta avatar Jul 05 '22 06:07 dszlachta

If you need more examples, happy to provide them.

chifra transactions 14513755.140

14514494.150
14513787.259
14549339.102
...

UliGall avatar Jul 05 '22 08:07 UliGall

If you need more examples, happy to provide them.

chifra transactions 14513755.140

14514494.150
14513787.259
14549339.102
...

What output do you get for chifra transactions 14513755.140?

I get:

{
  "data": [
    {
      "hash": "0x6094b0b28a8aee44323f7d3f2fb76e703ebed80f252c97d3544588dc86f0019e",
      "blockHash": "0x96a3493b7d7d0b88380808752fa4929a6ef04a849ff7b38a5b80b7701c3b54c1",
      "blockNumber": 14513755,
      "transactionIndex": 140,
      "nonce": 54,
      "timestamp": 1648997525,
      "from": "0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4",
      "to": "0x7f268357a8c2552623316e2562d90e642bb538e5",
      "value": 2.6e+17,
      "gas": 240291,
      "gasPrice": 63644565277,
      "maxFeePerGas": 67706536758,
      "maxPriorityFeePerGas": 1500000000,
      "input": "0xab834bab0000000000000000000000007f268357a8c2552623316e2562d90e642bb538e5000000000000000000000000d2894d1b968ee7e69fa30b8ce49e90f2d35eabf400000000000000000000000096a9d27076568a07e121d2945f5c1b4f610128de0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000baf2127b49fc93cbca6269fade0f7f31df4c88a7000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007f268357a8c2552623316e2562d90e642bb538e500000000000000000000000096a9d27076568a07e121d2945f5c1b4f610128de00000000000000000000000000000000000000000000000000000000000000000000000000000000000000005b3256965e7c3cf26e11fcaf296dfc8807c01073000000000000000000000000baf2127b49fc93cbca6269fade0f7f31df4c88a70000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002bc000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000039bb49f599a00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000006249b41600000000000000000000000000000000000000000000000000000000000000002b4eaae55d794aa1d11ef2c98e05999a84384c2086c6c1ede307521f6c01495e00000000000000000000000000000000000000000000000000000000000002bc000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000039bb49f599a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000624987a4000000000000000000000000000000000000000000000000000000006250f7f62dddd5e56dbaecf832f318061a7f8adb457104e2c9c1b703399353f3ef4f4cff0000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000006a000000000000000000000000000000000000000000000000000000000000007c000000000000000000000000000000000000000000000000000000000000008e00000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000b200000000000000000000000000000000000000000000000000000000000000b20000000000000000000000000000000000000000000000000000000000000001c000000000000000000000000000000000000000000000000000000000000001c8182d459205909c25ba6128bf81b96ab9170eaba7ca5f6caf7ccbc95dc151da26bea22841450315f7162b674492d2a788836ee4e13222ee3cc186b110b3057b18182d459205909c25ba6128bf81b96ab9170eaba7ca5f6caf7ccbc95dc151da26bea22841450315f7162b674492d2a788836ee4e13222ee3cc186b110b3057b1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e4fb16a5950000000000000000000000000000000000000000000000000000000000000000000000000000000000000000d2894d1b968ee7e69fa30b8ce49e90f2d35eabf400000000000000000000000031c823260528ba98e3bf797b2add2302af0973920000000000000000000000000000000000000000000000000000000000000258000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000c000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e4fb16a59500000000000000000000000096a9d27076568a07e121d2945f5c1b4f610128de000000000000000000000000000000000000000000000000000000000000000000000000000000000000000031c823260528ba98e3bf797b2add2302af0973920000000000000000000000000000000000000000000000000000000000000258000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000c000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e400000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e4000000000000000000000000000000000000000000000000000000000000000000000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
      "hasToken": 0,
      "receipt": {
        "contractAddress": "0x0",
        "gasUsed": 191519,
        "effectiveGasPrice": 63644565277,
        "logs": [
          {
            "address": "0x31c823260528ba98e3bf797b2add2302af097392",
            "logIndex": 291,
            "topics": [
              "0x8c5be1e5ebec7d5bd14f71427d1e84f3dd0314c0f7b2291e5b200ac8c7c3b925",
              "0x00000000000000000000000096a9d27076568a07e121d2945f5c1b4f610128de",
              "0x0000000000000000000000000000000000000000000000000000000000000000",
              "0x0000000000000000000000000000000000000000000000000000000000000258"
            ],
            "data": "0x"
          },
          {
            "address": "0x31c823260528ba98e3bf797b2add2302af097392",
            "logIndex": 292,
            "topics": [
              "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
              "0x00000000000000000000000096a9d27076568a07e121d2945f5c1b4f610128de",
              "0x000000000000000000000000d2894d1b968ee7e69fa30b8ce49e90f2d35eabf4",
              "0x0000000000000000000000000000000000000000000000000000000000000258"
            ],
            "data": "0x"
          },
          {
            "address": "0x7f268357a8c2552623316e2562d90e642bb538e5",
            "logIndex": 293,
            "topics": [
              "0xc4109843e0b7d514e4c093114b863f8e7d8d9a458c372cd51bfe526b588006c9",
              "0x00000000000000000000000096a9d27076568a07e121d2945f5c1b4f610128de",
              "0x000000000000000000000000d2894d1b968ee7e69fa30b8ce49e90f2d35eabf4",
              "0x0000000000000000000000000000000000000000000000000000000000000000"
            ],
            "data": "0x0000000000000000000000000000000000000000000000000000000000000000625cdefeb685fa53607857ecc66ab7a6941017f477e3e02cdadd982669a7bb9c000000000000000000000000000000000000000000000000039bb49f599a0000"
          }
        ],
        "status": 1
      },
      "gasUsed": 191519
    }
  ]
}

tjayrush avatar Jul 05 '22 11:07 tjayrush

uli@eth:~$ chifra transactions 14513755.140

yields

blockNumber     transactionIndex        date    timestamp       from    to      ether   ethGasPrice     gasUsed hash   isError  encoding        compressedTx
14513755        140     2022-04-03 14:52:05 UTC 1648997525      0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4      0x7f268357a8c2552623316e2562d90e642bb538e5      0.260000000000000000    0.000000063644565277    191519  0x6094b0b28a8aee44323f7d3f2fb76e703ebed80f252c97d3544588dc86f0019e      0       0xab834bab      0xab834bab(stub:0000000000000000000000007f268357a8c2552623316e2562d90e642bb538e5000000000000000000000000d2894d1b968ee7e69fa30b8ce49e90f2d35eabf400000000000000000000000096a9d27076568a07e121d2945f5c1b4f610128de0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000baf2127b49fc93cbca6269fade0f7f31df4c88a700000000000000000000000000000000000000000000, stub:0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007f268357a8c2552623316e2562d90e642bb538e500000000000000000000000096a9d27076568a07e121d2945f5c1b4f610128de00000000000000000000000000000000000000000000000000000000000000000000000000000000000000005b3256965e7c3cf26e11fcaf296dfc8807c01073000000000000000000000000, stub:baf2127b49fc93cbca6269fade0f7f31df4c88a70000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002bc000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000, stub:000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000039bb49f599a00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000006249b41600000000000000000000000000000000000000000000000000000000000000002b4eaae55d794aa1d11ef2c98e05999a84384c2086c6c1ed, stub:e307521f6c01495e00000000000000000000000000000000000000000000000000000000000002bc000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000039bb49f599a00000000000000000000000000000000, stub:00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000624987a4000000000000000000000000000000000000000000000000000000006250f7f62dddd5e56dbaecf832f318061a7f8adb457104e2c9c1b703399353f3ef4f4cff0000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000, stub:0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000, stub:00000000000100000000000000000000000000000000000000000000000000000000000006a000000000000000000000000000000000000000000000000000000000000007c000000000000000000000000000000000000000000000000000000000000008e00000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000b2000000000000000000000000000000000, stub:00000000000000000000000000000b20000000000000000000000000000000000000000000000000000000000000001c000000000000000000000000000000000000000000000000000000000000001c8182d459205909c25ba6128bf81b96ab9170eaba7ca5f6caf7ccbc95dc151da26bea22841450315f7162b674492d2a788836ee4e13222ee3cc186b110b3057b18182d459205909c25ba6128bf81b96ab9170eaba7ca5f6caf7ccbc95dc151da26bea22841450, stub:315f7162b674492d2a788836ee4e13222ee3cc186b110b3057b1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e4fb16a5950000000000000000000000000000000000000000000000000000000000000000000000000000000000000000d2894d1b968ee7e69fa30b8ce49e90f2d35eabf400000000000000000000000031c823260528ba98e3bf797b, stub:2add2302af0973920000000000000000000000000000000000000000000000000000000000000258000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000c0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000, stub:00000000000000000000000000e4fb16a59500000000000000000000000096a9d27076568a07e121d2945f5c1b4f610128de000000000000000000000000000000000000000000000000000000000000000000000000000000000000000031c823260528ba98e3bf797b2add2302af0973920000000000000000000000000000000000000000000000000000000000000258000000000000000000000000000000000000000000000000000000000000000000000000, stub:000000000000000000000000000000000000000000000000000000c000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e400000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000000000000000000000000000000000000000000000000000, stub:0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000, stub:00000000000000000000000000000000000000000000000000000000000000000000000000000000000000e4000000000000000000000000000000000000000000000000000000000000000000000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000, stub:000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000)

Okay,

with

uli@eth:~$ chifra transactions 14513755.140 -x json

{ "data": [
  {
    "hash": "0x6094b0b28a8aee44323f7d3f2fb76e703ebed80f252c97d3544588dc86f0019e",
    "blockHash": "0x96a3493b7d7d0b88380808752fa4929a6ef04a849ff7b38a5b80b7701c3b54c1",
    "blockNumber": 14513755,
    "transactionIndex": 140,
    "nonce": 54,
    "timestamp": 1648997525,
    "from": "0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4",
    "to": "0x7f268357a8c2552623316e2562d90e642bb538e5",
    "value": 260000000000000000,
    "gas": 240291,
    "gasPrice": 63644565277,
    "maxFeePerGas": 67706536758,
    "maxPriorityFeePerGas": 1500000000,
    "input": "0xab834bab0000000000000000000000007f268357a8c2552623316e2562d90e642bb538e5000000000000000000000000d2894d1b968ee7e69fa30b8ce49e90f2d35eabf400000000000000000000000096a9d27076568a07e121d2945f5c1b4f610128de0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000baf2127b49fc93cbca6269fade0f7f31df4c88a7000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000007f268357a8c2552623316e2562d90e642bb538e500000000000000000000000096a9d27076568a07e121d2945f5c1b4f610128de00000000000000000000000000000000000000000000000000000000000000000000000000000000000000005b3256965e7c3cf26e11fcaf296dfc8807c01073000000000000000000000000baf2127b49fc93cbca6269fade0f7f31df4c88a70000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002bc000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000039bb49f599a00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000006249b41600000000000000000000000000000000000000000000000000000000000000002b4eaae55d794aa1d11ef2c98e05999a84384c2086c6c1ede307521f6c01495e00000000000000000000000000000000000000000000000000000000000002bc000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000039bb49f599a0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000624987a4000000000000000000000000000000000000000000000000000000006250f7f62dddd5e56dbaecf832f318061a7f8adb457104e2c9c1b703399353f3ef4f4cff0000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000000000000000000000000000000000000000000000000006a000000000000000000000000000000000000000000000000000000000000007c000000000000000000000000000000000000000000000000000000000000008e00000000000000000000000000000000000000000000000000000000000000a000000000000000000000000000000000000000000000000000000000000000b200000000000000000000000000000000000000000000000000000000000000b20000000000000000000000000000000000000000000000000000000000000001c000000000000000000000000000000000000000000000000000000000000001c8182d459205909c25ba6128bf81b96ab9170eaba7ca5f6caf7ccbc95dc151da26bea22841450315f7162b674492d2a788836ee4e13222ee3cc186b110b3057b18182d459205909c25ba6128bf81b96ab9170eaba7ca5f6caf7ccbc95dc151da26bea22841450315f7162b674492d2a788836ee4e13222ee3cc186b110b3057b1000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e4fb16a5950000000000000000000000000000000000000000000000000000000000000000000000000000000000000000d2894d1b968ee7e69fa30b8ce49e90f2d35eabf400000000000000000000000031c823260528ba98e3bf797b2add2302af0973920000000000000000000000000000000000000000000000000000000000000258000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000c000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e4fb16a59500000000000000000000000096a9d27076568a07e121d2945f5c1b4f610128de000000000000000000000000000000000000000000000000000000000000000000000000000000000000000031c823260528ba98e3bf797b2add2302af0973920000000000000000000000000000000000000000000000000000000000000258000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000c000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e400000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000e4000000000000000000000000000000000000000000000000000000000000000000000000ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
    "hasToken": 0,
    "receipt": {
      "contractAddress": "0x0",
      "gasUsed": 191519,
      "effectiveGasPrice": 63644565277,
      "logs": [
        {
          "address": "0x31c823260528ba98e3bf797b2add2302af097392",
          "logIndex": 291,
          "topics": [
            "0x8c5be1e5ebec7d5bd14f71427d1e84f3dd0314c0f7b2291e5b200ac8c7c3b925",
            "0x00000000000000000000000096a9d27076568a07e121d2945f5c1b4f610128de",
            "0x0000000000000000000000000000000000000000000000000000000000000000",
            "0x0000000000000000000000000000000000000000000000000000000000000258"
          ],
          "data": "0x"
        },
        {
          "address": "0x31c823260528ba98e3bf797b2add2302af097392",
          "logIndex": 292,
          "topics": [
            "0xddf252ad1be2c89b69c2b068fc378daa952ba7f163c4a11628f55a4df523b3ef",
            "0x00000000000000000000000096a9d27076568a07e121d2945f5c1b4f610128de",
            "0x000000000000000000000000d2894d1b968ee7e69fa30b8ce49e90f2d35eabf4",
            "0x0000000000000000000000000000000000000000000000000000000000000258"
          ],
          "data": "0x"
        },
        {
          "address": "0x7f268357a8c2552623316e2562d90e642bb538e5",
          "logIndex": 293,
          "topics": [
            "0xc4109843e0b7d514e4c093114b863f8e7d8d9a458c372cd51bfe526b588006c9",
            "0x00000000000000000000000096a9d27076568a07e121d2945f5c1b4f610128de",
            "0x000000000000000000000000d2894d1b968ee7e69fa30b8ce49e90f2d35eabf4",
            "0x0000000000000000000000000000000000000000000000000000000000000000"
          ],
          "data": "0x0000000000000000000000000000000000000000000000000000000000000000625cdefeb685fa53607857ecc66ab7a6941017f477e3e02cdadd982669a7bb9c000000000000000000000000000000000000000000000000039bb49f599a0000"
        }
      ],
      "status": 1
    },
    "gasUsed": 191519
  }] }

Sorry, this one seems to be my mistake. Obviously, .csv cannot export the same detail as json.

UliGall avatar Jul 05 '22 11:07 UliGall

chifra list 0xd2894d1b968ee7e69fa30b8ce49e90f2d35eabf4 | wc

Hi. Thanks so much for all the work you did on this issue. I want to say that this issue should be fixed now. (Yes, it's taken this long.)

We're going to release the new version v.0.40.0 this weeked or very early next week.

I am going to close this issue, but, if, after moving to the new version (once it's ready) you find the error still happens (it doesn't for us), please re-open.

tjayrush avatar Sep 02 '22 01:09 tjayrush