Unable to generate transaction debugs for some transactions

[quickchase]

Describe the bug I have a large, but also incomplete, list of transactions that cannot be traced on an Optimism node: transactions.txt

Some, if not all of them, would OOM the node if you just tried to run debug_traceTransaction without any specific tracers or disableMemory - most of them will not even return with callTracer

For example, this returns nothing:

curl -s -H 'Content-Type: application/json' http://localhost:8545 -d '{"jsonrpc":"2.0","method":"debug_traceTransaction","params":["0xdfcd11b288d4f8a45c32e1d60d353043fe9593f1742b51df6e4d4601910f566e", {"tracer": "callTracer", "timeout": "1h"}],"id":0}'

It doesn't seem to crash the node or anything, it just doesn't return any data, or useful logs.

Even with {"disableMemory":true} this debug will eat through 60GB of RAM and crash the node.

{"jsonrpc":"2.0","method":"debug_traceTransaction","params":["0xdfcd11b288d4f8a45c32e1d60d353043fe9593f1742b51df6e4d4601910f566e", {"disableMemory":true}],"id":0}

This OOM thing is a known issue inherited from Geth that has been discussed many times: https://github.com/ethereum/go-ethereum/issues/22244

Nethermind and Erigon generally solved this problem by implementing JSON-RPC streaming but Geth has yet to provide a workaround outside of debug_standardTraceBlockToFile which requires direct physical access to the node's storage, something that generally can't be provided by RPC providers...

Regardless, I gave it a shot: The transaction 0xdfcd11b288d4f8a45c32e1d60d353043fe9593f1742b51df6e4d4601910f566e is in the following block:

BLOCKHASH="0x456422c35361bf34792dbeed17edbf7f91016aed788c67633dee25e74a2279ff"
curl -s -H 'Content-Type: application/json' http://localhost:8545 -d "
  {
      \"jsonrpc\":\"2.0\",
      \"id\":1,
      \"method\":\"debug_standardTraceBlockToFile\",
      \"params\":[
          ${BLOCKHASH},
          {
              \"disableMemory\":true,
              \"overrides\":{}
          }
      ]
  }
"

I expect this to return a file path in the form of something like:

{"jsonrpc":"2.0","id":1,"result":["/temp/block_0x456422c3-0-0xdfcd11b2-4028396738"]}

However, it does not. I DO see that A file is generated in /tmp but it's not really clear to me if this is full and complete working output, since I'm not getting a file path back, my concern is that it's crashing or stopping at some point and the file is incomplete?

I do not know enough about the output from these commands to analyze the file myself to know if it's good or not... I'm also just a middleman here in trying to provide traces over JSON-RPC.

I don't think we should have to resort to debugging to a file in the first place, but if there's no other way then I still expect the curl command to return success or failure and return the file path instead of an empty response:

> POST / HTTP/1.1
> Host: localhost:8545
> User-Agent: curl/7.68.0
> Accept: */*
> Content-Type: application/json
> Content-Length: 317
> 
* upload completely sent off: 317 out of 317 bytes
* Empty reply from server
* Connection #0 to host localhost left intact

Also related #2228

[pjanuario]

Would be useful to get some information about this problem.