`DatabaseHandles' in ethconfig.Config cannot be set through TOML

[barwod]

System information

Geth Version: 1.10.13-stable Git Commit: 7a0c19f813e285516f4b525305fd73b625d2dec8 Architecture: amd64 Go Version: go1.17.3 Operating System: linux GOPATH= GOROOT=go

Expected behaviour

Close files that aren't in use / have a settable limit

Actual behaviour

Every database file it opens stays open for the duration of the sync

Steps to reproduce the behaviour

Snap sync the blockchain on an AWS EFS volume

Backtrace

I use @holiman recomendation from issue https://github.com/ethereum/go-ethereum/issues/23922 but i get error:

"Fatal: /root/.ethereum/config.toml, line 13: field corresponding to `DatabaseHandles' in ethconfig.Config cannot be set through TOML"

My config.toml:

[Eth]
NetworkId = 4
SyncMode = "snap"
EthDiscoveryURLs = ["enrtree://AKA3AM6LPBYEUDMVNU3BSVQJ5AD45Y7YPOHJLEF6W26QOE4VTUDPE@all.rinkeby.ethdisco.net"]
SnapDiscoveryURLs = ["enrtree://AKA3AM6LPBYEUDMVNU3BSVQJ5AD45Y7YPOHJLEF6W26QOE4VTUDPE@all.rinkeby.ethdisco.net"]
NoPruning = false
NoPrefetch = false
TxLookupLimit = 2350000
LightPeers = 100
UltraLightFraction = 75
DatabaseCache = 512
DatabaseFreezer = ""
DatabaseHandles = 30000
TrieCleanCache = 154
TrieCleanCacheJournal = "triecache"
TrieCleanCacheRejournal = 3600000000000
TrieDirtyCache = 256
TrieTimeout = 3600000000000
SnapshotCache = 102
Preimages = false
EnablePreimageRecording = false
RPCGasCap = 50000000
RPCEVMTimeout = 5000000000
RPCTxFeeCap = 1e+00

Any idea how to fix this?

[ligi]

thanks - seems we need to add it to the TOML - as a workaround you can set it via CLI

[barwod]

How? Could you please provide this command here?

[kkosowsk]

hello, I have the same problem, but I don't see an option to set this parameter via CLI?

[holiman]

RIght, no that's true, it's not settabie, not via config nor cli. It's calculated on the fly,

[holiman]

I think a valid solution would be, not to set the max database handles explicitly, but rather to override the max file limit. So instead of geth checking the max file limit and deriving the db handle limit from that, we'd check use the smallest of fdlimit and override_fd_limit and use that to derive the db handle limit.

[eliane345]

Preciso muito de aprender a esta linguagem de códigos abertos em português

[eval-exec]

I think a valid solution would be, not to set the max database handles explicitly, but rather to override the max file limit. So instead of geth checking the max file limit and deriving the db handle limit from that, we'd check use the smallest of fdlimit and override_fd_limit and use that to derive the db handle limit.

Hello holiman, I would like to solve this issue. I found DatabaseHandles was set by MakeDatabaseHandles() at here: https://github.com/ethereum/go-ethereum/blob/fb3a6528cfa49f623570575c4fe9e8a716cfcdf7/cmd/utils/flags.go?_pjax=%23js-repo-pjax-container%2C%20div%5Bitemtype%3D%22http%3A%2F%2Fschema.org%2FSoftwareSourceCode%22%5D%20main%2C%20%5Bdata-pjax-container%5D#L1058-L1070

but, how can I derive the db handle limit from the smallest of fdlimit and override_fd_limit ? I think the smallest fdlimit of a process is uncertain.

[joaoepj]

but, how can I derive the db handle limit from the smallest of fdlimit and override_fd_limit ? I think the smallest fdlimit of a process is uncertain.

@eval-exec , I think this issue only arises on very busy nodes running under restricted enviroments like AWS. Using the minimum value between limit and DatabaseHandles as argument to fdlimit.Raise() would suffice to avoid the postponed filesystem errors. In the other hand, geth should free file descriptors as soon as possible.

[tuanggolt]

open

[s1na]

Closed by https://github.com/ethereum/go-ethereum/pull/24477.