Incomprehensible growth of `accounts`

[diman-io]

Problem

Time on the pictures is UTC+3

Last 30 days: Drops on the chart are restarts.

Last 24 hours:

Last 2 hours:

Log from 14:00 to 14:30 (17:00 - 17:30 on the picture) https://drive.google.com/file/d/1Pe569W9u0Iv-mid-Q-2r16UMwO93pjVa/view

First time I mentioned this here https://github.com/solana-labs/solana/issues/26259#issuecomment-1169116655

[t-nelson]

Which SW version was this node running? The startup messages have been pruned from the attached log

[brooksprumo]

@diman-io From your comment on the other issue, it looks like you are running v1.10.26. Is that correct?

[diman-io]

@t-nelson @brooksprumo Yes that's right, attached log from 1.10.26

Yesterday I ran 1.10.28 with accounts on disk (not ramdisk). I will write here when accounts start to grow.

[yokosta]

only the growth of the given metric, update the node was 08/04/2022, and again .

[jeffwashington]

Andrew and I ran into this issue today. We are investigating with @brooksprumo.

[jeffwashington]

https://discord.com/channels/428295358100013066/1008859241689399297/1008861578956918794

[jeffwashington]

#27461

[diman-io]

I found that the jump in the number of open files does not start when snapshots are enabled. Before that they were disabled (*-snapshot-interval-slots 0). After they were enabled a smooth growth continues, but there is no rapid growth. So this trick can be used for debugging (But the jump event will have to wait one-two days :( )

[diman-io]

@jeffwashington @brooksprumo I'm sorry for pinging, just to make sure you don't miss 👆

[brooksprumo]

@diman-io Do you have a node on testnet? If yes, did https://github.com/solana-labs/solana/pull/27461 fix your issue?

Edit: Also, you can pull in this change for v1.10 on mnb: https://github.com/solana-labs/solana/pull/27650

[diman-io]

@brooksprumo I didn't see this issue on my testnet node anytime

Thank you, I'll pull in #27650 at next update.