Ledger icon indicating copy to clipboard operation
Ledger copied to clipboard

Faster Database queue draining

Open SulpherStaer opened this issue 2 years ago • 3 comments

Is your feature request related to a problem?

Database queue is still draining. If you exit now actions WILL be lost

At the moment, the database queue does not drain fast enough on shutdown and the server gets cut off by the watchdog saying it took to long to shut down (presumed crash) and kills it after 60s. Ledger is not fast enough to handle this because of reasons?

Describe the solution you'd like.

I'd like Ledger to more quickly drain the queue so I do not lose any actions during restarts.

Describe alternatives you've considered.

I think more connections to the mysql database should be made to clear the queue more quickly. Another plugin I use (Plan, Player Analytics) uses a configurable 'max connections' that it will use to save its data.

I have the feeling Ledger just uses ONE connection to the mysql, because it is Sooooo slow to do any inspection or saving.

Agreements

  • [X] I have searched for and ensured there isn't already an open issue regarding this.
  • [X] I have ensured the feature I'm requesting isn't already in the latest supported mod version.

Other

No response

SulpherStaer avatar Mar 27 '22 21:03 SulpherStaer

Is there any update/progress on this? For context, I'm working on setting up a custom pack/server (GitHub). It's a quite technical community, so I'm expecting a fair amount of large-scale projects such as world eaters/terraforming. We ran a (somewhat) small test and it hadn't finished processing half an hour later when we restarted the server. It doesn't help that /ledger status gives no information about the size of the queue. I'd really like to be able to use Ledger, but taking hours to process the queue isn't very tenable.

object-Object avatar May 22 '23 16:05 object-Object

Ledger's MySQL connection seems to be as inefficient as it can get. Not only the database queue draining is slow, but the inefficient database connection is also causing gaps in the logs the mod is supposed to collect.

Ledger should use a pool of connections instead of just one single, inefficient connection. That's what CoreProtect does as well.

MeesJ avatar Nov 05 '23 20:11 MeesJ

Any update on this? Doesn't look like there's been of a migration over to a connection pool but it'd significantly lessen the performance hit of logging overall.

Database drain is a significant part of our server's shutdown time, so.

JosTheDude avatar Dec 03 '23 22:12 JosTheDude