frankframework icon indicating copy to clipboard operation
frankframework copied to clipboard

Allow liquibase checksums to be cleared (automatically on a daily basis) and lock table to be emptied via console.

Open Viv1003 opened this issue 2 years ago • 2 comments

Sometime systems give a 504 when trying to access the console.

According to cloudwatch it is trying to acquire a liquibase lock during startup but it is already taken.

This causes the startup to loop because the system will not get past this and it will cause AWS to startup a new instance with the same problem.

It is not clear when this occurs to prevent this loop from happening (if possible) allow liquibase checksums to be cleared automatically on a daily basis and lock table to be emptied via the console.

Reporter Vivienne Jacobs

Screenshots If applicable, add screenshots to help explain your problem.

Viv1003 avatar Jul 18 '23 09:07 Viv1003

Manual is a good idea (automatically/daily is a bad idea)

jacodg avatar Aug 07 '23 11:08 jacodg

Manual is a good idea (automatically/daily is a bad idea)

It's a 504 when you try to log on, but really the system is not up.

The adminstrator of the system is not aware the table has been erroneously locked. This can happen outside a regular deployment, for instance when an EMS sever is unreachable and the adapter is in error. The AWS healtcheck will reboot the server and a lock may occur without cleaning. Over time more instances may be rebooted due to failing healtchecks and eventually all systems will be unreachable. In that case there wil be an incident and the console will also be unavilable to unlock the table. You would have to clear the table from the RDS.

Is it possible to do a daily automatic lock check with a check on the duration of the lock? We can safely assume that there is a lock loop issue when the lock is over an hour(or other arbitrary length of time) long.

BonnoSmeele avatar Aug 07 '23 13:08 BonnoSmeele

I think an automatic lock check could make sense but where to implement that? On startup, before running Liquibase, check if the lock exists and exceeds a certain age, perhaps?

tnleeuw avatar Nov 19 '24 10:11 tnleeuw