mesh icon indicating copy to clipboard operation
mesh copied to clipboard

versionpurge job getting Job Queue stuck.

Open cemusta opened this issue 2 years ago • 4 comments

Gentics Mesh Version, operating system, or hardware.

  • v1.8.2

Operating System

  • Linux, mesh instance dockerized

Problem

  • There are some version purge jobs in the list. When we re-queue these jobs, the cms instance starts a deadlock. This deadlock makes the instance stop responding to API calls.

  • When we restart the instance, that job is still in the queue (not failed). if re-queue another job, the cms instance still takes the first version purge job ( 1st job in the queue ) instead. and the same process happens (deadlock and cms getting not responsive)

  • Because of this, all other jobs got stuck in the queue. We cannot make any schema changes. All schema migration jobs are in the job list and cannot be processed.

problematic job:

		{
			"uuid": "16eadf6a89a8486e9da27546ac854ee7",
			"created": "1970-01-01T00:00:00Z",
			"type": "versionpurge",
			"status": "QUEUED",
			"properties": {
			},
			"completionCount": 0,
			"warnings": [
			]
		}

Reproducer

  • reprocessing jobs via API re-produce the same problem.

Expected behaviour and actual behaviour.

  • Job should either fail or got completed.

cemusta avatar May 03 '22 10:05 cemusta

Hi @cemusta , Thanks for the ticket. Do you have any chance to get the Mesh logs? Normally on Mesh restart all the queued jobs start, and if something happen, this is usually depicted in the logs. We have also fixed one issue regarding the job stability, releasing it as https://github.com/gentics/mesh/releases/tag/1.8.5 / - couldn't you please check if this version solves you problem? Thanks in advance!

plyhun avatar Jul 05 '22 08:07 plyhun

Hi,

We will update to 1.8.5 and update this issue afterward.

Best, Cem

cemusta avatar Jul 05 '22 10:07 cemusta

Mesh v1.8.6 is recently released, targeting more job improvements - couldn't you please try it?

plyhun avatar Jul 11 '22 15:07 plyhun

Will try and update.

cemusta avatar Jul 12 '22 13:07 cemusta