Issue with triggers in phase 'after'

[kraney]

Expected Behavior (Mandatory)

Adding a trigger in phase 'after' does not disrupt any operations

Actual Behavior (Mandatory)

When I create a trigger in phase 'after' - even when I reduce it to simply doing "RETURN NULL", it causes some transactions to fail with a stack trace like the following:

2020-03-17 23:39:53.368+0000 WARN  Error executing trigger onNewDeviceConfig_PublishForModel in phase after Getting deleted relationship data should have been covered by the tx state
java.lang.IllegalStateException: Getting deleted relationship data should have been covered by the tx state
	at org.neo4j.kernel.impl.coreapi.TxStateTransactionDataSnapshot.relationship(TxStateTransactionDataSnapshot.java:354)
	at org.eclipse.collections.impl.lazy.primitive.CollectLongToObjectIterable$2.next(CollectLongToObjectIterable.java:82)
	at org.neo4j.kernel.impl.util.ValueUtils.asListValue(ValueUtils.java:192)
	at org.neo4j.kernel.impl.util.ValueUtils.of(ValueUtils.java:106)
	at org.neo4j.kernel.impl.util.ValueUtils.asParameterMapValue(ValueUtils.java:249)
	at org.neo4j.kernel.impl.factory.GraphDatabaseFacade.execute(GraphDatabaseFacade.java:404)
	at apoc.trigger.Trigger$TriggerHandler.lambda$executeTriggers$0(Trigger.java:282)
	at java.util.concurrent.ConcurrentHashMap.forEach(ConcurrentHashMap.java:1597)
	at apoc.trigger.Trigger$TriggerHandler.executeTriggers(Trigger.java:272)
	at apoc.trigger.Trigger$TriggerHandler.afterCommit(Trigger.java:305)
	at org.neo4j.kernel.internal.TransactionEventHandlers.afterCommit(TransactionEventHandlers.java:149)
	at org.neo4j.kernel.internal.TransactionEventHandlers.afterCommit(TransactionEventHandlers.java:48)
	at org.neo4j.kernel.impl.api.TransactionHooks.afterCommit(TransactionHooks.java:76)
	at org.neo4j.kernel.impl.api.KernelTransactionImplementation.afterCommit(KernelTransactionImplementation.java:922)
	at org.neo4j.kernel.impl.api.KernelTransactionImplementation.commit(KernelTransactionImplementation.java:745)
	at org.neo4j.kernel.impl.api.KernelTransactionImplementation.closeTransaction(KernelTransactionImplementation.java:594)
	at org.neo4j.internal.kernel.api.Transaction.close(Transaction.java:178)
	at org.neo4j.bolt.v1.runtime.TransactionStateMachine$State.closeTransaction(TransactionStateMachine.java:469)
	at org.neo4j.bolt.v1.runtime.TransactionStateMachine$State$2.commitTransaction(TransactionStateMachine.java:402)
	at org.neo4j.bolt.v1.runtime.TransactionStateMachine.commitTransaction(TransactionStateMachine.java:144)
	at org.neo4j.bolt.v3.runtime.TransactionReadyState.processCommitMessage(TransactionReadyState.java:93)
	at org.neo4j.bolt.v3.runtime.TransactionReadyState.processUnsafe(TransactionReadyState.java:53)
	at org.neo4j.bolt.v3.runtime.FailSafeBoltStateMachineState.process(FailSafeBoltStateMachineState.java:48)
	at org.neo4j.bolt.v1.runtime.BoltStateMachineV1.nextState(BoltStateMachineV1.java:144)
	at org.neo4j.bolt.v1.runtime.BoltStateMachineV1.process(BoltStateMachineV1.java:92)
	at org.neo4j.bolt.messaging.BoltRequestMessageReader.lambda$doRead$1(BoltRequestMessageReader.java:89)
	at org.neo4j.bolt.runtime.MetricsReportingBoltConnection.lambda$enqueue$0(MetricsReportingBoltConnection.java:68)
	at org.neo4j.bolt.runtime.DefaultBoltConnection.processNextBatch(DefaultBoltConnection.java:191)
	at org.neo4j.bolt.runtime.MetricsReportingBoltConnection.processNextBatch(MetricsReportingBoltConnection.java:86)
	at org.neo4j.bolt.runtime.DefaultBoltConnection.processNextBatch(DefaultBoltConnection.java:139)
	at org.neo4j.bolt.runtime.ExecutorBoltScheduler.executeBatch(ExecutorBoltScheduler.java:171)
	at org.neo4j.bolt.runtime.ExecutorBoltScheduler.lambda$scheduleBatchOrHandleError$2(ExecutorBoltScheduler.java:154)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.neo4j.internal.kernel.api.exceptions.EntityNotFoundException: Unable to load RELATIONSHIP with id 147.
	at org.neo4j.kernel.impl.storageengine.impl.recordstorage.RecordStorageReader.relationshipVisit(RecordStorageReader.java:258)
	at org.neo4j.kernel.impl.coreapi.TxStateTransactionDataSnapshot.relationship(TxStateTransactionDataSnapshot.java:349)
	... 35 more

The specific transactions that fail with this error seem to be ones that include a deleted relationship. And that relationship was itself deleted by a 'before' trigger, if that matters.

How to Reproduce the Problem

I have not yet been able to reduce the problem to a straightforward procedure to reproduce it. It seems to be a complex interaction between triggers.

Steps (Mandatory)

1. Add any trigger to phase 'after'
2. Attempt to execute a transaction that delete a relationship from a 'before' trigger.

Specifications (Mandatory)

Currently used versions

Versions

• OS: Linux
• Neo4j: 3.5.16
• Neo4j-Apoc: 3.5.0.9

[kraney]

Correction - that should be version 3.5.16-enterprise

[vga91]

@kraney With the latest Apoc version there is a phase: "afterAsync" to prevent some Transaction errors like this. Please, could you try with this?

[magaton]

I am having the same problem. I have the trigger in before phase that is listening to removed property on a deleted relationship (HAS_BOSS). And I want to rename relationships (not the one I am listening to) but the incoming HAS_BOSS relationship of the start node (Client)

     CALL apoc.trigger.add('hasBossStartDateRemoved', 
    "UNWIND $removedRelationshipProperties['startDate'] AS map
      WITH map.relationship AS rel, map.old AS startDate WHERE type(map.relationship) = 'HAS_BOSS'
      WITH rel, startNode(rel) AS client, startDate
      MATCH (client)<-[ohs:HAS_BOSS]-(orphan:Client) 
      WITH ohs
      CALL apoc.refactor.setType(ohs, 'HAD_BOSS') YIELD output
      RETURN count(*)")

  This blows with `Getting deleted relationship data should have been covered by the tx state`
  but the strange thing is that all updates/deletes actuallysucceed.

[srnnukala]

Were you able to solve the problem? I have a similar issue for which I opened an issue (https://github.com/neo4j-contrib/neo4j-apoc-procedures/issues/3216)

Expected Behavior: Delete nodes and relationships while having apoc.trigger.propertiesByKey($removedNodeProperties,"progenitorId")

Sample apoc trigger in production :

CALL apoc.trigger.add('requiredIdentifiers_removedNodeProperties_preventRemovalOnProgenitorId', 'UNWIND apoc.trigger.propertiesByKey($removedNodeProperties,"progenitorId") AS prop WITH prop.node AS n, prop.old AS oldValue SET n.progenitorId=oldValue RETURN 0', {phase:'before'}); CALL apoc.trigger.add('requiredIdentifiers_removedRelationshipProperties_preventRemovalOnExternalId', 'UNWIND apoc.trigger.propertiesByKey($removedRelationshipProperties,"externalId") AS prop WITH prop.relationship AS r, prop.old AS oldValue SET r.externalId=oldValue RETURN 0', {phase:'before'});

Actual Behavior: Neo.ClientError.Transaction.TransactionHookFailed: Error executing trigger requiredIdentifiers_removedNodeProperties_preventRemovalOnProgenitorId in phase before org.neo4j.graphdb.TransactionFailureException: Unable to complete transaction.

How to Reproduce the Problem -> CREATE DATABASE dummy -> CALL apoc.trigger.add('requiredIdentifiers_removedNodeProperties_preventRemovalOnProgenitorId', 'UNWIND apoc.trigger.propertiesByKey($removedNodeProperties,"progenitorId") AS prop WITH prop.node AS n, prop.old AS oldValue SET n.progenitorId=oldValue RETURN 0', {phase:'before'}); -> CREATE (node:dummyActivity {progenitorId : 123456}) -> MATCH (n:Activity) where n.progenitorId = 123456 delete n apoc_trigger_error.pdf

@conker84 - can you please look into this when you get time? I am using ( Neo4j: 4.4.8, Neo4j-Apoc: 4.4.0.4 ) Background: With apoc.triggers in place, we don't allow users to delete/update the node properties or relationship properties, however we give them access delete nodes and relationship and are successful using neo4j 3.5.18 and apoc 3.5.0.12-all.

[vga91]

@srnnukala See https://github.com/neo4j-contrib/neo4j-apoc-procedures/issues/3216#issuecomment-1275932742