Andrus Adamchik

Need help reproducing this. I just [created a test](https://github.com/nhl/link-move/blob/master/link-move/src/test/java/com/nhl/link/move/itest/ExtractorReloadingIT.java) that changes the extractor and reruns the task. Everything seems to work correctly. Could there be an error in the XML...

One caveat is that `LmTask` is not a functional interface, so we'll need to figure out how to deal with passing parameters and sync tokens between the stages.

After further analysis, the low hanging fruit here is executing batches of the same task in parallel. Batches are presumably independent from each other.

+1 on DbEntity sync. I had a similar idea when we discussed it with @rzen . @atomashpolskiy : let's discuss offline some time after July 3 (or sooner over video...

Yeah, as a Cayenne developer, I am very familiar with the issue (see for instance https://github.com/apache/cayenne/blob/master/cayenne-server/src/main/java/org/apache/cayenne/access/translator/select/TrimmingQualifierTranslator.java ). We will probably need to write tests with cross-DB sources at some point...

On MySQL the magic sauce seems to be `useCursorFetch=true&defaultFetchSize=500` parameters added to the URL that switches the ResultSet to the batch mode. Here are the before and after memory profiles:...

@vitalz : could you post your code changes that help to address the problem?

I opened a Jira in Cayenne: https://issues.apache.org/jira/browse/CAY-2034 This is a limitation of Cayenne expression parser.

I guess that depends on how this is implemented. I haven't given that much thought yet, but theoretically this can be a special abstraction handled by LM core.

I am thinking more in the direction of a new task, like `SoftDeleteTask` or something.