Clarify/fix documentation of skips and controlling rollback with FaultTolerantStepBuilder

[goatfryed]

trafficstars

I'm referring to 5.1.5. Configuring Skip Logic and 5.1.7. Controlling Rollback of the Spring batch reference documentation.

I think, the current documentation does not accurately document the current functionality and would like to help to improve on it.

In short:

• The current documentation does not accurately describe the actual behavior
• It contradicts the javadoc
• It does not clearly state behavior of ItemProcessor, which creates additional confusion in combination with the above.

The documentaton states that we should use .skip(FlatFileParseException.class) to skip exceptions, that do not cause errors to the batch. I'd expect that this means that it behaves more or less like a filter with regards to processing, but it actually rolls back the current chuck and processes it one item at a time, which might be kind of unexpected, if we use non-transactional readers and processors. It's possible to find better documentation at other places, but it sounds misleading here.

Now, in 5.1.7, the documentation states

By default, regardless of retry or skip, any exceptions thrown from the ItemWriter cause the transaction controlled by the Step to rollback. If skip is configured as described earlier, exceptions thrown from the ItemReader do not cause a rollback. However, there are many scenarios in which exceptions thrown from the ItemWriter should not cause a rollback, because no action has taken place to invalidate the transaction. For this reason, the Step can be configured with a list of exceptions that should not cause rollback, as shown in the following example:

If skip is configured as described earlier This can only refer to .skip(FlatFileParseException.class). So it reenforces the idea, that skip shouldn't cause a rollback. Also, this paragraph directly contradicts the javadocs that state, that the noRollback definition is ignored in writing.

	/**
	 * Mark this exception as ignorable during item read or processing operations. Processing continues with no
	 * additional callbacks (use skips instead if you need to be notified). Ignored during write because there is no
	 * guarantee of skip and retry without rollback.
	 *
	 * @param type the exception to mark as no rollback
	 * @return this for fluent chaining
	 */
	public FaultTolerantStepBuilder<I, O> noRollback(Class<? extends Throwable> type) { /* ... */}

So, to skip exceptions in item reader, processor and writer, but rollback the current chunk and retry one by one, we should use

stepBuilderFactory.get("...")
                .faultTolerant()
                    .skip(ValidationException.class)

To silently skip exceptions in item reader or processor, while continuing with the current chunk, we should use

stepBuilderFactory.get("...")
                .faultTolerant()
                    .noRollback(ValidationException.class)

And if we want to skip an exception in item reader or processor, but handle it in a SkipListener, we have to define

stepBuilderFactory.get("...")
                .faultTolerant()
                    .skip(ValidationException.class)
                    .noRollback(ValidationException.class)

But we should take extra care, because this would also skip the same exception in a writer, together with a rollback.

If you agree with me that the documentation might benefit from a bit clarification and improvement here and I haven't stated anything wrong, I'd be happy to propose an update here

[kenchanmc]

I also notice that if noRollback and a skip limit is configured for an exception , the skip will not work as I expect. For example, in case of this exception only happen in the ItemProcessor , I would expect if the times of this exception happen is more than the skip limit , the whole job will fail and stop running , but now it will just slightly ignore these failed item and contiunes processing .

What are confuse me more is that if retry is also configured at the same time , the behaviour seems to be inconsisent and do not work . I try to summary them as below based on my observation . Please note that I assume that the exception will only thrown from the ItemProcessor :

skip	retry	observe
configure with some limit	disable	the step does not stop even the expection happen exceed the skip limit. It just filter out the failed item
configure with some limit	enable	same as above , and it also does not retry the the failed item....
not configured	disable	I belive if the skip is not configured , it is default to LimitCheckingItemSkipPolicy with zero limit which mean it will fail fast if an exception happen and the job should stop? . It just filter out the failed item
not configured	enable	Same as above , but this time it can retry

If there is nonoRollback configured , it works exactly as what the document describes .I am not sure if it is the expected behaviuour for the noRollback exception .If yes , I also agree to fix the documentation. Otherwise , it seems that skip and retry are broken when work with noRollback exception...

[finci-square]

Not to hijack this thread, but i'm also somewhat confused about the intended outcome of these configurations. I posted a SO question here:

https://stackoverflow.com/questions/69932026/how-to-configure-step-which-fails-on-any-read-exception-never-rolls-back-but-s

[xenogew]

I tried with

• .noRollback()
• .skip()
• .skip().noRollback() On the same Exception class, and I have custom ItemWriter that writing the file each line of item that was read from ItemReader, but behavior still run it twice that why when ever the Exception was thrown, the line of item that have been written is doubled.

Could anyone give me a hint?

[ievgennaida]

Have the same feeling that docs are really misleading and it's takes a not of debug efforts to configure needed behavior. For example: I have two exception types: termination and warning. Both should just flush messages to the database (commit) and 'termination' should stop the batch job.

You will never get on how to configure something like this from the documentation. Especially considering where error has occurred: reader, writer, processor