Which exceptions are expected during rec template construction?

[Zettelkasten]

In _SubnetworkRecCell._construct_template, we currently fail directly on these exception types:

fail_directly_exception_types = (DataNotFound, LayerNotFound, BehaviorVersion.RequirementNotSatisfied)

When debugging, it in general is much nicer to just fail on the first exception, because many times all other exceptions are consequences of the first one. And with the quite messy RETURNN error log, this is annoying to find.

In the config I work with, the only type of recurrence is via the targets of the previous frame. I think, in this case, no exception at all is expected. So in my case, I'd ideally like to always fail directly when some exception occurs.

How is this in general? Which exceptions are expected? And do we even need to rely to expect any exception, isn't there a better, more direct way, like to specify the out shape explicitly for such layers?

In particular, I wonder about VerifyOutShapeException and AssertionError. If a VerifyOutShapeException occurs, this means that the template could be constructed, but didn't give the shape I expected. This means, all layers that depend on this layer will also be constructed and in most cases then also throw other errors. Can we add this to fail_directly_exception_types?

AssertionError on the other hand can be thrown almost everywhere. Often, this can also cause such cascade of errors. But I guess sometimes this is also expected. Can we clean this up? Maybe by throwing more explicit error types in the case where such error could be expected, and then to directly fail on AssertionError?

[albertz]

VerifyOutShapeException is also expected. When the template construction runs into a dependency loop, so there is some layer which can not be constructed, it assumes some simple dummy shape for it (currently [B]), and then it tries to construct the others again. That way, some dims are initially not there but during the successive template construction they will be added. With a specified out_shape, it would not work that way.

Example:

cumsum: {class: combine, kind:add, from: ["prev:cumsum", "data:source"]}

This has the dependency loop right there. It cannot really know the shape of prev:cumsum, so it uses a dummy shape. Then cumsum gets some shape (via data:source here) (and thus prev:cumsum as well). Then it again constructs it, now with given prev:cumsum shape.

With out_shape:

cumsum: {class: combine, kind:add, from: ["prev:cumsum", "data:source"], out_shape: {B,F}}

Now it would directly fail with VerifyOutShapeException. So this does not even work here. But in other cases for more complex networks, maybe it can recover via some other layer, and then some earlier VerifyOutShapeException will not occur anymore.

Actually, we could maybe use a specified out_shape instead of a dummy shape when it is specified. Or when VerifyOutShapeException occurs. But at some point we have to check all layers again.

[albertz]

Maybe, if all layers which are used via prev:... specify out_shape, and we use that shape always, then in fact no other exceptions should occur.

[Zettelkasten]

Thanks for the example, I never really understood how this logic worked before. This logic is pretty cool (essentially computing a fix point), but also pretty implicit. I would prefer if template construction would never expect any exception because that would make make the process more direct and easier to debug. So like you suggest, I would probably enforce the user to specify out_shape always for layers used via prev:.... We anyway enforce this already for extended broadcasting.

But I haven't used recurrency in my configs for a very long time now, so I don't really know what people making use of this "fix point computation logic" actually think about it. I also remember that it was buggy in the past (#357). Maybe it's nicer to use now.