Support dim tag with static shape and dyn_size_ext

[albertz]

dyn_size_ext means that it has different sequence lengths per batch entry (or whatever dyn_size_ext shape is).

Currently this implies a dynamic shape (None dimension), and vice versa, a dynamic shape (None dimension) implies that we have some dyn_size_ext.

We should remove this assumption, and allow a static shape together with dyn_size_ext. It's mostly already supported.

This is needed for example for TPU (see #1162) or also for TFLite or ONNX in some cases.

[albertz]

Note there is already Dim.is_dynamic but a lot of code is checking dimension is None directly. All such code should use is_dynamic instead.

Note that the is_dynamic implementation in principle might look like return self.dyn_size_ext is not None. However, this is probably not exactly correct, as we need to care about two special cases:

• Dim not yet defined, and then used as out_spatial_dim or so. Then dimension is None to signal this is yet undefined, and also dyn_size_ext is None.
• Dim is dynamic, but the dyn_size_ext is unknown yet.

We cannot really properly distinguish those cases.

[albertz]

We might need to introduce a new flag, specifically for this, like _is_dynamic or so.

[albertz]

Note, I'm already continually fixing any dimension is None checks.

Note that those checks don't necessarily always map to is_dynamic. They sometimes also check whether the dim is defined/known yet. If that is the actual meaning (and you have to see that from the context), then it should be replaced by not is_dim_known. Or even not dim.is_dim_known() or (dim.dyn_size_ext and dim.dyn_size_ext.placeholder is None).

[albertz]

We also need to be careful in get_for_batch_ctx. In the case the dim is not yet defined, this should also not do anything. Otherwise a static dim cannot properly be set.