[Feature Request] DynamicTensorSpec a TensorSpec for sequential data

[dtsaras]

Motivation

There is currently no TensorSpec such that it allows sequential data. For example, if an environment was supposed to represent a sentence, it would be represented as (batch_size, sequence_len, embedding_size). Such a TensorSpec could also be used for graphs or any other application that is represented as sequence.Since, the length sequence is not fixed it would helpful to allow to define a TensorSpec with shape (batch_size, -1, embedding_size). In such cases, the embedding_size is fixed and -1 represent the dynamic shape of the length of the sequence.

Solution

Such a TensorSpec would have to be described by either a nested tensor, LazyStackedTensorDict or a padded tensor.

Alternatives

Currently, it's not possible to have any such structure in TorchRL. By using other TensorSpecs can solve the issue if as long as we avoid enforcing the shape restrictions and we don't use batched ends.

Checklist

• [x] I have checked that there is no similar issue in the repo (required)