returnn
returnn copied to clipboard
rwth-i6
The RWTH extensible training framework for universal recurrent neural networks
In the case of net dicts created by returnn_common, the construction heuristics (#1129) to resolve circular loops are never needed. For all recurrent layers (all layers accessed via `"prev:..."`), we...
For the `tf.data` pipeline (#292) using: ```python def dataset_pipeline(context): from returnn.tf.compat import v1 as tf dataset = context.get_returnn_dataset() dataset = dataset.padded_batch( batch_size=12, padded_shapes=tf.data.get_output_shapes(dataset), drop_remainder=False, ) dataset = context.map_producer_to_consumer(dataset) dataset =...
See our multi-GPU training doc: https://returnn.readthedocs.io/en/latest/advanced/multi_gpu.html In case you do not have very fast direct connections between the GPUs (nvlink, only for the big professional cards), we always recommend async...
In principle, RETURNN supports arbitrary dtype, as `Data` can just have any `dtype`. However, many layers do not really allow to configure that. Most layers would just take the same...
I'm not really sure this is a bug, or what we can really do about. However, I open this now because I noticed again a quite huge effect: ``` output/exp_fs_base/conformer_pre10_d384_h6_blstmf2_specaug_attdrop01_posdrop01_aux48_bhv14/recog_results_per_epoch/040...
At some point we need to have a cleanup logic for the PyTorch engine, similar to `cleanup_old_models` (https://github.com/rwth-i6/returnn/blob/master/returnn/tf/engine.py#L2103). Do we want to have a compatible logic with the same defaults,...
> A bit more meta: With all our logic for dim tags, which should actually make it easier to avoid any reshape problems or other shaping problems, why do we...
Now that we only need to target Python >=3.7 (#487), we can make use of annotations, and also use forward declarations (no need to stringify types), although this needs: ```python...
We should be able to use [HuggingFace datasets](https://huggingface.co/docs/datasets/index) directly in RETURNN. I guess the most canonical way would be to write a RETURNN `Dataset` for this. Maybe derived from `CachedDataset2`....
Currently we are working on converting LMs from TensorFlow to Onnx. As Onnx does not allow persistent state between session runs, in contrast to TF, we need to manage the...
