Prabhu Teja

Imdb lstm problem

4

Hi, This pull request is to include the IMDb LSTM sentiment classification task (from https://arxiv.org/abs/1910.11758 ) I've added the dataset, network architecture, and I've tested it with the example code...

Hi, I'm trying to get the Synthia transfer to work, and I cant see to do so. I followed the hyperparameter settings in #14 and using the source model in...

**Describe the bug** Torch nn.Modules have a feature enabled by [`set_extra_state`](https://pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module.set_extra_state) and [`get_extract_states`](https://pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module.get_extra_state) that are used to store additional buffers to state_dict. The current implementation of the [`get_fp32_state_dict_from_zero_checkpoint`](https://deepspeed.readthedocs.io/en/stable/model-checkpointing.html#deepspeed.utils.zero_to_fp32.get_fp32_state_dict_from_zero_checkpoint) doesn't seem...

Most of the `DataLoader` instances use no additional workers for data loading. This can be a bottleneck when performing non-trivial data augmentations on images with a large enough batch size,...

The [Component](https://github.com/awslabs/Renate/blob/dev/src/renate/updaters/learner_components/component.py#L74) class implements a [`_register_parameters` ](https://github.com/awslabs/Renate/blob/dev/src/renate/updaters/learner_components/losses.py#L34) method that registers the loss' constructor arguments. This currently works without glitches where Renate manages the `state_dict` of Learner, Model and others. However,...

The current experimentation code in [benchmarking](https://github.com/awslabs/Renate/blob/main/src/renate/benchmark/experimentation.py#L295) runs evaluation in the same thread as the (subsequent) trainings. This is a problem when using DDP as the first evaluation (Line 295) creates...