Robert Flynn

icon indicating a website link http://www.robertflynn.co.uk

icon company The University of Sheffield icon location Sheffield icon location PhD Student at Sheffield University @ CDT for Speech and Language Technologies - ASR + LMs

Results 3 issues of Robert Flynn

Default expansion factor of 2 for inner dimension in conv module not present in other implementations

Hey lucidrains, firstly thanks for providing this implementation! I've noticed in the ConformerConvModule the inner dimension is increased by an expansion_factor which defaults to 2 `inner_dim = dim * expansion_factor`...

[CLI]: Crashing run due to logging gradient data

2 comment

### Describe the bug Had my run crash due to logging gradient data. Is there a reason why there are no try except clauses so wandb crashing never crashes training...

cli

using ssm_state and conv_state during training

10 comment

Currently, from the way the library is written conv_state and ssm_state are only used when generating one step at a time using the InferenceParams. It would be useful to use...