Damian Nardelli

I just ran some tests for this long text: https://pastebin.com/raw/FE3maftq I see tons of `scala.collection.immutable.$colon$colon` object instances are getting created consuming almost 10gb of data to process that text I...

Hello @santisra could you please clarify what @davidrissato is asking for? I think it's important to understand from the documentation if the AtomicCounter issue can be worked around or not...

Can torch.cuda.amp be used only for inferences on a FP32 model? See https://github.com/NVIDIA/apex/issues/750 and https://github.com/NVIDIA/apex/issues/809 I couldn't find an example in https://pytorch.org/docs/master/notes/amp_examples.html Maybe just wrapping up the model call with...

@mcarilli what about the opt_level O1 / O2 , etc... I can't find whether that's already natively supported by `torch.cuda.amp` - it looks like there's no opt_level option in `torch.cuda.amp`...

Another question: Will this be supported by Torchscript?

I reduced the amount of memory with `torch.no_grad()`... But fp16 definitely improved the inference times in pytorch. What times are you getting for tensorflow and for what length of tokens?