AnnaTest

msvsr_reds

Too much time during next in dataloader

1

### Bug description during training, I find that the next() in dataloader will spend 10~20s, I already set higher num_worker form 8 to 32, It's still spend long time in...

Add enable_delay scale_loss for sharding stage2

5

### PR types Others ### PR changes Others ### Description Others

[defaultAllocator.cpp::deallocateAsync::64] Error Code 1: Cuda Runtime (operation not supported)

5

## Description when I convert onnx to tensorrt it alway error like: ` Error[1]: [defaultAllocator.cpp::deallocateAsync::64] Error Code 1: Cuda Runtime (operation not supported)XXX failure of TensorRT X.Y when running XXX...