Xiaodong (Vincent) Huang
Xiaodong (Vincent) Huang
@yuc8939 , the input shape is `[S, B, 3*E, 1, 1]` because we use conv1x1 in the builder, also we plan to rework this to use MM layer, so that...
@yuc8939 , use `S` for the outermost, sample code to follow: https://github.com/NVIDIA/TensorRT/blob/main/demo/BERT/builder.py#L385 thanks!
Closing since no activity for more than 3 weeks, please reopen if you still have question, thanks!
@haitong , The fused kernels are not open sourced since it has internal tool chain dependency, we just encode the raw cubin in the github. Only the unfused implementation are...
Closing since no activity for more than 3 weeks, please reopen if you still have question, thanks!
> We don't support float shape tensor now but I know there is some development on it, @ttyio do we have a target release for this feature? We will have...
@zhanghongyong123456 , before 8.5, maybe try constant folding using polygraphy: polygraphy surgeon sanitize model.onnx --fold-constants --output model_folded.onnx If this cannot be folded, we have to wait 8.5, the schedule is...
@zhanghongyong123456 , it's around this week, thanks!
@Henry-Avery will be supported in 8.5. We will release it soon. Thanks!
> > @Henry-Avery will be supported in 8.5. We will release it soon. Thanks! > > Any update？ FYI, 8.5EA is available in 22.09. https://docs.nvidia.com/deeplearning/frameworks/pytorch-release-notes/rel-22-09.html#rel-22-09