AITemplate icon indicating copy to clipboard operation
AITemplate copied to clipboard

Published 20 hours ago • verified publisher icon facebookincubator

Could AITemplate support mix precision inference?

Open sanbuphy opened this issue 1 year ago • 2 comments

like fp32/fp16 , fp32/bf16.

Thank you very much .

sanbuphy avatar Aug 09 '23 08:08 sanbuphy

@sanbuphy Unfortunately AITemplate doesn't support mixed precision/auto-cast functionality.

@chenyang78 CC'ing you for visibility, given the recent discussion on the topic

sgrigory avatar Aug 15 '23 07:08 sgrigory

@sanbuphy Unfortunately AITemplate doesn't support mixed precision/auto-cast functionality.

@chenyang78 CC'ing you for visibility, given the recent discussion on the topic

I see, so I cannot use AIT to compile any mixed precision models with amp.autocast in PyTorch. Thank you for your response.

sanbuphy avatar Aug 15 '23 08:08 sanbuphy