benchmark icon indicating copy to clipboard operation
benchmark copied to clipboard

Published 20 hours ago • verified publisher icon pytorch

Fix llama_v2_7b_16h for torch.jit.trace

Open thiagocrepaldi opened this issue 1 year ago • 1 comments

Original error: Attention using SDPA can not be traced with torch.jit.trace when no attention_mask is provided. To solve this issue, please either load your model with the argument attn_implementation="eager" or pass an attention_mask input when tracing the model.

thiagocrepaldi avatar Jan 17 '24 23:01 thiagocrepaldi