Kava

Failing to generate MLIR for Llama3 using TorchMLIR

1

I downloaded Llama3 model to hf-files directory and then trying to use AutoModelForCausalLM to load the model, and then convert the transformer portion to MLIR. > huggingface-cli download meta-llama/Meta-Llama-3-8B --local-dir...

Failed to legalize operation 'torch.aten.scaled_dot_product_attention' (Self attention torch -> tosa conversion)

5

I am running Llama'3 attention layer's Torch mlir to TOSA mlir conversion pipeline (command below), but seeing torch.aten.scaled_dot_product_attention as illegal function. Can someone help me figure out a pass for...

'torch.aten.mm' that was explicitly marked illegal

2

I am running torch mlir to tosa mlir conversion pipeline (command below), but keep seeing torch.aten.mm as illegal fn. Can someone help me figure out a pass for tosa conversion...

When generating Torch MLIR, is there a way to write weights and model in separate file? My model weights are huge for Llama3 layers, and this is leading to giant...