Add NPU support for Llava

[HelloWorldBeginner]

[NPU] Support Llava training and inference for Ascend NPU.

I've modified some codes to add support for NPU, allowing LLAVA to perform both training and inference on NPU. It works fine on NPU. NPU Training NPU Evaluate NPU Inference

[HelloWorldBeginner]

Hi @haotian-liu Can you review my codes?

[MengqingCao]

cool! It helps for NPU users like me, thx!

[xslasd]

非常棒，我运行起来了

[zmf2022]

when I run this code，RuntimeError: "slow_conv2d_cpu" not implemented for 'Half'

[ReyRen]

size mismatch for O.weight: copying a param with shape torch.Size ([4096, 1024]) from checkpoint, the shape in current model is torch.! size mismatch for O. bias: copying a param with shape torch.Size ( [4096]) from checkpoint, the shape in current model is torch.Size ([0] size mismatch for 2.weight: copying a param with shape torch.Size([4096, 4096]) from checkpoint, the shape in current model is torch. size mismatch for 2.bias: copying a param with shape torch.Size ([40961) from checkpoint, the shape in current model is torch.Size([0].

@HelloWorldBeginner When I run with thses commits, the error occured like above. Can you please give me some hints? thx

[ReyRen]

I get "RuntimeError: The size of tensor a (32) must match the size of tensor b (1258) at non-singleton dimension 3" error, need help, please.

[Yangr116]

Can you reproduce the results using NPU? @HelloWorldBeginner