Llama7B repo empty currently

[grundman]

The LLama 7B repo is currently empty, I can't find the conversion scripts or find converted .so files on the hugging face page. Is there an ETA for those to be uploaded?

https://github.com/quic/ai-hub-models/tree/main/qai_hub_models/models/llama_v2_7b_chat_quantized

[srikris-sridhar]

We cannot distribute the 4-bit quantized model so we are working on making it such that you can create this yourself. Stay tuned!

[YixinSong-e]

We cannot distribute the 4-bit quantized model so we are working on making it such that you can create this yourself. Stay tuned!

Is there any update?

[Junhyuk]

can you please update the way how to convert to qnn model from llama 7B ?

[r-wheeler]

Can you please release this?

[mestrona-3]

Hi All, we are happy to see all the interest around Llama 7B. We are actively working on this and will share an update via Qualcomm AI Hub Community Slack when it is ready, we're working to get this out as soon as possible!

[bhushan23]

Hi @grundman @r-wheeler @YixinSong-e @Junhyuk

Llama2 export scripts are out now https://github.com/quic/ai-hub-models/tree/main/qai_hub_models/models/llama_v2_7b_chat_quantized

Please give it a try and let us know how it goes

[ao-zz]

@bhushan23 Will the .so or .bin files be uploaded on the hugging face page ? Since I do not have a device with 40+ GB RAM to meet the demand of export script.

If not, will the RAM required by export script be smaller in the future ?

[bhushan23]

We do emit instructions to increase swap space to help you export these models if memory is low.

We cannot share exported models as we cannot distribute weights. But, our goal is to make it easy for users to be able to export this.