LlamaSpeak cannot run with Llama-3.1-70B-Instruct

[SuWeipeng]

I'm trying to run a 70B model on my Jetson AGX Orin(64x64GB), but it automatically interrupts when I simply replace the 8B model. How can I get the 70B model to run?

When I run the command below, something interrupt the process automatically.

jetson-containers run --env HUGGINGFACE_TOKEN=hf_xxxxx  \
  dustynv/nano_llm:r36.3.0   \
  python3 -m nano_llm.agents.web_chat --api=mlc  --debug   \
    --model meta-llama/Meta-Llama-3.1-70B-Instruct     \
    --asr=whisper --tts=piper

If I run with 8B model, it works very well, for example:

jetson-containers run --env HUGGINGFACE_TOKEN=hf_xxxxx  \
  dustynv/nano_llm:r36.3.0   \
  python3 -m nano_llm.agents.web_chat --api=mlc  --debug   \
    --model meta-llama/Meta-Llama-3.1-8B-Instruct     \
    --asr=whisper --tts=piper