lucylq
lucylq
Run with: ``` python -m examples.models.llama2.runner.generation --pte --tokenizer --prompt= ```
Now require kernel support for bf16. Export: python3 torchchat.py export llama3 --dtype [fp16 | bf16] --output-pte-path llama3.pte ``` (.venv) (base) [[email protected] /data/users/lfq/torchchat (lfq.export-bf16)]$ python3 torchchat.py generate llama3 --device cpu --pte-path...
Export bf16 is good, require kernel support now. ``` (.venv) (base) [[email protected] /data/users/lfq/torchchat (lfq.export-bf16)]$ python3 torchchat.py generate llama3 --device cpu --pte-path llama3.pte --prompt "Hello my name is" Warning: checkpoint path...
Revert https://github.com/pytorch/executorch/pull/4101