Teerapol Saengsukhiran
Teerapol Saengsukhiran
@st1vms There is a problem. The version of BNB is 0.35.4 which is kind of outdated, and the latest version of Peft requires `bitsandbytes>=0.37.0`
@st1vms I tried BNB 0.39.0 The dependencies seem fine. However, when I tried to lora finetune according to this https://colab.research.google.com/drive/1jCkpikz0J2o20FBQmYmAGdiKmJGOMo-o?usp=sharing The Jupyter kernel crash, reason: undefined ...
@st1vms I retried with a new virtual env and change from .ipynb to .py This is the result. (torch3) win@win-MS-7E02:/mnt/1df6b45e-20dc-41ca-9a04-b271fd3a4940/Learn$ /usr/bin/env /home/win/torch3/bin/python /home/win/.vscode-oss/extensions/ms-python.python-2023.20.0-universal/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher 60843 -- /mnt/1df6b45e-20dc-41ca-9a04-b271fd3a4940/Learn/finetune.py /home/win/torch3/lib/python3.10/site-packages/transformers/utils/generic.py:441: UserWarning: torch.utils._pytree._register_pytree_node is...
@arlo-phoenix pls add support for ROCM 5.7
@arlo-phoenix rocm 6.0 docker has arrived https://hub.docker.com/r/rocm/pytorch/tags
@65a Thank, It works.
Agree, we need to fulfill this instruction/chat template for each model
@radna0 I only have triton 3 that come with pytorch for Rocm. Btw, I wonder if matmulfreellm can be implemented in Jax efficiently.
do it work fine with this model too? https://huggingface.co/scb10x/llama-3-typhoon-v1.5-8b-vision-preview since it is trained by Bunny method