CrispStrobe

or you can just use a llamafied version of the model

for illustration: [Phi3_ORPO_Unsloth.ipynb](https://github.com/CrispStrobe/unsloth/blob/main/Phi3_ORPO_Unsloth.ipynb)

nice! (btw in unsloth/Phi-3-mini-4k-instruct modelcard also says q4 but that is ofc unquantized)

Great work! I have noted some suggestions, corrections, and additions here: https://github.com/snackpackwayne/social-app/pull/1/commits/b5e225eea0243c32eaa0a0583d2e46e384c4cd5b Please excuse that due to lack of time i am unable to review above comments or engage in...

yes tried this but same problem remains

the "template not found" will occur no matter what yaml config. ok i am starting to look into this quickly. when i set template_id="uj9cgdrng7" (where i run script https://gist.githubusercontent.com/CrispStrobe/57e138a872caa04a15275c13095fb63d/raw/7c2ae3995c367a74199449678a25063e73f5c335/gistfile1.txt )...

in my tries it used it thus https://github.com/CrispStrobe/unsloth/blob/main/Phi3_ORPO_Unsloth.ipynb but now you can adapt to the non-llamafied model ofc

have you seen https://github.com/cognitivecomputations/extract-expert/blob/main/extract.py and https://huggingface.co/mmnga/Mixtral-Extraction-4x7B-Instruct-v0.1/blob/main/notebook/convert_mixtral_8x7b_to_4x7b_extract.ipynb ?

thank you. i think it actually did work once, but just tested it and it is not working, token generation speed is veeeeerrrrrryyyy sllloooooow... from the logs: time=2024-03-12T21:41:12.613+01:00 level=INFO source=images.go:710...

i actually had one video tdr failure blue screen when testing this now. anyway, i can quite ollama (from the icon in taskbar) and restart, and, with some luck, sometime...