thement
thement
I did run test dump and then parsed it with the analyzer script and it works. I would maybe think about making a separate directory for all the analysis-related scripts...
What about writing tests that compare the python implementation of tokenizer from original llama code with the current tokenizer implementation in llama.cpu and then fixing the llama.cpu tokenizer? This way...
> https://guillaume-be.github.io/2020-05-30/sentence_piece seems to document the SentencePiece algorithm fairly well I implemented something very similar here: https://github.com/ggerganov/llama.cpp/pull/242 I haven't tested it yet, I'm on a train and can't download the...
> Alpaca-lora author here. I've added a script to merge and convert weights to state_dict in my repo [(link)](https://github.com/tloen/alpaca-lora/blob/main/export_state_dict_checkpoint.py). Curious to see it run on llama.cpp :) Ha! You've got...
> Alpaca-lora author here. I've added a script to merge and convert weights to state_dict in my repo [(link)](https://github.com/tloen/alpaca-lora/blob/main/export_state_dict_checkpoint.py). Curious to see it run on llama.cpp :) Anyway, here's a...
How about making a directory named `contrib` and put the script there? It could be a place to put other related and useful tools.
Could you please check how it behaves with the BPE tokenizer which is not yet merged? https://github.com/ggerganov/llama.cpp/pull/252 Could you also copy here the tokens that were generated for the "list...
> > There has been a conversion script posted previously: > > [ggerganov/llama.cpp/issues/324#issuecomment-1476227818](https://github.com/ggerganov/llama.cpp/issues/324#issuecomment-1476227818) > > with a link to this script: > > https://gist.github.com/eiz/828bddec6162a023114ce19146cb2b82 > > I tried it once...
Any progress on this issue? I'm trying to figure out how to add external expression pedal to G1Four (I've ordered one and it's on the way). > The next question...
Hey, thanks for the research. The reason I was looking into this was that I was gonna use a control voltage (from LFO) instead of expression pedal, so I was...