honeybee icon indicating copy to clipboard operation
honeybee copied to clipboard

add sentencepiece to pip requirements

Open ojh6404 opened this issue 1 year ago • 0 comments

sentencepiece is needed when from pipeline.interface import get_model; model, tokenizer, processor = get_model(ckpt_path, use_bf16=True). I added sentencepiece to pip requirements for LlamaTokenizer.

ojh6404 avatar Jan 24 '24 11:01 ojh6404