Gavin Li
> Same issue for airllm "...pytorch_model.bin.index.json should exists" with HF model "argilla/notus-7b-v1-lora". Could you please fix this? Thanks This is a Mistral model, can you try the following: ``` from...
> Hi @lyogavin , > > > Which hf model repo ID are you using? > > It's a 7B llama2 model The repo is https://huggingface.co/bkai-foundation-models/vietnamese-llama2-7b-40GB This is gated however....
working on systematic benchmark.
` quantization_config=BitsAndBytesConfig( load_in_4bit=args.bits == 4, load_in_8bit=args.bits == 8, llm_int8_threshold=6.0, llm_int8_has_fp16_weight=False, bnb_4bit_compute_dtype=compute_dtype, bnb_4bit_use_double_quant=args.double_quant, bnb_4bit_quant_type=args.quant_type ),` 这个部分去掉就可以。
I ran into the same issue. Is there any updates? @gwenzek Can you pls help checking with https://dl.fbaipublicfiles.com/'s owner what's their rate limiting policy?
好像有个bug，改了以后好像了没有这个问题了。我再试试。 set_peft_model_state_dict新版本不return model了。 关于bitsandbytes，从这个提示看： /home/ubuntu/miniconda3/envs/ghostaienv/lib/python3.8/site-packages/bitsandbytes/cuda_setup/main.py:136: UserWarning: WARNING: Compute capability < 7.5 detected! Only slow 8-bit matmul is supported for your GPU! V100 bitsandbytes感觉只是会慢一些，我先试试。
I have this issue too. It's critical for SEO apps. currently there's no way to set Head aschild
I have to do this: ``` const AccordionTrigger = React.forwardRef< React.ElementRef, React.ComponentPropsWithoutRef >(({ className, children, ...props }, ref) => { const content = props.asChild ? ( {children} ) : (...
现在reference model有两个作用，一个是作为initial model，另一个是作为DPO optmization时候的reference 计算KL散度，保证优化结果不会偏离太远。 第一个的话需要保持两个模型一致的。 第二个理论上讲可以不一样。
will look into supporting gptq