mergekit icon indicating copy to clipboard operation
mergekit copied to clipboard

Published 20 hours ago • verified publisher icon arcee-ai

Add --load-in-4bit and --load-in-8bit for HF eval backend

Open cg123 opened this issue 8 months ago • 0 comments

Allows using bitsandbytes quantization in mergekit-evolve when a) not using vLLM and b) not using in-memory mode.

cg123 avatar May 26 '24 23:05 cg123