ONE icon indicating copy to clipboard operation
ONE copied to clipboard

Published 20 hours ago • verified publisher icon Samsung

PoC: Block weight quantize tool for LLM [skip ci]

Open hseok-oh opened this issue 6 months ago • 0 comments

  • Block quantization for LLM: FullyConnected, Gather
  • Decide quantize type by circle-quantizer parameter: --block_quantize_weights (Q4_0, Q8_0)
  • Skip quantization by circle-quantizer parameter: --skipsize_block_quantize (default: 0)

Caution: It's for PoC of circle format and test model generation. Not for compiler implementation. #13742 #13743

hseok-oh avatar Aug 26 '24 11:08 hseok-oh