vector-quantize-pytorch icon indicating copy to clipboard operation
vector-quantize-pytorch copied to clipboard

Published 20 hours ago • verified publisher icon lucidrains

What is the difference between Grouped VQ and Multi-codebook VQ when used for Autoregressive LLM training?

Open JohnHerry opened this issue 1 year ago • 0 comments
trafficstars

If I use a Grouped-VQ with two groups and codebook size =100, then the count of final speech tokens is 100 * 100 = 10000; If I use a 2-codebook VQ with codebook size= 100, then the count of the final speech tokens is also 100 * 100 = 10000?

JohnHerry avatar Aug 30 '24 05:08 JohnHerry