vector-quantize-pytorch

vector-quantize-pytorch copied to clipboard

Allow for sample_codebook_temp to be annealed?

1

Hi! I've seen commit 1da641b (version = '1.5.16') with message 'allow for temperature to be annealed'. But the sample_codebook_temp seems constant during training. How to turn on annealed temperature?

Pachark

How about ChamferDistance instead of cdist to calculate KNN?

2

Hi, I propose to use KNN function that is used in point cloud field instead of `torch.cdist`. VectorQuantization finds nearest vector inside of the codebook (B, M, D) with given...

Kitsunetic

Support perplexity to measure index collapse

4

Hi! Thanks for you work. In the implementation of VQVAE (https://github.com/deepmind/sonnet/blob/v2/sonnet/src/nets/vqvae.py#L89C1-L89C1), perplexity is used as an evaluation measure for VQ codebook. It monitors whether index collapse happens. Does this project...

Pachark

Hi，I meet some debug when i inference the model

3

What are the causes of the following problems? Missing key(s) in state_dict: "quantizer.vq.layers.12._codebook.inited", "quantizer.vq.layers.12._codebook.cluster_size", "quantizer.vq.layers.12._codebook.embed", "quantizer.vq.layers.12._codebook.embed_avg", "quantizer.vq.layers.13._codebook.inited", "quantizer.vq.layers.13._codebook.cluster_size", "quantizer.vq.layers.13._codebook.embed", "quantizer.vq.layers.13._codebook.embed_avg", "quantizer.vq.layers.14._codebook.inited", "quantizer.vq.layers.14._codebook.cluster_size", "quantizer.vq.layers.14._codebook.embed", "quantizer.vq.layers.14._codebook.embed_avg", "quantizer.vq.layers.15._codebook.inited", "quantizer.vq.layers.15._codebook.cluster_size", "quantizer.vq.layers.15._codebook.embed", "quantizer.vq.layers.15._codebook.embed_avg", "quantizer.vq.layers.16._codebook.inited", "quantizer.vq.layers.16._codebook.cluster_size",...

GitYesm

Error when using `sync_codebook=True` with `torch.compile`

1

Probably because https://github.com/pytorch/pytorch/issues/96742

BakerBunker

codebook merge

1

How to merge two codebook into one codebook?

zhouilu

Will the 16 or 32 codebook dimension hamper the representational capacity of autoencoders?

2

Thanks for the excellent collection and implementation of vector quantization techniques! It is very helpful for me to get to know about this technique and study the details. I've integrated...

yilinliu77

Support half precision for VQ and FSQ

6

Hi, I notice it is not support other preicsion. I made this tiny change and I tried a simple sample and it works on fp16 and bf16. I noticed that...

JunityZhan

[BUG] Residual VQ - self.embed.data[ind][mask] = sampled - RuntimeError: shape mismatch:

10

Hi all, I noticed that using `ResidualVQ` as: ``` ResidualVQ( dim=Z_CHANNELS, # 512 num_quantizers=NUM_QUANTIZERS, # 2 codebook_size=CODEBOOK_SIZE, # 16 * 1024 stochastic_sample_codes=True, shared_codebook=True, commitment_weight=1.0, kmeans_init=True, threshold_ema_dead_code=2, quantize_dropout=True, quantize_dropout_cutoff_index=1, quantize_dropout_multiple_of=1, )...

dwromero

LSQ half precision problem #116

7

I believe there is a similar problem to #116. ``` File "/tmp/ray/session_2024-06-30_09-41-50_254745_1/runtime_resources/pip/ed0d17a5f9a959a3d03116db0bba20a6c15cac27/virtualenv/lib/python3.10/site-packages/vector_quantize_pytorch/lookup_free_quantization.py", line 273, in forward x = self.project_in(x) File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, **kwargs) File "/opt/conda/lib/python3.10/site-packages/torch/nn/modules/module.py",...

denadai2