vector-quantize-pytorch

vector-quantize-pytorch copied to clipboard

Loss and Backprop Details

3

Hi, During training the vqvae backprops on multiple losses. While inputting feature maps to the model, we are given a loss, shoud I manually backpropagate and update weights through (the...

Malik7115

Missing parameter of beta

3

Hi, in the original VQVAE paper, the commit_loss is defined as ``` (quantize.detach()-x) ** 2 + beta * (quantize - x.detach() ** 2) ``` where the beta is usually to...

Corleone-Huang

No way of training the codebook

5

Hi! Could you please explain how the codebook vectors are updated if the codebook vectors are not required to be orthogonal? 1) `embed` tensors in both Euclidean and CosineSim codebooks...

RafailFridman

RVQ loss

2

firstly,thanks to ur code,and then i have a quentison,when i use RVQ,it will return 8 loss,how do u solve this problem,u add all loss to one?

zxj329

How to gather cluster_size and embed_sum when use DataParallel (DP)

1

I notice the code can gather data for EMA in DDP, but I filed it when I use DP, because distributed.all_reduce need distributed.init_process_group firstly. How to gather data in DP

liuyoude

Plugging vector-quantize-pytorch into taming-transformers

2

Hi, I noticed your architecture could be plugged within the pipeline from https://github.com/CompVis/taming-transformers. I have proposed a code here (https://github.com/tanouch/taming-transformers) doing that. It enables to properly compare the different features...

tanouch

VectorQuantize not JIT-safe

8

The current code controls the program based on the values of tensors (like the "initted" buffer) and will not work when compiling a jit trace.

JackMcCoy

Hi can you explain the DDP example? Do you know what is needed for it to work with pytorch lightning & tpu?

cajoek

How to train this?

2

Hi, I want to use this package to experiment with data different than images (multivariate time series). I see that the `commitment_loss` that is returned is not a tensor, but...

umbertov

There are two interesting features (low implementation overhead) from the paper [Robust Training of Vector Quantized Bottleneck Models](https://arxiv.org/pdf/2005.08520.pdf): - 3.A Importance of proper scaling - Batch normalisation - This is...

danieltudosiu