reparam-discrete-diffusion icon indicating copy to clipboard operation
reparam-discrete-diffusion copied to clipboard

about decoding topk_masking

Open violet-sto opened this issue 9 months ago • 1 comments

Hi

Thanks for your excellent work. I have a question about the rate schedule for topk_masking.

As described in the appendix, "To ensure that the degree of noise decreases as the generation process proceeds, we schedule k to increase from 1 to N monotonically as the diffusion step t goes from T to 1." However, in the code (https://github.com/HKUNLP/reparam-discrete-diffusion/blob/26ee286b281edc6284d74f809465b3e6d42507a6/discrete_diffusion/discrete_diffusions/discrete_diffusion_base.py#L177), the masked k tokens with the lowest confidence instead of the highest. Are there any inconsistencies here?

Best regards

violet-sto avatar May 10 '24 15:05 violet-sto