denoising-diffusion-pytorch icon indicating copy to clipboard operation
denoising-diffusion-pytorch copied to clipboard

Published 20 hours ago • verified publisher icon lucidrains

A question on time embedding

Open shixishi opened this issue 2 years ago • 1 comments

Snipaste_2022-11-24_16-42-01

I don't understand why (half_dim - 1) is used here. According to the transformer's time-coding formula, there should be "emb = math.log(10000) / half_dim", I don't think half_dim should minus 1 here.

shixishi avatar Nov 24 '22 08:11 shixishi

@shixishi i'm not sure, it was faithfully transcribed from https://github.com/hojonathanho/diffusion/blob/master/diffusion_tf/nn.py#L100

lucidrains avatar Nov 24 '22 17:11 lucidrains