position embedding？ i cannot understand,the code?

[henbucuoshanghai]

half = dim // 2 freqs = torch.exp( -math.log(max_period) * torch.arange(start=0, end=half, dtype=torch.float32) / half ).to(device=timesteps.device) args = timesteps[:, None].float() * freqs[None] embedding = torch.cat([torch.cos(args), torch.sin(args)], dim=-1)

[henbucuoshanghai]

can u explain it，tks

[xiaohu2015]

you can refer to transformer paper

[henbucuoshanghai]

whereis 10000?