tianylin
tianylin
When I use `animation.save('out.gif')` without specifying writer, it seems that celluloid would use `ffmpeg` as the default writer, thus resulting in an error (it tries to mux H.264 into GIF)....
I just read the code but found it hard to understand how the data is organized. To be specific, i can't seem to understand what the label stand for. Is...
Hi, I was wondering if you could provide a list of dependencies in this repo (e.g., in the form of `requirements.txt`). I think this should help with the reproducibility of...
batch内序列肯定有padding的，这部分好像在loss func里没有看到有处理的。
In `RandomAttention`, seems like you're using a batch of random matrix for synthetic random attention map: ```python self.random_attn = torch.randn(batch_size, n_head, max_seq_len, max_seq_len, requires_grad = True) ``` I don't think...
阅读了一下这里的代码，对MoCo的实现是在dataloader里面维护了一个队列存储了历史样本（而不是样本的embedding），但是Momentum encoder的思想，如果我理解的无误，应该是缓存momentum encoder的输出embedding，并通过动量更新的方法使embedding空间的波动不至于太大（同时不用占用额外显存）。是否是实现有误呢？
scripts中给到的脚本中的路径没有处理好（e.g., `scripts/bert/run_simcse_grid.sh`），一些变量的命名也不很直观（例如，`output_dir`，`new_output_dir`），需要运行代码又必须手动修改这里的路径，能否提供一个路径的实例和一个可以运行的简单脚本呢？
这一行末尾`\`前少了个空格，导致参数读取有问题，请修正： https://github.com/Namco0816/PT-BERT/blob/dba403c01aa2acdf8659b7a7167d41471aacb656/scripts/re-produce_result.sh#L12
# 公式13.3 KL散度的标记出错 P311 - [ ] 这里KL中间的逗号按惯例应该修改为：