Bryan Li

I have this issue as well. It looks like the guide might not be updated for the latest version of openSMILE? I say this because, for example, the guide still...

+1

I guess this commit cites the paper that does 1 headed attention: https://github.com/lucidrains/memorizing-transformers-pytorch/commit/9f77fd5e4e449d70c02b9cd25a98e1d5ef5f0a72

Thanks! What about this question: Why is there only one Linear layer to_kv, instead of 2 linear layers to_k and to_v?

I see, thanks! An unrelated question, just to confirm my understanding, regarding the following line: https://github.com/lucidrains/memorizing-transformers-pytorch/blob/83fa1479d6f7881dd977fbff55681e709e3b250e/memorizing_transformers_pytorch/knn_memory.py#L153 Can I ask why we have `num_indices` (i.e. `batch_size`) number of `KNN` objects for...

~To provide more context of the `KNNMemory.add` function, here's an example on my understanding:~ https://github.com/lucidrains/memorizing-transformers-pytorch/blob/83fa1479d6f7881dd977fbff55681e709e3b250e/memorizing_transformers_pytorch/knn_memory.py#L201 ~Suppose we have in the batch (size=4) the key vectors corresponding to the sentences:~ ```...

> @manestay I think Figure 3 in the paper answers your question. Each batch appears to steam documents to maintain a consistent within-document memory. And then, a batch of size...

> deepspeed doesn't seem to load the anaconda environment variable correctly in the case of multiple nodes. For example, my ninja path is /home/xxx/anaconda3/envs/NLP/bin/ninja, but deepspeed does not add this...

I have the same issue. I verified that `import torch` works in a python shell from that directoy.