Ali Hassani
Ali Hassani
Hi, thank you for your interest. First off, great catch! This tiny difference was a result of a bad merge in a very old version of our repository. We were...
Hello, Thank you, yes it would not result in any significant difference, at least that is what we observed in our experiments. I'll check this PR and make separate models...
Hello and thank you for your interest. Yes, in theory, dot product self attention can be replaced with NA/DiNA. There's a few things I'd point out: * NA2D expects inputs...
Can you confirm the path `/dataset/Imagenet` exists?
I'm pretty sure that's the problem, it's literally failing at checking if the dataset path exists.
It is literally failing here: ``` assert os.path.exists(root) AssertionError ``` Also, could you clarify what the "full path of the dataset" is? Can you please `ls /dataset/ImageNet` and share the...
In that case it should be `dataset/ImageNet` and not `/dataset/ImageNet` (no forward slash in the beginning.)
Thank you for your interest. There is a basic 3D NA in NATTEN right now, you'd just need to build it from source. However, it's a very limited implementation, so...
Apologies in advance if I misunderstood your question; I think you're suggesting that we implement neighborhood attention by just modifying the attention mask logic in FMHA/FAV2 and use the existing...
>I'm still somewhat confused. Is it not possible to implement 2D Natten with a 1D FMHA + attention mask? It is possible; I just very seriously doubt it'll have any...