[Question] Swapping `query` and `candidate` towers

[ydennisy]

Hi @maciejkula - its been a while hope you are well :)

I am back with another question!

---

In my application I can easily swap query and candidate towers, because I just need the final vectors for both, my system will not be hosted like a traditional query -> candidate model.

I was wondering what effect should occur when one swaps the two around? Does it affect any of the internal of how loss is calculated etc?

I am finding it hard in my case to just test and see, because in my case I have a very high imbalance where I have millions of items on one side and hundreds on the other. So this means if I run the experiment and just swap the towers, my metrics no longer correlate as the task becomes either much harder, or much simpler.

Would love your view on this! Or any of the other fine people who hang about in this repo!

[maciejkula]

Two tower models are mostly symmetric.

In your case it looks like one task (few queries -> lots of candidates) is just much harder than going the other way.