YoutubeMatch OOM Error

[apdullahyayik]

While training YoutubeMatch on a a dataset having almost 29M users, after the first epoch, in the _set_latent_vectors method OOM (out of memory) error is thrown on GPU that has 32GB memory .

Note that 28872629 x 80 refers that there are 5 features each of which is represented by 1 x 16 embedding vector.

Lower embedding size or building most of features as dense may solve, but I am trying to reach a solution out of these.

Any suggestion is appreciated.

[massquantity]

I couldn't think of a clean way to do this. The _set_latent_vectors function is just used to speed up recommendation. In other words, it uses memory to exchange for speed. So maybe you could split all of users into batches, and compute scores separately. But no doubt this will slow down the evaluation.

[apdullahyayik]

By the way, I wonder that in this library there is a limitation of users that have few interactions or not. Like discarding users that have only 1 or 2 interactions with item/s. As far as I see, at retrieval task training the model to be able to project users at N dim space with insufficient item interactions may lead to under-fitting. What do you think about this case?

[apdullahyayik]

Another question: 1- In libreco library ,when evaluating a model of ranking task , do you make recommendation considering all available item profiles, at train, test, and eval sets, or just considering the ones at test set for a user query? This choice will directly effect the performance metrics.

[massquantity]

Yeah in real world scenarios it is not uncommon to filter users with few interactions. But the library doesn't support that ​and I think it's easy to filter them using Pandas.

The recommendation will only consider items appeared in train set. If an item only appears in eval or test set, it will NOT be recommended at all. So if your data has a lot of items that don't appear in train set, it would be difficult to get a good eval result.