[Question] Ranking returning always same order

[giovana-morais]

trafficstars

hello, there! I was exploring the basic ranking tutorial and it seems that, for a fixed set of items, the ranking values vary very little when comparing different users and even cold-start users. At first, I thought this was happening because the retrieval step would be responsible for selecting the "right" items to be ranked, but then I manually selected items that a specific user gave a high score, and yet the model would give the same ranking for both this user and a cold start user.

test_movie_titles = [
    "M*A*S*H (1970)", "Dances with Wolves (1990)", "Speed (1994)", 
    "Swingers (1996)", "Jurassic Park (1993)", "In the Line of Fire (1993)"
]

test_ratings = {}
for movie_title in test_movie_titles:
    test_ratings[movie_title] = model({
      "user_id": np.array(["276"]),
      "movie_title": np.array([movie_title])
    })

print("Ratings:")
for title, score in sorted(test_ratings.items(), key=lambda x: x[1], reverse=True):
    print(f"{title}: {score}")
    

regular user ("276")

Ratings:
M*A*S*H (1970): [[3.9895713]]
Swingers (1996): [[3.8475957]]
In the Line of Fire (1993): [[3.7985928]]
Dances with Wolves (1990): [[3.773054]]
Jurassic Park (1993): [[3.7534351]]
Speed (1994): [[3.677456]]

cold start user ("")

Ratings:
M*A*S*H (1970): [[3.70247]]
Swingers (1996): [[3.5549703]]
In the Line of Fire (1993): [[3.508114]]
Dances with Wolves (1990): [[3.480867]]
Jurassic Park (1993): [[3.4616296]]
Speed (1994): [[3.387645]]

I trained the model using some suggestions given on https://github.com/tensorflow/recommenders/issues/591 to make sure the model was not undertrained, and also added more movie titles to the test, but the result is consistent.

Is this an expected behavior or did I miss something in the tutorial? I imagined the ranking would have a substantial difference between users.

Thanks in advance!

[mSounak]

Hi @giovana-morais, You closed the issue. Did you find what replicated this behavior?

[giovana-morais]

@mSounak not really. I tried different things but I got no success. I closed the issue because I started doing something else, but I can reopen it.

[mSounak]

@mSounak not really. I tried different things but I got no success. I closed the issue because I started doing something else, but I can reopen it.

Yes, please.