[QST] Understanding Model Predictions and visualize_response Function

[SaadAhmed433]

❓ Questions & Help

Details

Hi all, I am trying to understand the model output and how its converted into top_k items using the function provided here

def visualize_response(batch, response, top_k, session_col="session_id"):
    """
    Util function to extract top-k encoded item-ids from logits
    Parameters
    ----------
    batch : cudf.DataFrame
        the batch of raw data sent to triton server.
    response: tritonclient.grpc.InferResult
        the response returned by grpc client.
    top_k: int
        the `top_k` top items to retrieve from predictions.
    """
    sessions = batch[session_col].drop_duplicates().values
    predictions = response.as_numpy("output")
    top_preds = np.argpartition(predictions, -top_k, axis=1)[:, -top_k:]
    for session, next_items in zip(sessions, top_preds):
        print(
            "- Top-%s predictions for session `%s`: %s\n"
            % (top_k, session, " || ".join([str(e) for e in next_items]))
        ) 

So my questions are

1. When we do model.eval() and the model(batch)["predictions"] we get a tensor with logit values, now can you please tell me if the size of it corresponds to this (batch_szize, total_items)

2. np.argpartition gives us the sorted value at the kth index. The elements to the left are smaller than the value of the element at the kth index and the are greater to the right of the kth index. So when we do top_preds = np.argpartition(predictions, -top_k, axis=1)[:, -top_k:] and select the last top_k elements of the array returned after np.argpartition than I am not sure if all the elements are correctly ranked. Please let me know if my understanding is correct ?

Can I applynp.sortto sort the logit values and then select their indexes to get the topk items?

[liguo88]

Based on my understanding to answer questions, hope it helps.

1. For the first question, it is yes. You can print out model(batch)['predictions] to check out.

2. For second question, np.argpartition does not show exactly the order and you can compare it with the model(batch)['predictions][0].topk(5).indices. They will include same indices but not in the same order. And you can use np.sort functions, this is what I am using np.flip(predictions.argsort(axis=-1)[:, -top_k:], axis=-1)

[hosseinkalbasi]

@liguo88 how are we supposed to fetch the final result? When you get the prediction index, do you do something like interactions_df['item_id'].unique()[<index_from_predicion>] to find the recommended item?

On another note, something weird that I've found (possibly my understanding of it is not complete), in the Yoochoose example (here), there are 52739 total unique item ids in the dataset we use for training. However, when I get the prediction from the model (inference server), there are a total of 52743 predictions (logit values). Is this normal?

[liguo88]

@hosseinkalbasi Hi, I did not understand your question clearly. Based on my understanding, to get your recommendation item from predicted index, you need to use the mapping table which is saved in "workflow_etl/categories/unique.item.parquet", notice that the mapping index is shifted by 1. You can double check this using your example.

For your second question, I did not use Yoochoose example, but the prediction numbers should be the same with your number of unique item ids.

[hosseinkalbasi]

@liguo88 that's great! understood it now! About the point that the mapping index is shifted by 1, here is the one for my example:

As you can see index=0 (encoded-item-id=0) has no item_id assigned to it. Are you saying I should just ignore index=0, or shall I instead shift indices so that 0-5739 becomes 0-5378 (after I delete the first row)?

On the second point, I see, thanks. I'll have to check this with another example. Looks like this is because of the wrong value in the schema_demo.pb here. Is this schema file something I could generate via NVTabular?

[SaadAhmed433]

If your predicted item_id from the model is 1 than you should search the original item_id like this category_item_id.iloc[1,0] this should give you the original item id.

You dont need to delete anything, the item_id at index 0 is for items not seen in the dataset which can be the case in real life where you might get some items that are new and so to map them they have the <NA> id. This issue here might give you more information about parquet ids

You can create the schema file when you store your workflow using NVtabular.

[hosseinkalbasi]

Thanks @SaadAhmed433, this was very helpful.

[rnyak]

closing this issue since there is no further question.