Understanding of results

[ahtsham58]

I have trained the inspired system successfully and got the results for both recommendation and conversational tasks. I would appreciate it if you could please clarify the following points.

The results

1. For the recommendation task, inspired results are mentioned under the BERT row. is that correct?
2. The conversational results are mentioned under the transformer row. is that correct?

Finally, the original inspired system implemented two models, one with strategy and one without a strategy. So the results I received in the conversational task belong to which model?

Looking forward to your urgent clarification. Thanks in advance!

[ahtsham58]

could you please spare few moments

[ToheartZhang]

Thanks for your feedback! Due to the special social strategy annotation of the original Inspired dataset, we currently implement only the model without the strategy module considering the compatibility of the dataset. Therefore, we did not report replication results for this model in README.

[ahtsham58]

Thanks for your response.

Could you please help me to get the final responses generated with the inspired model along with item recommendations? In other words, for qualitative analysis, we need complete responses given the user utterance.

You may point out the right place of code and maybe some tips so that I can modify the code in order to get the required results.

[ToheartZhang]

Sorry for the late reply, both the conversation model and the recommendation model provide generated results.

https://github.com/RUCAIBox/CRSLab/blob/09ecdadb11729b0811a15cf469bc30038ed53bf1/crslab/system/inspired.py#L103

https://github.com/RUCAIBox/CRSLab/blob/09ecdadb11729b0811a15cf469bc30038ed53bf1/crslab/system/inspired.py#L127

However, we do not provide an interface to store the generated results for now. In order to record the generated results, you can modify step function to return results in the validation.