Stephen Roller

wow this heroic change set

To fix your immediate problem, add —skip-generation false. If you want to add some custom metrics, we should talk more and you should describe what kind of metric you want...

I need to better document this but the gist is: If your metric is good for being computed at the teacher (string) level, then adding a `custom_evaluation` function to your...

We have considered a proposal like `@register_metric` similar to what you've suggested, but it hasn't been executed.

Honestly, no one on our team has looked into it. If you spent a little time digging and shared feedback or comments, we would welcome them.

Thanks for reporting. I suspect the issue is that nltk isn’t installed, but the system should work around that. I’ll put out a fix shortly

What version of python were you on?

Still needs to be fixed.

TorchAgent's History keeps track of all the turns: https://github.com/facebookresearch/ParlAI/blob/72c304fa7cac16ed19d8bc75a017f17c8073dd2f/parlai/core/torch_agent.py#L280-L301 So you can use self.history.history_vecs to get the tokenized prior utterances, and self.history.history_strings to get the prior utterances as strings. You...

No, that seems like a clever way to do it. I've never trained an HRED model, so I don't know how important the cache invalidation is at training time. A...