dimeldo

How can I ignore <unk> tokens while decoding?

3

Hi, how can I ignore unk tokens while decoding? It probably just means setting the weight to zero. So I want to do something like this: ```Python if ignore_unk: next_probs[:,...

Error when setting num_candidates=1 and in dataset there's only one candidate during validation

7

When setting args.num_candidates to 1, and the actual length of the candidates list of each entry is 1, I get this error during validation: ``` ERROR:ignite.engine.engine.Engine:Current run is terminating due...

Repo not properly designed

4

Thank you for your contribution. I noticed a few things about your repo: - Variable names are vague and uninformative, which makes it hard to understand the code and adapt...

What format data files need to be before running prepro.py?

2

My dataset is a .txt file where each line represents an entire dialogue where inside each turn is separated by a tab. How can I convert it to your tour...

This repo https://github.com/lucidrains/reformer-pytorch implements the Reformer model that can handle inputs up to 32K tokens (theoretically even more). The implementation seems to be straightforward to incorporate into this repo.

Distributed training doesn't work.

2

At least using xlnet model. When using high max_len, it doesn't print any error just crashes. Training with 1 GPU works well. When setting low max_len I get the error...

Training on distributed machine is slow. Using 8 Nvidia V100.

8

I'm using aws p3dn.24xlarge to train my data on 8 Nvidia V100 GPU's but the training seem slower than 1 GPU. This is the config in `train-horovod.py`: ``` def train_main(dataset,...

My data, before encoding, looks like this: ``` Person 1: Something something something. Person 2: Something something something. Person 1: Something something something. Person 1: Something something something. Person 2:...