Help with creating a matrix of features

[sjelassi]

Hi everyone,

Question: I am training a transformer model on IWSLT'14 German to English (translation task). After the training is done, I want to get the matrix of features where each row corresponds to features of a datapoint in IWSLT'14 German to English. Does anyone know how to do it?

What I have tried: I know that I can use model.extract_features() to get the features given a datapoint x after loading the model through checkpoint. But I couldn't figure out how to load the data and how to iterate(with shuffling turned off for reproducibility as well) over the whole data.

Thank you in advance for your help !

[gmryu]

If you want to do it yourself, I suggest you to copy and edit fairseq's generate.py https://github.com/facebookresearch/fairseq/blob/main/fairseq_cli/generate.py#L187-L211

I assume you have used fairseq-generate, then you can just use this generate.py with the same command line arguments. That hypos = task.inference_step( is where a batch is sent to the model. So you can just change it to whatever you want. Following for loops tells you how to handle id, etc. Also there is no shuffling in generation, as you can find .next_epoch_itr(shuffle=False) is written for the batch iterator.