Oana Camburu

Why is output vocab including positional embeddings?

2

Hi, I was wondering why is the output softmax of dimension n_vocab + n_special + n_ctx as opposed to just n_vocab + n_special? We don't really need to output "tokens"...

Hi, I noticed that if I just increase n_ctx (it is 77 in https://github.com/huggingface/pytorch-openai-transformer-lm/blob/master/train.py#L214 and I tried different values larger than that) I get different results. For example, with n_ctx=200...

Hi, I understand that there is a non-determinism due to GPU ops and I observed this as well when running twice the same code on the same GPU gave significant...

Hi, I was wondering if, during the pre-training of the LM alone, the sentences were prepended with a _start_ symbol, just like they are during fine-tuning. If that's the case,...

pip install adversarialnlp doesn't work

4

Hi @thomwolf , Thanks for this, just wanted to point out that pip installation doesn't find adversarialnlp ``` Could not find a version that satisfies the requirement adversarialnlp (from versions:...

Addressing the Differing Data Distributions in the GAP Test Set

1

As the authors mentioned, the instances in GAP have certain differences between the female and male groups, e.g., the correct candidate is on average further away from the pronoun in...

Compile the feature/20160617_cb_softattention branch of our fork of Caffe

3

Hi, I cannot seem to find the feature directory nor any **softattention** file. Could you please tell us where can we find this? Thanks, Oana

Dataset files

2

Hi, Could you please what are the files v2_mscoco_train2014_annotations.json and the v2_mscoco_val2014_annotations.json meant to be in Annotations/ and the v2_OpenEnded_mscoco_train2014_questions.json and v2_OpenEnded_mscoco_val2014_questions.json files from the Questions/ ? I couldn't find...