Asking for the source code of Transformer/Multi. BERT/XLM

[Jiahuan-Pei]

Hi Zihan,

Thanks for contributing your source code of the paper https://arxiv.org/pdf/1911.09273.pdf.

In this source code, I have not find the Transformer/Multi. BERT/XLM models, as they are the state-of-the-art models as reported in the paper.

Could you please share the models or let us know how to reproduce the reported model.

Regards, Jiahuan

[Jiahuan-Pei]

Could you please also share the settings for reproducing the reported results?

This is what we get using your settings in the README file.

Goal ACC: 0.4840. Joint ACC: 0.1306. Request ACC: 0.8092. Avg ACC: 0.4699

[zliucr]

Hi,

In our experiments, using multilingual word embeddings can achieve comparable or sometimes better results than using Multi. BERT/XLM. If you want to use Multi.BERT/XLM, you can simply replace the word embeddings with the Multi. BERT/XLM embeddings.

As for reproducing the results, we have provided the scripts in the README (in "How to run" section).

Thank you!

[Jiahuan-Pei]

Thanks for your kind reply!

As for reproducing the results, we have provided the scripts in the README (in "How to run" section). ==> Yes, we use the the scripts in the README (in "How to run" section), DST Task, Zero-shot Adaptation for German. And the reproduced results are: Goal ACC: 0.4840. Joint ACC: 0.1306. Request ACC: 0.8092. Avg ACC: 0.4699 The results reported in the paper are:

As for model, I do not think it's only the problem of embeddings. (1) In the Figure 3 in the paper (https://arxiv.org/pdf/1911.09273.pdf), it shows you used Transformer encoder. So I assume it's not only need to change Multi. BERT/XLM embeddings. (2) In the Table 1, for German sub table, in the paper, it shows you used Model namely, MUSE XLM (MLM)∗ + Transformer XLM (MLM+TLM)∗+ Transformer Multi. BERT∗+ Transformer

Could you please provide more details about the Model and settings for reproducing the models?

[zliucr]

Hi,

The script you run should be able to reproduce the results of MUSE model. The hyper-parameter settings are in config.py file. Can you check whether you use the correct embeddings we have provided in the data folder? Thanks.

As for the code for Multi. BERT/XLM + Transformer, we are sorry that we didn't provide the code, since it is a bit messy in our codebase. If you need, we can try to wrap the corresponding code and upload it in next following days.

Thank you!

[Jiahuan-Pei]

Can you check whether you use the correct embeddings we have provided in the data folder? Thanks. ==> Yes, to make sure the reproducibility, we have NOT changed anything and only use all the codes/data in this repository.

[Jiahuan-Pei]

If you need, we can try to wrap the corresponding code and upload it in next following days. ==>Yes, of course :) It would be much more than appreciated! Thank you!

[bumbumya]

@zliucr Thanks for your great work. Can you share the code for Multi. BERT/XLM + Transformer?