Baptiste Roziere
Baptiste Roziere
Hi. I'm experiencing what I think is a related issue: With ``` println!("{}", var1 + var2); ``` the "," and "+" are not in the ast.
Hi, Is this line the command you run in a terminal? ``` %run codegen_sources/preprocessing/preprocess data/test_dataset 20 python obfuscation 8 500 200 400 roberta False data/bpe/cpp-java-python/vocab False data/bpe/cpp-java-python --1 ``` I...
Hi. You can use our released DOBF models to do that (those models are using the roberta tokenizer and architecture). You can start with the DOBF + DAE version for...
Hi, Are the logs (.out and .err) files empty? Are some `.tok` files generated or does it fail directly? You may want to try using a local parallelism (`--local_parallelism 4`)...
Hi, Yes it should have been something that works on any cluster. Sorry about that. This should fix it: https://github.com/facebookresearch/CodeGen/commit/14a2f9983b26b56dca4c3821b007387cbe1fd93e
Hi, Your code contains `enc_path, map_location=lambda storage, loc: storage.cuda(params.local_rank))` on line 181 in __init__.py while the current version is different and this line is not in the file. https://github.com/facebookresearch/CodeGen/blob/9720a8bdba18552cc499975e1f1fb6a7eca74612/codegen_sources/model/src/model/__init__.py#L181 Could...
Hi 1. You just need to pass your .code and .vocab files to `preprocess.py` using the fastbpe_vocab_path and fastbpe_code_path parameters. It will reload your BPE instead of computing a new...
Hi @dineshkh, 1. You still need to keep the same bt_steps parameters no matter what your test/valid set is. If you don't have unit tests, you can set --eval_computation to...
I'll detail the steps you'd need to take to translate from COBOL to python: - Get monolingual COBOL and python data using google bigquery or something else - preprocess your...
Yes and you can probably adapt the C++ processor for C https://github.com/facebookresearch/CodeGen/blob/c83433217fdba964d1f15aa4d45a78c75d6bfa12/codegen_sources/preprocessing/lang_processors/cpp_processor.py