skye95git
skye95git
> For question 1: Yes! You can refer to [this part](https://github.com/Jun-jie-Huang/CoCLR#vanilla-model-1) of README to reproduce the results. And the model without CoCLR means training with original data to some extend....
> As for question 1, we checked the code and found there's a bug, which lead to the mis-use of the model. Now we fix it, you can use the...
For the question3, I find the description in the paper `We initialize CoCLR with microsoft/codebert-base repretrained on CodeSearchNet Python Corpus`. How do you repretrained Codebert? I just find the method...
Thanks for your reply! After your explanation, my understanding is that there are three models in your method: **The first one**: your model CoCLR trained on CodeSearchNet. I have a...
> Are you sure you pasted the exact command you ran? I have no error when trying it on my side and the config is successfully updated. To use distributed...
> Are you sure you pasted the exact command you ran? I have no error when trying it on my side and the config is successfully updated. To use distributed...
"Using deprecated `--per_gpu_train_batch_size` argument which will be removed in a future version. Using `--per_device_train_batch_size` is preferred." `per_device_train_batch_size` specifies the batch size to be processed by each GPU, right?
@sgugger I used the 'LineByLineTextDataset' script as above to continue pre-train Roberta on multiple cards in a single machine. It seemed to be an unbalanced load.  Is the single-machine...
> 由于uer的modelzoo里面没有提供英文bert的下载地址，所以去 huggingface 下载了一个再转换格式。 转换格式期间遇到错误。 > > 可能是您用的旧版本的huggingface bert. 我刚下载的 huggingface bert (https://huggingface.co/bert-base-uncased/tree/main), key 已经变了 > > ```python > output_model["embedding.layer_norm.gamma"] = input_model["model.encoder.layernorm_embedding.weight"] > ``` > > 报错：`KeyError: 'model.encoder.layernorm_embedding.weight'` 我看到 input...
> 1.If you want to use the model in UER format, you can download it directly from the link https://github.com/dbiir/UER-py/wiki/Modelzoo 2.Yes, you can customize the input by modifying the Dataset...