henryj18

get_tokenizer in utils.py needs to be cleaned up

6

def get_tokenizer(tokenizer_name): if "t5" in tokenizer_name: # rankgen tokenizer = T5Tokenizer.from_pretrained(tokenizer_name, truncation_side="left") else: tokenizer = AutoTokenizer.from_pretrained(tokenizer_name) if "galactica" in tokenizer_name: tokenizer.add_special_tokens({"pad_token": "", "eos_token": ""}) return tokenizer The line of code...

reward/instructor/trainer.py needs to handle default values of config vars in training_conf (loaded from .yml file)

4

I have tried to run the training "python trainer.py configs/electra-base-dis-webgpt.yml" and ran into an issue of "Keyerror: scheduler" at line 140 lr_scheduler_type=training_conf["scheduler"] Further inspection revealed that it is caused by...

Experiment with https://github.com/lucidrains/vit-pytorch and replace the current ImageEmbedding with such ViT to see whether it can improve the NEKO performance

VQA loss is not decreasing during training, we have tried several ways to reduce it, this is one of the tests. So far we have been following GATO paper to...

Refer to https://github.com/ManifoldRG/NEKO/pull/30 about the PR to add caption and vqa task. Refer to https://github.com/ManifoldRG/NEKO/pull/77 about the PR to merge the code for caption task and vqa task (all of...

Add image-caption task into NEKO project

1

After adding text modality to the NEKO project on top of the continuous control, discrete control tasks etc., we are adding this task of image-caption, it is self-explanatory - train...

gato_policy. py, the following lines pad_len = self.context_len - seq_len if pad_len > 0: assumes seq_len is always smaller than context_len, but it could be the other way also. Need...

Issues found in code review - series #1

1

This is one of the series of issues found during the code review under the branch "add_text_modality" (some issues are inherited from the master branch), we will keep track of...

In an effort to decrease the training loss of VQA task, we are experimenting another ViT, timm ViT, this is a pre-trained ViT. So far the test shows that VQA...