Moses Hu

run wrongly with CPU

2

I excute the code textgen = textgenrnn() textgen.reset() file="reddit_rarepuppers_politics_2000_context.csv" textgen.train_from_file(file, context=True,new_model=True, num_epochs=10, gen_epochs=5, word_level=True) if i use GPU to run this code, it's OK. but with CPU, it gave me...

how to custom config file for Bert?

1

in your notebook : https://colab.research.google.com/github/gmihaila/ml_things/blob/master/notebooks/pytorch/pretrain_transformers_pytorch.ipynb#scrollTo=UCLtm5BiXona get_model_config() method load config from model_path, how to change it to custom config?, for example config = BertConfig( vocab_size=10000, hidden_size=256, num_hidden_layers=6, num_attention_heads=4, intermediate_size=3072, hidden_act="gelu", hidden_dropout_prob=0.1,...

I use ctc to train a ASR model, but the result value is ,the below is the result of keras.backend.ctc_decode(pred, input_length=input_len, greedy=True). what happend to my program?the log probability if...

I dont unstand what this method "create_look_ahead_mask" using for? if there is no this fucntion,what will happend?

how to train English-Hokkien Speech-to-speech Translation?

1

Is the S2ST training process public?

关于合并中文token跟原始LLama 的token

1

### 我使用 LLamaTokenzier 的add_token 方法来扩张词汇表，跟你代码中使用 sentenpiece来训练有什么不同么？ 这个对训练embedding会有影响么？

关于合并llama-alpaca 的问题

3

### Chinese-Alpaca-Plus-7B 和 Chinese-LLaMA-Plus-7B在训练中，target_modules 这个参数都是一样的吧？我看LoraConfig的配置都差不多，都把embedding做了训练 如果我是基于Chinese-Alpaca-Plus-7B 的 adapater.bin做prompt训练，我只需要在代码中设置 model.resize_token_embeddings(tokenzier_vocab_size)( 这里的model 是将facebook权重跟 Chinese-LLaMA-Plus-7B做了合并的) lora config也需要LoraConfig.LoraConfig.from_pretrained("adpater_config_dir")( /adpater_config_dir/adapter_config.json) 比如: config = LoraConfig.LoraConfig.from_pretrained("adpater_config_dir") model = LlamaForCausalLM.from_pretrained("llama_weight") tokenizer = LlamaTokenizer.from_pretrained("alpaca_plus_7B") model.resize_token_embeddings(len(tokenizer)) model = get_peft_model(model, config)...

``` def tokenize(prompt, add_eos_token=True): # there's probably a way to do this with the tokenizer settings # but again, gotta move fast result = tokenizer( prompt, truncation=True, max_length=cutoff_len, padding=False, return_tensors=None,...

Hi! I try to use the fituning.py to train my dataset with alpaca.But when I inference the result is UNK? so what's wrong with my traing?is it overfitting?