MagicSource

@LinB203 Shouldn't be 1.8 MoE? Does the model open? Does the pretrained (no finetune) model able to do simple image caption ?

@LinB203 1. When will the model release? Looks like better than currect model, is it also 1.8bx4 moe? 2. Which pretrained data you were using? sharegpt4v_instruct_gpt4-vision_cap100k.json or pt part? Does...

@LinB203 I think the pretrain loss hard to be 0.2, the official pretrain loss of llava is about 1.9:  How did u guys manageed trained pretrain loss so small?

@LinB203 I found the sharegpt4v also lack of Chinese part data. Do u think any hight quality Chinese pretrain image-text pair can be used to enhance Chinese ability?

@LinB203 Do u think raw ocr image-text paris can be used in pretrain data?

openllama版本啥时候发布呢

@peakji Ithink its not related about model. For model am simple using Llama. The reason is that when we decode same id, compare with decode ids in a sentence, tokenizers...

Or maybe these is something missed inside your StreamTokenizer? (like ignored some ids). Can u try get decode ids one by one and print it? ``` outputs = [] for...

@peakji Thanks, I just using tokenizer of StreamModel and the Chinese decoding error problems still exist.  And I still can not get the spaces between engliesh words . I...

I got no space and Chinese were wrong either (try print(word, end='')) I don't want change line in every word and I don't want unexpect spaces in un-English characters.