Shibo Hao

icon indicating a website link https://ber666.github.io/

icon company UC San Diego icon location San Diego icon location Ph.D. student at UC San Diego; B.S. in Computer Science at Peking University

Results 4 issues of Shibo Hao

Why doesn't the model collapse?

7 comment

Hi, thanks for the great work. From the paper, the only objective function is the distance between predicted and encoded representations of target patches. Why does the model not converge...

Image generation evaluation

Hi, thanks for this great work! Do you plan to release the evaluation script of image generation (ms-coco)?

Speed on A100

4 comment

Hi, thanks for the cool project. I am testing [Llama-2-70B-GPTQ](https://huggingface.co/TheBloke/Llama-2-70B-GPTQ) with 1 * A100 40G, the speed is around 9 t/s Is this the expected speed? I noticed in some...

The length of samples

1 comment

It seems each sample in the deita dataset consists of a lot of turns and is super long (>10k tokens). Your paper mentioned the max length of input is 2048...