trouble-maker007

Is there any plans to add deepspeed to pretrain large language model?

请问支持相对位置编码的Bert吗?

1

请问支持 relative position embedding 类似 [nezha](https://github.com/huawei-noah/Pretrained-Language-Model/tree/master/NEZHA-TensorFlow), [roformer](https://github.com/ZhuiyiTechnology/roformer) ，如果是roformer，是不是需要将模型的参数名称改为和Bert一样才能使用

IndexError: list index out of range

5

while I select two points and press drag it ``` Traceback (most recent call last): File "/data/miniconda3/envs/env-novelai/lib/python3.10/site-packages/gradio/routes.py", line 414, in run_predict output = await app.get_blocks().process_api( File "/data/miniconda3/envs/env-novelai/lib/python3.10/site-packages/gradio/blocks.py", line 1320, in...

the demo caption is very simple, reproduce the result in paper

10

the demo caption is very simple, not like the detailed one in the paper, did you limit the output max length?

我使用img2dataset下载wudaomm数据集，发现Flower.json中大量类似如下的图片链接无法下载，这是链接失效了吗？ ``` { "name": "5d532c4a9cd24fbf1653ed3486c99244.jpg", "tag": "花卉", "url": "http://img5.iplant.cn/image2/b/1871135.jpg", "captions": "秘鲁天轮柱属" }, { "name": "dc12ee1d49998f2f58ed4b6932a8ce90.jpg", "tag": "花卉", "url": "http://img6.iplant.cn/image2/b/1871166.jpg", "captions": "重瓣榆叶梅" }, { "name": "0e656f0b4a40fd118c113aaef7416a3f.jpg", "tag": "花卉", "url": "http://img7.iplant.cn/image2/b/1871167.jpg", "captions":...

which llama 7b version use?

4

did you use llama 7b in the training with the InternViT−6B? and is there any plan to release a technical report?

Does this support in the webui api use

2

@bbc-mc How to use [sdweb-clip-changer](https://github.com/bbc-mc/sdweb-clip-changer) in webui api