Brian Qu comments

Results 44 comments of


                                            Brian Qu

Image loading in dataloader code

Hello, I also busy with it! I'm curious about how can we get the images and corresponding Q-A pairs. Do you have any experience?

Image loading in dataloader code

Thanks for your reply! So I thought that you first generate the `memmaps`. After that, you use the memmaps to build the dataloader, and save the results(image-QA pairs) into a...

Image loading in dataloader code

Thank you very much!!! I want convert the memmaps into png files because I want to use the images as inputs. I want to ask that what process repo did...

finetune时如何支持单卡组batch训练呢？

Hi, @yuhangzang . Thanks for this great work! In my opinion, the `batch_size` here works more similar to `sequence packing`. I think you should rename it to advoid some mistakes.

About the Visual Prompt

Thanks! I will update it! Do you mean that the visual-prompts are added to every transformer block?

Thanks for your answer! I've got the visual prompt ops. Additionally, if convinient, I would like to ask you for the performance difference between VisionTransformerPromptDeep and VisionTransformerPrompt, i.e. whether adding...

DocOwl1.5 training code?

hello, how about the venv requirements? I've not seen the requirements.txt.

How to transfer the multipage pdfs to images(png)?

Hello，have you ever solved this problem？ I'm also puzzled with this.

分布式设置错误

> os.environ['RANK'] = '0' > os.environ['WORLD_SIZE'] = '4' # 因为我需要只用其中的4张卡 > os.environ['LOCAL_RANK'] = '0' 不要指定三项如果只用四张卡直接运行时加入`CUDA_VISIBLE_DEVICES=x,x,x,x` 如果还是超时，请删除掉两个关于NCCL的环境变量

请教一下关于SFT的问题

你应该是运行的7B版本的sft吧，我在sft load模型的时候，模型会先load到cpu，然后再pipeline parallel到对应的GPU上，你这里我感觉应该是CPU内存不够，存不下半精度的7～8B的模型（7～8B模型半精度，约需要14～16GB的存储空间）