Andy Lynd

In fact, I used 3 GPUs, each of which is 11g. The strange thing is that sometimes it works normally, and sometimes it is reported that the storage is insufficient.

Yes. I passed --max-sentences 2, and it ran normally, but I was worried that it would affect performance. I don't know if it will have a significant impact? Besides, why...

I see, thank you.

 In fact, we found that when SCST was running, one of the GPU memory loads suddenly became too high. There is a serious load imbalance between GPUs. Do you...

Don't worry, I found that during the running process, the memory usage gradually increased. This is the running state at --max-sentence 3. 

Almost every time I encounter it, the strange thing is that it reports a memory error after SCST runs one or two.

> _No description provided._ 是指论文的链接，我使用了很多本地链接（相对地址），你可以直接将本项目下载，然后打开Awesome-NAS/4NAS相关论文/README.md 使用typora打开，ctrl+单击直接打开对应的pdf文件

> FYI, thanks. Would like to ask if gcc3M supports breakpoint download? Also, does gcc3M have 436 .tar files? Is it possible to pre-train on gcc3M alone? thanks.

> > Would like to ask if gcc3M supports breakpoint download? > > As far as I know, the `img2dataset` doesn't support incremental download at this moment. Is it normal...

run `python convert_dataset/create_subset.py --input-dir . --output-dir . --subset yfcc100m_subset_data.tsv` report an error: ``` INFO: Pandarallel will run on 20 workers. INFO: Pandarallel will use Memory file system to transfer data...