Kwong Chun Ho
Kwong Chun Ho
Do I need to transform the tar format of dataset to csv format? I download the dataset using img2dataset, and its result is tar files containing jpg, json and text...
yes, it is the version of 2.15.0
what is testing part for? Checking whether the environments are eligible or otherwise？
Sample single-process running code: --train-data="/path/to/train_data.csv" \ --val-data="/path/to/validation_data.csv" \ _________________________________________________________________________________ I am confused about the dataset format, it recommends img2dataset to download and organize the data which is tar files, but...
cd open_clip/src torchrun --nproc_per_node 4 -m training.main \ --train-data '/（my own datapath）-{00000..00031}.tar' \ --train-num-samples 10968539 \ --dataset-type webdataset \ --batch-size 320 \ --precision amp \ --workers 4 \ I ran...
I downloaded this dataset MSCOCO using img2dataset, forming a webdataset, I just checked the tar files. All is fine except the last one 00031.tar, so I change the code of...
I figured out that maybe some corrupted, and others fine. It seems that the script could pass the corrupted tar files and continue to use the fine tar files. So...
call for a resolution too, I want to train on 7 GPUs, how to set parallel training?
同样需求，翻来翻去好像没有这个功能，只能把整个数据集的文本都攻击后导出，然后再按需使用。