4vicii

hi , im really preciated by the work. but i have a question about "pre_logits". Did you set the "**representation_size**" as defualt **None** when you finetune **imagenet21k_ViT-B_16.npz** with **imagenet 1k**?...

I have 8 GPUs, --gpu_id '0,1,2,3,4,5,6,7',but every time stucked in the model forward **outputs = model(images)** and i try to use a single GPU,it works. my env torch 2.0 cuda...

Hi，I have 8 GPU(GTX 4090).when i set the --gpu_id as '0,1,2,3,4,5,6,7' , the process will be stucked in **outputs = model(images)** Unless ,I set one GPU how can i fix...

我调用DataCollatorForWholeWordMask，发现返回的结果里只有input_ids和label，缺少了attention_mask和 token_type_ids这两个信息，torch_call方法返回的也只有input_ids和label两个键值对，我发现您的示例代码里，在trainer里传递了一个tokenizer，这是否说明在训练前您还使用了一次tokenizer切词？