NUS-HPC-AI-Lab
Distilled datasets
Hello, thank you for your excellent work. However, I have a small question. I downloaded and tested the distilled datasets you provided, and most of the data is fine. However, the accuracy of CIFAR100 with IPC=10 is only 37%. Could you please confirm whether the dataset was misplaced or if there is another issue?
Can you provide more details? Did you evaluate using our script and hyper-parameters? I think the uploaded version is correct.
python evaluation.py --lr_dir /home/datm/CIFAR-100/IPC10/lr_best.pt --data_dir /home/datm/CIFAR-100/IPC10/images_best.pt --label_dir /home/mtt/datm/CIFAR-100/IPC10/labels_best.pt --zca --data_path /home/mtt/data --dataset CIFAR100
This is my cmd with evaluation.py, and the distilled dataset is downloaded from this github, below is the result:
CUDNN STATUS: True
Files already downloaded and verified
Files already downloaded and verified
Train ZCA
Test ZCA
<class 'kornia.enhance.zca.ZCAWhitening'>
Hyper-parameters:
{'dataset': 'CIFAR100', 'subset': 'imagenette', 'model': 'ConvNet', 'eval_mode': 'S', 'epoch_eval_train': 1000, 'batch_real': 256, 'dsa': True, 'dsa_strategy': 'color_crop_cutout_flip_scale_rotate', 'data_path': '/home/data', 'zca': True, 'lr_teacher': 0.01, 'no_aug': False, 'batch_train': 128, 'parall_eva': False, 'data_dir': '/home/datm/CIFAR-100/IPC10/images_best.pt', 'label_dir': '/home/datm/CIFAR-100/IPC10/labels_best.pt', 'lr_dir': '/home/datm/CIFAR-100/IPC10/lr_best.pt', 'device': 'cuda', 'zca_trans': ZCAWhitening(), 'im_size': (32, 32), 'dc_aug_param': None, 'dsa_param': <utils.utils_baseline.ParamDiffAug object at 0x7f4ff7834220>, 'distributed': True}
Evaluation model pool: ['ConvNet']
Evaluating: ConvNet
[2025-03-11 11:35:58] Evaluate_00: epoch = 1000 train time = 63 s train loss = 0.690364 train acc = 0.0080, test acc = 0.3879
This is wired, could you try to evaluate using https://huggingface.co/spaces/logits/DD-Ranking ? I think the evaluation they performed is also based on the version uploaded in this repo.