MiniImagenet Integrity Check Failure

[venuv]

Hello, This seems like an error in my torchmeta setup, but wanted to get your perspective if otherwise. thanks in advance - VV

(dsai20) dg8965@mb-qs-pp-v100:~/GBML$ python3 main.py --alg=Reptile using gpu: 0 Traceback (most recent call last): File "main.py", line 225, in main(args) File "main.py", line 115, in main target_transform=Categorical(num_classes=args.num_way) File "/home/dg8965/.conda/envs/dsai20/lib/python3.6/site-packages/torchmeta/datasets/miniimagenet.py", line 89, in init download=download) File "/home/dg8965/.conda/envs/dsai20/lib/python3.6/site-packages/torchmeta/datasets/miniimagenet.py", line 127, in init raise RuntimeError('MiniImagenet integrity check failed') RuntimeError: MiniImagenet integrity check failed

[Franklin-Yao]

This is very simple to solve. put all your files in data_path/miniimagenet/. If you put a breakpoint there, you can find this solution by yourself.

[renesax14]

If you put a breakpoint there, you can find this solution by yourself.

Put a breakpoint where? And then what do I check?

[kgarg8]

https://github.com/sungyubkim/GBML/blob/master/main.py#L115 just add ,download=True

[brando90]

https://github.com/sungyubkim/GBML/blob/master/main.py#L115 just add ,download=True

I do have download true...

import torch

import torchvision.transforms as transforms

from torchmeta.datasets.helpers import miniimagenet
from torchmeta.utils.data import BatchMetaDataLoader

from tqdm import tqdm

from pathlib import Path

meta_split = 'train'
data_path = Path('~/data/').expanduser()
dataset = miniimagenet(data_path, ways=5, shots=5, test_shots=15, meta_split=meta_split, download=True)
dataloader = BatchMetaDataLoader(dataset, batch_size=16, num_workers=4)
print(f'len normal = {len(dataloader)}')

num_batches = 10
with tqdm(dataloader, total=num_batches) as pbar:
    for batch_idx, batch in enumerate(pbar):
        train_inputs, train_targets = batch["train"]
        print(train_inputs.size())
        # print(batch_idx)
        if batch_idx >= num_batches:
            break

print('success\a')

[brando90]

hopefully this will solve it: https://stackoverflow.com/questions/64451179/how-does-one-resolve-the-miniimagenet-data-integrity-issue-with-torchmeta

[brando90]

I am going to try to send my local copies after deleting all the HPC copies.