Dataloader error in test_bscdfsl.py

[W-XuWX]

Hi there,

I encountered an error when running test_bscdfsl.py on --cdfsl_domains CropDisease ChestX.

While scanning through to find the best learning rate for the fine-tuning dataset, the loop works for the first learning rate, then fails during the second learning rate.

Error message:

# first learning rate loop
dict_items([('n_ways', <utils.deit_util.SmoothedValue object at 0x7f9dad7bf160>), ('n_imgs', <utils.deit_util.SmoothedValue object at 0x7f9dad7bf100>), ('acc1', <utils.deit_util.SmoothedValue object at 0x7f9dad7bf250>), ('acc5', <utils.deit_util.SmoothedValue object at 0x7f9dad7bf2b0>), ('loss', <utils.deit_util.SmoothedValue object at 0x7f9dc84bad90>)])


* Acc@1 93.467 Acc@5 100.000 loss 0.634
{'n_ways': 5.0, 'n_imgs': 100.0, 'acc1': 93.46667022705078, 'acc5': 100.0, 'loss': 0.6343524515628814, 'acc_std': 5.312924861907959}
*lr = 0: acc1 = 93.46667022705078
lr: 0.0001
type of dataloader: <class 'datasets.get_bscd_loader.<locals>._Loader'>
10
Test: Total time: 0:00:00 (0.0000 s / it)

# second learning rate loop - loss is missing
dict_items([('n_ways', <utils.deit_util.SmoothedValue object at 0x7f9d936a0f40>), ('n_imgs', <utils.deit_util.SmoothedValue object at 0x7f9d936a0fa0>), ('acc1', <utils.deit_util.SmoothedValue object at 0x7f9d936a0f10>), ('acc5', <utils.deit_util.SmoothedValue object at 0x7f9d936a0ee0>)])


Traceback (most recent call last):
  File "test_bscdfsl.py", line 125, in <module>
    main(args)
  File "test_bscdfsl.py", line 68, in main
    test_stats = evaluate(data_loader_val, model, criterion, device, seed=1234, ep=5)
  File "/home/aorus01/projects/pmf_cvpr22/engine.py", line 118, in evaluate
    return _evaluate(data_loaders, model, criterion, device, seed)
  File "/home/aorus01/miniconda3/envs/pmf/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/aorus01/projects/pmf_cvpr22/engine.py", line 168, in _evaluate
    .format(top1=metric_logger.acc1, top5=metric_logger.acc5, losses=metric_logger.loss))
  File "/home/aorus01/projects/pmf_cvpr22/utils/deit_util.py", line 102, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'MetricLogger' object has no attribute 'loss'

steps taken to debug: I printed the ret_dict (under engine.py, _evaluate function) and found that the "loss" metric was missing (refer to code block above). Suspected that the dataloader (as a generator) reached the end of the iteration at the first evaluation call and was not 'reset' afterwards. Hence, the second evaluation call passed in a generator dataloader with nothing to iterate.

steps taken to address: under test_bscdfsl.py, I shifted the data_loader_val = get_bscd_loader(domain, args.test_n_way, args.n_shot, args.image_size) line to inside the learning rate scanning loop to reload the dataloader after each loop.

this affected the next evaluate call (under # final classification), likewise, I added another data_loader_val = get_bscd_loader(domain, args.test_n_way, args.n_shot, args.image_size) line to address this