AttributeError: 'ConcatDatasetBatchSampler' object has no attribute 'batch_size'

[wawjwyy123]

[cai525]

It is helpful for me to change the torch lighting mode to "dp", according to this issue: issues38. But the problem is that, setting torch-lighting to "dp" mode causes two problems when working with multi-gpu:

• As issues38 has mentioned, the size of "filename" and "audio" are not matched. My solution is to add the hashing value of "filename" in the return list of dataset:

  def __getitem__(self, item):
     if self.return_filename:
        out_args.append(c_ex["mixture"])
        out_args.append(torch.tensor(hash(c_ex["mixture"]), dtype=torch.int64, requires_grad=False))
     return out_args
  

  and the program gets the right filename set according to the hashing value:

  def validation_step(self, batch, batch_indx):
       audio, atst_feats, labels, padded_indxs, filenames, hash_values = batch
       hash_dict = {np.int64(hash(s)):s for s in filenames}
       filenames = [hash_dict[np.int64(i.cpu())] for i in hash_values]
       ...
  

• Another problem is that the torchmetrics.classification.f_beta.MultilabelF1Score doesn't work well in multi-GPU situation(refer to this issue). I don't no how to fix the bug(I'm not familiar with torch-lighting). My solution is to comment out the code associated with using torch-metric. Maybe someone knows how to fix this bug?

[popcornell]

The baseline code right now does not work well with multi-GPU and IMO it is not needed as the training is already very fast. It is better to do hyper-parameter optimization across GPUs. I ve added an optuna script for that.