Bringing-Old-Photos-Back-to-Life icon indicating copy to clipboard operation
Bringing-Old-Photos-Back-to-Life copied to clipboard

Empty tensor error after some epochs

Open hayanakamura opened this issue 3 years ago • 5 comments

I got this type of error message when implemented train_domain_A.py

File "train_domain_A.py", line 70, in losses, generated = model(Variable(data['label']), Variable(data['inst']), File "/home/anaconda3/lib/python3.8/site-packages/torch/nn/modules/module.py", line 722, in _call_impl result = self.forward(*input, **kwargs) File "/home/Desktop/study/Bringing-Old-Photos-Back-to-Life-master/Global/models/pix2pixHD_model_DA.py", line 196, in forward real_old_feat=torch.cat(real_old_feat,0) RuntimeError: There were no tensor arguments to this function (e.g., you passed an empty list of Tensors), but no fallback function is registered for schema aten::_cat. This usually means that this function requires a non-empty list of Tensors. Available functions are [CPU, CUDA, QuantizedCPU, Autograd, Profiler, Tracer, Autocast]

My question is why this type of error arised after some epochs. And my hypothesis is that this occurred accidentally because of data['inst'] are all 0 or 1 (it may happen because I set batchSize to low number)

Please tell me your insights, thank you.

hayanakamura avatar Apr 07 '21 09:04 hayanakamura

I`m also having the problem, seems random

MengXinChengXuYuan avatar Apr 07 '21 14:04 MengXinChengXuYuan

It's a data loader problem. When training domainA, there are 2 or 3 types of input: real_old_rgb, real_old_l, clean. In one batch, the number of old and clean are random. If one of them is zero, this error will happen because torch.cat can't handle empty tensor. I think there are 2 ways to solve this issue: 1: Use a big batch size to lower the rate of this issue 2: Rewrite data loader(class UnPairOldPhotos_SR) to make sure there will always exists 2 types of input I prefer 2.

syfbme avatar Apr 08 '21 00:04 syfbme

It's a data loader problem. When training domainA, there are 2 or 3 types of input: real_old_rgb, real_old_l, clean. In one batch, the number of old and clean are random. If one of them is zero, this error will happen because torch.cat can't handle empty tensor. I think there are 2 ways to solve this issue: 1: Use a big batch size to lower the rate of this issue 2: Rewrite data loader(class UnPairOldPhotos_SR) to make sure there will always exists 2 types of input I prefer 2.

could you show me how to rewrite the loader?

wykn86 avatar Apr 12 '21 16:04 wykn86

It's a data loader problem. When training domainA, there are 2 or 3 types of input: real_old_rgb, real_old_l, clean. In one batch, the number of old and clean are random. If one of them is zero, this error will happen because torch.cat can't handle empty tensor. I think there are 2 ways to solve this issue: 1: Use a big batch size to lower the rate of this issue 2: Rewrite data loader(class UnPairOldPhotos_SR) to make sure there will always exists 2 types of input I prefer 2.

could you show me how to rewrite the loader?

You can add code in getitem function like below to get different types of photo one by one: if self.switch == 0: A = self.get_one_type(index) self.switch = 1 else: A = self.get_other_type(index) self.switch = 0

syfbme avatar May 26 '21 08:05 syfbme

It's a data loader problem. When training domainA, there are 2 or 3 types of input: real_old_rgb, real_old_l, clean. In one batch, the number of old and clean are random. If one of them is zero, this error will happen because torch.cat can't handle empty tensor. I think there are 2 ways to solve this issue: 1: Use a big batch size to lower the rate of this issue 2: Rewrite data loader(class UnPairOldPhotos_SR) to make sure there will always exists 2 types of input I prefer 2.

could you show me how to rewrite the loader?

You can add code in getitem function like below to get different types of photo one by one: if self.switch == 0: A = self.get_one_type(index) self.switch = 1 else: A = self.get_other_type(index) self.switch = 0

I added it at online_dataset_for_old_photos.py line 215:

    if self.switch == 0:
AttributeError: 'UnPairOldPhotos_SR' object has no attribute 'switch'

Dominic-ZZ avatar May 26 '21 09:05 Dominic-ZZ