pytorch-nested-unet icon indicating copy to clipboard operation
pytorch-nested-unet copied to clipboard

hellow I prepare my own data according your suggestion , but it's fault.

Open jackft2 opened this issue 4 years ago • 8 comments

Traceback (most recent call last): File "train.py", line 367, in main() File "train.py", line 327, in main train_log = train(config, train_loader, model, criterion, optimizer) File "train.py", line 112, in train for input, target, _ in train_loader: File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 345, in next data = self._next_data() File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 856, in _next_data return self._process_data(data) File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 881, in _process_data data.reraise() File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/_utils.py", line 395, in reraise raise self.exc_type(msg) TypeError: Caught TypeError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 178, in _worker_loop data = fetcher.fetch(index) File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/ubuntu/.local/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/ubuntu/Desktop/pytorch-nested-unet-master/dataset.py", line 64, in getitem mask.append(cv2.imread(os.path.join(self.mask_dir, str(i),img_id + self.mask_ext), cv2.IMREAD_GRAYSCALE)[..., None]) TypeError: 'NoneType' object is not subscriptable

jackft2 avatar Jun 11 '20 03:06 jackft2

I met the same problem.How can I solve it?

chenjingyi-star avatar Jun 30 '20 07:06 chenjingyi-star

I have the same error. Anyone can help?

Fotouhif avatar Nov 19 '20 18:11 Fotouhif

I could fix it. You should change the function " getitem" in dataset.py as follows:

""" def getitem(self, idx): img_id = self.img_ids[idx]

    img = cv2.imread(os.path.join(self.img_dir, img_id + self.img_ext))

    mask = np.array([])
    for i in range(self.num_classes):
    	if cv2.imread(os.path.join(self.mask_dir, str(i),img_id + self.mask_ext)) is not None:
        	mask = cv2.imread(os.path.join(self.mask_dir, str(i),img_id + self.mask_ext), cv2.IMREAD_GRAYSCALE)[..., None]
        	mask = np.dstack(mask)

    if self.transform is not None:
        augmented = self.transform(image=img, mask=mask)
        img = augmented['image']
        mask = augmented['mask']
    
    img = img.astype('float32') / 255
    img = img.transpose(2, 0, 1)
    mask = mask.astype('float32') / 255
    mask = mask.transpose(2, 0, 1)
    
    return img, mask, {'img_id': img_id}

"""

Fotouhif avatar Nov 20 '20 16:11 Fotouhif

The actual problem is in your dataset mask label, you shouldn't just add the line: "if cv2.imread(os.path.join(self.mask_dir, str(i),img_id + self.mask_ext)) is not None:" The source codes obtain the image name, and according to it search all the mask folders (0,1,2,... all your class folders), and of course not all your image contains e.g., class 0 label, that causes the problem “'NoneType' object is not subscriptable”. Adding the code to check if the class label exists will solve this problem, but will create another issue, mask tensor shape not equal. For example, your input mask size is 96x96, you will have different shape like [2,96,96], [4,96,96],[3,96,96]... The first dimension is the number of classes in that particular image. ie, the code will iter all the mask class folders, add the existing label mask into the mask array, then use "mask = np.dstack(mask)" to transfer to a tensor. Therefore, the right solution to this problem is to complete your mask dataset rather than mending any codes.

x55901 avatar Sep 14 '21 08:09 x55901

The actual problem is in your dataset mask label, you shouldn't just add the line: "if cv2.imread(os.path.join(self.mask_dir, str(i),img_id + self.mask_ext)) is not None:" The source codes obtain the image name, and according to it search all the mask folders (0,1,2,... all your class folders), and of course not all your image contains e.g., class 0 label, that causes the problem “'NoneType' object is not subscriptable”. Adding the code to check if the class label exists will solve this problem, but will create another issue, mask tensor shape not equal. For example, your input mask size is 96x96, you will have different shape like [2,96,96], [4,96,96],[3,96,96]... The first dimension is the number of classes in that particular image. ie, the code will iter all the mask class folders, add the existing label mask into the mask array, then use "mask = np.dstack(mask)" to transfer to a tensor. Therefore, the right solution to this problem is to complete your mask dataset rather than mending any codes.

I had the same trouble. My data set is made using Labelme software and has only one class. How do you make a dataset that uses this source code correctly?

JlexZhong avatar Sep 16 '21 07:09 JlexZhong

If you only have 1 class then you should not have this problem, this issue only occur when you have 2 or more classes. What’s the error message?

发件人: @.> 发送时间: 2021年9月16日 15:59 收件人: @.> 抄送: Simon @.>; @.> 主题: Re: [4uiiurz1/pytorch-nested-unet] hellow I prepare my own data according your suggestion , but it's fault. (#27)

The actual problem is in your dataset mask label, you shouldn't just add the line: "if cv2.imread(os.path.join(self.mask_dir, str(i),img_id + self.mask_ext)) is not None:" The source codes obtain the image name, and according to it search all the mask folders (0,1,2,... all your class folders), and of course not all your image contains e.g., class 0 label, that causes the problem “'NoneType' object is not subscriptable”. Adding the code to check if the class label exists will solve this problem, but will create another issue, mask tensor shape not equal. For example, your input mask size is 96x96, you will have different shape like [2,96,96], [4,96,96],[3,96,96]... The first dimension is the number of classes in that particular image. ie, the code will iter all the mask class folders, add the existing label mask into the mask array, then use "mask = np.dstack(mask)" to transfer to a tensor. Therefore, the right solution to this problem is to complete your mask dataset rather than mending any codes.

I had the same trouble. My data set is made using Labelme software and has only one class. How do you make a dataset that uses this source code correctly?

― You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/4uiiurz1/pytorch-nested-unet/issues/27#issuecomment-920677210, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AVFUHN4EIRRMRG56FBW3LPLUCGPVHANCNFSM4N3AZOBQ. Triage notifications on the go with GitHub Mobile for iOShttps://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Androidhttps://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

x55901 avatar Sep 16 '21 09:09 x55901

I had the same problem, and I solved it. I confirm this is a Pytorch problem, please use Pytorch 1.6.

The actual problem is in your dataset mask label, you shouldn't just add the line: "if cv2.imread(os.path.join(self.mask_dir, str(i),img_id + self.mask_ext)) is not None:" The source codes obtain the image name, and according to it search all the mask folders (0,1,2,... all your class folders), and of course not all your image contains e.g., class 0 label, that causes the problem “'NoneType' object is not subscriptable”. Adding the code to check if the class label exists will solve this problem, but will create another issue, mask tensor shape not equal. For example, your input mask size is 96x96, you will have different shape like [2,96,96], [4,96,96],[3,96,96]... The first dimension is the number of classes in that particular image. ie, the code will iter all the mask class folders, add the existing label mask into the mask array, then use "mask = np.dstack(mask)" to transfer to a tensor. Therefore, the right solution to this problem is to complete your mask dataset rather than mending any codes.

I had the same trouble. My data set is made using Labelme software and has only one class. How do you make a dataset that uses this source code correctly?

I had the same problem, and I solved it. I confirm this is a pyTorch version issue, please use Pytorch 1.6.

# CUDA 10.2
pip install torch==1.6.0 torchvision==0.7.0

I had a lot of problems configuring this project.This is my environment:

  • numpy==1.17.0
  • labelme==3.16.2
  • ipdb==0.13.9
  • pandas==1.1.5
  • tqdm==4.42.1
  • matplotlib==3.1.2
  • torch==1.6.0
  • torchvision==0.7.0
  • albumentations==1.0.3
  • opencv_python_headless==4.5.3.56
  • Pillow==8.3.1
  • scikit_learn==0.24.2
  • opencv-python==4.1.2.30
  • opencv-contrib-python==4.1.2.30

Albumentations has now been updated to version 1.0.3, and you need to change the way some packages are imported:

from albumentations.augmentations.geometric.rotate import RandomRotate90
from albumentations.augmentations.geometric.resize import Resize 
from albumentations.augmentations import transforms
from albumentations.core.composition import Compose, OneOf

JlexZhong avatar Sep 20 '21 14:09 JlexZhong

实际问题在于您的数据集掩码标签中,您不应该只是添加一行:“如果cv2.imread(os.path.join(self.mask_dir,str(i),img_id + self.mask_ext))不是无:”源代码获取图像名称,并根据它搜索所有掩码文件夹(0,1,2,...所有你的类文件夹),当然不是所有的图像都包含例如类0标签,这会导致“'NoneType'对象不可下标”的问题。添加代码来检查类标签是否存在将解决此问题,但会产生另一个问题,掩码张量形状不相等。例如,您的输入掩码大小为 96x96,您将具有不同的形状,如 [2,96,96]、[4,96,96]、[3,96,96]...第一个维度是该特定图像中的类数。即,代码将迭代所有掩码类文件夹,将现有的标签掩码添加到掩码数组中,然后使用“mask = np.dstack(mask)”传输到张量。因此,此问题的正确解决方案是完成掩码数据集,而不是修补任何代码。

请问我应该怎么完善我的掩码数据集?我有两个类,运行报错: mask = np.dstack(mask) ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 0, the array at index 0 has size 740 and the array at index 1 has size 800 我的图片和掩码数据大小均为740*1471的,我反复验证了我的图片,不应该出现此问题,但它确实出现了。我应该怎么做?

yangu49 avatar May 07 '22 02:05 yangu49