FileNotFoundError: with Nature Dataset

[Pi-31415]

I was trying to follow the custom training for PixelLib with Nature dataset from (https://pixellib.readthedocs.io/en/latest/custom_train.html), and the following error shows up on my computer.

There are 600 listed files in folder train.
Converting labelme annotations to COCO format:   0%|          | 0/600 [00:00<?, ?it/s]
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-5-1d174ed15597> in <module>()
      5 
      6 vis_img = instance_custom_training()
----> 7 vis_img.load_dataset("Nature")
      8 vis_img.visualize_sample()

3 frames
/usr/local/lib/python3.7/dist-packages/PIL/Image.py in open(fp, mode, formats)
   2951 
   2952     if filename:
-> 2953         fp = builtins.open(filename, "rb")
   2954         exclusive_fp = True
   2955 

FileNotFoundError: [Errno 2] No such file or directory: '/content/Nature/train/..\\butterfly (176).jpg'

I tried it again on Google Colab provided on the documentation at (https://colab.research.google.com/drive/1LIhBcxF6TUQUQCMEXCRBuF4a7ycUmjuw?usp=sharing), and it still gives the same error.

May I know how to resolve this, thanks.

[ZhanqiZhang66]

same error here

[SIME-LAB]

The same error [Errno 2] No such file or directory: '/content/Nature/train/..\butterfly4.jpg'

[marcpre]

Same error here

I used the following google colab to recreate the error.

https://colab.research.google.com/drive/1mIZaS-kwoYSQL2SvAbwQ-L_YAiAwwD19?usp=sharing

The file definitely exists, BUT is not found:

[secco88]

maybe there is an error inside the json file: "imagePath": "..\butterfly (1).png"

the ..\ part isn't necessary

[p4bloR]

maybe there is an error inside the json file: "imagePath": "..\butterfly (1).png"

the ..\ part isn't necessary

This fixed it for me, can't thank you enough man.

[greg007-web]

The new version didn't work. I had the same problem. Try this setting:

!pip3 uninstall tensorflow
!pip3 install tensorflow==2.4.1
!pip3 install tensorflow--gpu
!pip3 install imgaug
!pip install pixellib==0.5.2
!pip install labelme2coco==0.1.2

```I used an old version and now is working.

[tmmay232]

maybe there is an error inside the json file: "imagePath": "..\butterfly (1).png" the ..\ part isn't necessary

This fixed it for me, can't thank you enough man.

I'm having the same issue. How did you fix the path error inside the json file?

[p4bloR]

maybe there is an error inside the json file: "imagePath": "..\butterfly (1).png" the ..\ part isn't necessary

This fixed it for me, can't thank you enough man.

I'm having the same issue. How did you fix the path error inside the json file?

First let me warn you. Yes, I did fix this issue, but I ended up having a few issues down the line (I think those were unrelated, so you're probably not going to run into them). To avoid those I just ended up using my own dataset.

Ok, so in colab I made a function to iterate through the dataset json files, and delete the necessary substrings in the "imagePath" column.

Here's the code:

from google.colab import drive
drive.mount('/content/drive', force_remount = True)

import os 
import json 

def delete_substrings(path, ext, column, substring, replacement):
  for files in os.listdir(path):
    if files.endswith(ext):
      with open(path + files, 'r+') as f:
        json_data = json.load(f)
        before = json_data[column]
        #print(before)
        if substring in before:
          
          after = before.replace(substring, replacement)
          print("filename: ", files)
          print("before: ", before, " after: ",after)

          json_data['imagePath'] = after
          f.seek(0)
          f.write(json.dumps(json_data))
          f.truncate()

Some json files had a ".." before the filename while others had a "images\" or "images", so I had to run that function a few times targetting each substring I wanted to delete.

Here's an example:

path = '/content/drive/MyDrive/'
ext = ('.json')
substring = "images" + "\\" + "\\"
column  = "imagePath"
replacement = ""

delete_substrings(path, ext, column, substring, replacement)

If the use of "\\" + "\\" confuses you I recommend a google search, it turns out that in python it's basically ilegal to end a string with a single "\", but if you make a string like this "\\" and decide to print it, it prints: "\", so with: "\\" + "\\" I get to make a string that actually contains two "\". Confusing, I know.

Well, that's basically it man, let me know if it works for you.

PS: I also recommend you google what f.seek() and f.truncate() do.

[darlannakamura]

I solved this problem creating a simple function to fix_image_path, that removes ..\ from imagePath attribute of each .json:

from glob import glob
import json

def fix_image_path(directory: str):
  for filename in glob(directory):
    with open(filename, 'r') as f:
      content = json.load(f)
    content['imagePath'] = content['imagePath'].replace('..\\', '')
    
    with open(filename, 'w') as f:
      json.dump(content, f)

fix_image_path("Nature/train/*.json")
fix_image_path("Nature/test/*.json")

Just run this code before that block of code:

vis_img = instance_custom_training()
vis_img.load_dataset("Nature")
vis_img.visualize_sample()

Further, I had to reinstall Pillow in the old version (9.0.0) because of error: module 'pil.image' has no attribute 'resampling'.

If you have this issue, run:

pip install --ignore-installed Pillow==9.0.0

And restart runtime.

[naseemap47]

I was trying to follow the custom training for PixelLib with Nature dataset from (https://pixellib.readthedocs.io/en/latest/custom_train.html), and the following error shows up on my computer.

There are 600 listed files in folder train.
Converting labelme annotations to COCO format:   0%|          | 0/600 [00:00<?, ?it/s]
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-5-1d174ed15597> in <module>()
      5 
      6 vis_img = instance_custom_training()
----> 7 vis_img.load_dataset("Nature")
      8 vis_img.visualize_sample()

3 frames
/usr/local/lib/python3.7/dist-packages/PIL/Image.py in open(fp, mode, formats)
   2951 
   2952     if filename:
-> 2953         fp = builtins.open(filename, "rb")
   2954         exclusive_fp = True
   2955 

FileNotFoundError: [Errno 2] No such file or directory: '/content/Nature/train/..\\butterfly (176).jpg'

I tried it again on Google Colab provided on the documentation at (https://colab.research.google.com/drive/1LIhBcxF6TUQUQCMEXCRBuF4a7ycUmjuw?usp=sharing), and it still gives the same error.

May I know how to resolve this, thanks.

!pip3 install tensorflow==2.6.0 !pip3 install keras==2.6.0 !pip3 install imgaug !pip3 install pillow==8.2.0 !pip install pixellib==0.5.2 !pip install labelme2coco==0.1.2

Its work for me..

[palvors]

I solved this problem creating a simple function to fix_image_path, that removes ..\ from imagePath attribute of each .json:

from glob import glob
import json

def fix_image_path(directory: str):
  for filename in glob(directory):
    with open(filename, 'r') as f:
      content = json.load(f)
    content['imagePath'] = content['imagePath'].replace('..\\', '')
    
    with open(filename, 'w') as f:
      json.dump(content, f)

fix_image_path("Nature/train/*.json")
fix_image_path("Nature/test/*.json")

Just run this code before that block of code:

vis_img = instance_custom_training()
vis_img.load_dataset("Nature")
vis_img.visualize_sample()

Further, I had to reinstall Pillow in the old version (9.0.0) because of error: module 'pil.image' has no attribute 'resampling'.

If you have this issue, run:

pip install --ignore-installed Pillow==9.0.0

And restart runtime.

Great, another issue begenning butterfly (256).json ... the path has "image\butterfly (256).png" So I juste duplicate le line and add a new replace.

content['imagePath'] = content['imagePath'].replace('..\', '') content['imagePath'] = content['imagePath'].replace('images\', '')