OSError: [Errno 22] Invalid argument: '/path/to/file.json'

[AdamDimech]

Describe the bug PlantCV v.4.0.1 is unable to write results to a large JSON file.

To Reproduce Steps to reproduce the behavior (including sample data and code really help):

1. Submit 1156 images for analysis
2. Let the analysis run.
3. When the process is ready to complete (after 12 hours), get the error OSError: [Errno 22] Invalid argument: '/path/to/file.json'

Expected behavior JSON file is written to with results from analysis.

Local environment (please complete the following information):

• OS: Ubuntu 22.04 (Windows Subsystem for Linux 2)
• Environment: venv with pip
• PlantCV Version: 4.0.1

Additional context Running this process on a Windows 10 machine. This post suggests that this error may be due to the size of the JSON file.

Any ideas?

[nfahlgren]

I would be surprised if the JSON file was over a size limit, we have generated multi-gigabyte JSON files from the analysis of 100K+ images.

To me it looks like there is an invalid path somewhere like the configuration file where it's reporting /path/to/file.json

[AdamDimech]

Well, I ran the same code to the same file path with a subset of images and it worked. Then when I proceed to analyse the entire set, it fails. It is very odd, because it seems to be generating all of the output images associated with the pipeline and then failing on the last part.

[nfahlgren]

It sounds like it can potentially be OS-specific and I have only generated the larger JSON files on Linux, so maybe on Windows it's hitting a limit.

In your configuration file do you have cleanup set to false? If so you have all the temporary files that are meant to get merged into the final output JSON file and we could try to work with that to figure out a workaround.

[AdamDimech]

Hi @nfahlgren I had cleanup set to true, however I have found the cause of the problem (although not the reason for it):

This particular analysis was from a one-off specialised non-LemnaTec image data set. Rather than run the analysis on our Linux server, I ran it locally with the images stored on a portable hard-drive. PlantCV 4.0.1 in Linux on WSL2 was able to access the images and write all of the output images again to the portable hard drive, but the JSON file would never write. In the end, I set the config file to write the JSON file to the local hard drive in Linux and it worked.

So I am unsure what was going on there, but I now have a workaround. This information may help others running local versions of PlantCV 4.