Error with file path when running hello world notebook

[vokcow]

Hi,

I am trying to run the "hello world" notebook example, however, I am getting an error after calling runner.run(). The issue seems to occur after the line 488 in the method _train_algo_in_sequence, in auto_runer.py

After this line, in the generated search.py script in the dints_0 directory, the "config_file" path, rather than a single path, is a list of paths casted as a string, thus raising an error when passing this variable to ConfigParser:

This is the content of config_file: config_file = 'C:\Users\Me\Documents\Experiments\Auto3dSeg\helloworld_work_dir\dints_0\configs\hyper_parameters.yaml','C:\Users\Me\Documents\Experiments\Auto3dSeg\helloworld_work_dir\dints_0\configs\hyper_parameters_search.yaml','C:\Users\Me\Documents\Experiments\Auto3dSeg\helloworld_work_dir\dints_0\configs\network.yaml','C:\Users\Me\Documents\Experiments\Auto3dSeg\helloworld_work_dir\dints_0\configs\network_search.yaml','C:\Users\Me\Documents\Experiments\Auto3dSeg\helloworld_work_dir\dints_0\configs\transforms_infer.yaml','C:\Users\Me\Documents\Experiments\Auto3dSeg\helloworld_work_dir\dints_0\configs\transforms_train.yaml','C:\Users\Me\Documents\Experiments\Auto3dSeg\helloworld_work_dir\dints_0\configs\transforms_validate.yaml'

This is the stacktrace from the error after passing config_file to ConfigParser:

Traceback (most recent call last): File "C:\Users\Me\Anaconda3\envs\h22\lib\site-packages\monai\apps\auto3dseg\bundle_gen.py", line 183, in _run_cmd normal_out = subprocess.run(cmd.split(), env=ps_environ, check=True, capture_output=True) File "C:\Users\Me\Anaconda3\envs\h22\lib\subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['python', 'C:\Users\Me\Documents\Experiments\Auto3dSeg\helloworld_work_dir\dints_0\scripts\search.py', 'run', "--config_file='C:\Users\Me\Documents\Experiments\Auto3dSeg\helloworld_work_dir\dints_0\configs\hyper_parameters.yaml','C:\Users\Me\Documents\Experiments\Auto3dSeg\helloworld_work_dir\dints_0\configs\hyper_parameters_search.yaml','C:\Users\Me\Documents\Experiments\Auto3dSeg\helloworld_work_dir\dints_0\configs\network.yaml','C:\Users\Me\Documents\Experiments\Auto3dSeg\helloworld_work_dir\dints_0\configs\network_search.yaml','C:\Users\Me\Documents\Experiments\Auto3dSeg\helloworld_work_dir\dints_0\configs\transforms_infer.yaml','C:\Users\Me\Documents\Experiments\Auto3dSeg\helloworld_work_dir\dints_0\configs\transforms_train.yaml','C:\Users\Me\Documents\Experiments\Auto3dSeg\helloworld_work_dir\dints_0\configs\transforms_validate.yaml'", '--searching#num_iterations=8', '--searching#num_iterations_per_validation=4', '--searching#num_images_per_batch=2', '--searching#num_epochs=2', '--searching#num_warmup_iterations=4']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "C:\Users\Me\Anaconda3\envs\h22\lib\site-packages\monai\apps\auto3dseg\bundle_gen.py", line 188, in _run_cmd raise RuntimeError(f"subprocess call error {e.returncode}: {errors}, {output}") from e RuntimeError: subprocess call error 1: b'Traceback (most recent call last):\r File "C:\Users\Me\Anaconda3\envs\h22\lib\site-packages\fire\core.py", line 466, in _Fire\r component, remaining_args = _CallAndUpdateTrace(\r File "C:\Users\Me\Anaconda3\envs\h22\lib\site-packages\fire\core.py", line 681, in CallAndUpdateTrace\r component = fn(*varargs, **kwargs)\r File "C:\Users\Me\Experiments\Auto3dSeg\code
sclc\helloworld_work_dir\dints_0\scripts\search.py", line 46, in run\r parser.read_config(config_file)\r File "C:\Users\Me\Anaconda3\envs\h22\lib\site-packages\monai\bundle\config_parser.py", line 300, in read_config\r content.update(self.load_config_files(f, **kwargs))\r File "C:\Users\Me\Anaconda3\envs\h22\lib\site-packages\monai\bundle\config_parser.py", line 403, in load_config_files\r for k, v in (cls.load_config_file(i, **kwargs)).items():\r File "C:\Users\Me\Anaconda3\envs\h22\lib\site-packages\monai\bundle\config_parser.py", line 379, in load_config_file\r raise ValueError(f'unknown file input: "{filepath}"')\r ValueError: unknown file input: "'C:\Users\Me\Experiments\Auto3dSeg\code
sclc\helloworld_work_dir\dints_0\configs\hyper_parameters.yaml','C:\Users\Me\Experiments\Auto3dSeg\code
sclc\helloworld_work_dir\dints_0\configs\hyper_parameters_search.yaml','C:\Users\Me\Experiments\Auto3dSeg\code
sclc\helloworld_work_dir\dints_0\configs
etwork.yaml','C:\Users\Me\Experiments\Auto3dSeg\code
sclc\helloworld_work_dir\dints_0\configs
etwork_search.yaml','C:\Users\Me\Experiments\Auto3dSeg\code
sclc\helloworld_work_dir\dints_0\configs\ ransforms_infer.yaml','C:\Users\Me\Experiments\Auto3dSeg\code
sclc\helloworld_work_dir\dints_0\configs\ ransforms_train.yaml','C:\Users\Me\Experiments\Auto3dSeg\code
sclc\helloworld_work_dir\dints_0\configs\ ransforms_validate.yaml'"\r ', b'\x1b[0m'

These are my system specifications: Edition Windows 10 Enterprise Version 21H2 Experience Windows Feature Experience Pack

I don't know if relevant but when installing monai I needed to install a missing dependency: the fire package. I installed it manually trhough conda in my virtual enviroment. Maybe that has anything to do with this?

Many thanks in advance, Best wishes,

Victor

[mingxin-zheng]

Hi Victor @VokCow , I think this is a similar problem to: https://github.com/Project-MONAI/MONAI/pull/5280

The problem is that Auto3DSeg for Windows is still under testing, and this should be resolved very soon.

[mingxin-zheng]

Hi @VokCow , the problem should've fixed with the latest weekly build. If you're using Pip to install MONAI, Can you try

pip install monai-weekly --upgrade

to update the monai package.

[vokcow]

Hi @mingxin-zheng, thanks for your reply. The version I tested on Windows was the milestone release. When trying to test it on the weekly release I get the following error:

OMP: Error #15: Initializing libiomp5md.dll, but found libiomp5md.dll already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://www.intel.com/software/products/support/.

Anyway, I ended up simply running my code on a Linux machine, which brought me to another problem: how to overwrite the loss function in the generated network templates so I can use my custom loss?

Many thanks,

Victor

[wyli]

it seems this script will address the OMP issue

import os
os.environ['KMP_DUPLICATE_LIB_OK']='True'