error when running NAS:minionn_cifar10.py

[bingbangcc]

Unable to ray.init() on windows. This is the printout C:\Users\16335\anaconda3\envs\delphi\python.exe C:\Users\16335\delphi\python\minionn\minionn_cifar10.py 2023-06-29 00:18:13,094 INFO resource_spec.py:212 -- Starting Ray with 4.05 GiB memory available for workers and up to 2.03 GiB for objects. You can adjust these settings with ray.init(memory=, object_store_memory=). 2023-06-29 00:18:13,402 INFO services.py:1165 -- View the Ray dashboard at localhost:8265 E0629 00:18:20.501169 23688 6524 raylet_client.cc:69] Retrying to connect to socket for pathname tcp://127.0.0.1:31180 (num_attempts = 1, num_retries = 10) E0629 00:18:23.538391 23688 6524 raylet_client.cc:69] Retrying to connect to socket for pathname tcp://127.0.0.1:31180 (num_attempts = 2, num_retries = 10) E0629 00:18:26.571873 23688 6524 raylet_client.cc:69] Retrying to connect to socket for pathname tcp://127.0.0.1:31180 (num_attempts = 3, num_retries = 10) E0629 00:18:29.599208 23688 6524 raylet_client.cc:69] Retrying to connect to socket for pathname tcp://127.0.0.1:31180 (num_attempts = 4, num_retries = 10) E0629 00:18:32.631668 23688 6524 raylet_client.cc:69] Retrying to connect to socket for pathname tcp://127.0.0.1:31180 (num_attempts = 5, num_retries = 10) E0629 00:18:35.681198 23688 6524 raylet_client.cc:69] Retrying to connect to socket for pathname tcp://127.0.0.1:31180 (num_attempts = 6, num_retries = 10) E0629 00:18:38.727272 23688 6524 raylet_client.cc:69] Retrying to connect to socket for pathname tcp://127.0.0.1:31180 (num_attempts = 7, num_retries = 10) E0629 00:18:41.749498 23688 6524 raylet_client.cc:69] Retrying to connect to socket for pathname tcp://127.0.0.1:31180 (num_attempts = 8, num_retries = 10) E0629 00:18:44.785377 23688 6524 raylet_client.cc:69] Retrying to connect to socket for pathname tcp://127.0.0.1:31180 (num_attempts = 9, num_retries = 10) F0629 00:18:45.786785 23688 6524 raylet_client.cc:78] Could not connect to socket tcp://127.0.0.1:31180 *** Check failure stack trace: *** @ 00007FFFC2D63A8C public: __cdecl google::LogMessage::~LogMessage(void) __ptr64 @ 00007FFFC2BD8954 public: virtual __cdecl google::NullStreamFatal::~NullStreamFatal(void) __ptr64 @ 00007FFFC2C1351B public: void __cdecl google::NullStreamFatal::vbase destructor'(void) __ptr64 @ 00007FFFC2C15B5E public: void __cdecl google::NullStreamFatal::vbase destructor'(void) __ptr64 @ 00007FFFC2B23B98 public: class google::LogMessageVoidify & __ptr64 __cdecl google::LogMessageVoidify::operator=(class google::LogMessageVoidify const & __ptr64) __ptr64 @ 00007FFFC2B21C00 public: class google::LogMessageVoidify & __ptr64 __cdecl google::LogMessageVoidify::operator=(class google::LogMessageVoidify const & __ptr64) __ptr64 @ 00007FFFC2B200ED public: class google::LogMessageVoidify & __ptr64 __cdecl google::LogMessageVoidify::operator=(class google::LogMessageVoidify const & __ptr64) __ptr64 @ 00007FFFC2B1F9C3 public: class google::LogMessageVoidify & __ptr64 __cdecl google::LogMessageVoidify::operator=(class google::LogMessageVoidify const & __ptr64) __ptr64 @ 00007FFFC2A9F179 public: virtual __cdecl google::LogSink::~LogSink(void) __ptr64 @ 00007FFFF48EE4E4 _PyType_Name @ 00007FFFF4896702 _PyObject_FastCallKeywords @ 00007FFFF4964646 PyEval_GetFuncDesc @ 00007FFFF4961438 _PyEval_EvalFrameDefault @ 00007FFFF4963285 _PyEval_EvalCodeWithName @ 00007FFFF4896D8B _PyFunction_FastCallKeywords @ 00007FFFF496463F PyEval_GetFuncDesc @ 00007FFFF496148C _PyEval_EvalFrameDefault @ 00007FFFF4963285 _PyEval_EvalCodeWithName @ 00007FFFF4896D8B _PyFunction_FastCallKeywords @ 00007FFFF496463F PyEval_GetFuncDesc @ 00007FFFF496148C _PyEval_EvalFrameDefault @ 00007FFFF4963285 _PyEval_EvalCodeWithName @ 00007FFFF49BDF2D PyRun_FileExFlags @ 00007FFFF49BDBA7 PyRun_FileExFlags @ 00007FFFF49BBAFE PyRun_SimpleFileExFlags @ 00007FFFF49BA763 PyRun_AnyFileExFlags @ 00007FFFF4818B00 _PyCoreConfig_SetGlobalConfig @ 00007FFFF481D08A _PyMainInterpreterConfig_Read @ 00007FFFF481DCD2 _PyMainInterpreterConfig_Read @ 00007FFFF481DED7 Py_Main @ 00007FF6365814F8 OPENSSL_Applink @ 00007FF8388055A0 BaseThreadInitThunk

and the port of tcp://127.0.0.1:31180 is a random number. the version are as follows: python3.7 tensorflow1.15.0 ray[tune]0.8.6

-1073740791 (0xC0000409)

[ryanleh]

Hi, this is an issue with Ray, not the delphi so it might be more productive to raise an issue there. Is there any log entry that mentions trying to start a service at port 31180 and failing?