microsoft
getting a parallel run error when submitting a run in MACOS
You can view the traces from local: http://localhost:54796/v1.0/ui/traces/?#run=groundedness_variant_0_20240420_165405_648733 [2024-04-20 16:54:05,790][promptflow._sdk._orchestrator.run_submitter][INFO] - Submitting run groundedness_variant_0_20240420_165405_648733, log path: /Users/joyeed/.promptflow/.runs/groundedness_variant_0_20240420_165405_648733/logs.txt [2024-04-20 16:54:09,345][promptflow._sdk._orchestrator.run_submitter][WARNING] - Run groundedness_variant_0_20240420_165405_648733 failed when executing in executor with exception Unexpected error occurred while executing the batch run. Error: (RuntimeError) An attempt has been made to start a new process before the current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
To fix this issue, refer to the "Safe importing of main module"
section in https://docs.python.org/3/library/multiprocessing.html
..
Traceback (most recent call last): File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/batch/_batch_engine.py", line 254, in run return async_run_allowing_running_loop( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/_utils/async_utils.py", line 40, in async_run_allowing_running_loop return asyncio.run(async_func(*args, **kwargs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/nest_asyncio.py", line 30, in run return loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/nest_asyncio.py", line 98, in run_until_complete return f.result() ^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/futures.py", line 203, in result raise self._exception.with_traceback(self._exception_tb) File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/tasks.py", line 277, in __step result = coro.send(None) ^^^^^^^^^^^^^^^ File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/batch/_batch_engine.py", line 406, in _exec_in_task return task.result() ^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/futures.py", line 203, in result raise self._exception.with_traceback(self._exception_tb) File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/tasks.py", line 277, in __step result = coro.send(None) ^^^^^^^^^^^^^^^ File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/batch/_batch_engine.py", line 465, in _exec results, is_timeout = await self._executor_proxy._exec_batch( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/_proxy/_python_executor_proxy.py", line 110, in _exec_batch with LineExecutionProcessPool( File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/executor/_line_execution_process_pool.py", line 146, in enter self.start() File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/executor/_line_execution_process_pool.py", line 158, in start manager = Manager() ^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/context.py", line 57, in Manager m.start() File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/managers.py", line 563, in start self._process.start() File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) ^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/context.py", line 288, in _Popen return Popen(process_obj) ^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/popen_spawn_posix.py", line 32, in init super().init(process_obj) File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/popen_spawn_posix.py", line 42, in _launch prep_data = spawn.get_preparation_data(process_obj._name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/spawn.py", line 164, in get_preparation_data _check_not_importing_main() File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/spawn.py", line 140, in _check_not_importing_main raise RuntimeError(''' RuntimeError: An attempt has been made to start a new process before the current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
To fix this issue, refer to the "Safe importing of main module"
section in https://docs.python.org/3/library/multiprocessing.html
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.
To fix this issue, refer to the "Safe importing of main module"
section in https://docs.python.org/3/library/multiprocessing.html
.
[2024-04-20 16:54:10,850][promptflow._sdk._orchestrator.run_submitter][WARNING] - Run groundedness_variant_0_20240420_165401_806540 failed when executing in executor with exception Unexpected error occurred while executing the batch run. Error: (EOFError) .. Traceback (most recent call last): File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/batch/_batch_engine.py", line 254, in run return async_run_allowing_running_loop( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/_utils/async_utils.py", line 40, in async_run_allowing_running_loop return asyncio.run(async_func(*args, **kwargs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/nest_asyncio.py", line 30, in run return loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/nest_asyncio.py", line 98, in run_until_complete return f.result() ^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/futures.py", line 203, in result raise self._exception.with_traceback(self._exception_tb) File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/tasks.py", line 277, in __step result = coro.send(None) ^^^^^^^^^^^^^^^ File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/batch/_batch_engine.py", line 406, in _exec_in_task return task.result() ^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/futures.py", line 203, in result raise self._exception.with_traceback(self._exception_tb) File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/tasks.py", line 277, in __step result = coro.send(None) ^^^^^^^^^^^^^^^ File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/batch/_batch_engine.py", line 465, in _exec results, is_timeout = await self._executor_proxy._exec_batch( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/_proxy/_python_executor_proxy.py", line 110, in _exec_batch with LineExecutionProcessPool( File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/executor/_line_execution_process_pool.py", line 146, in enter self.start() File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/executor/_line_execution_process_pool.py", line 158, in start manager = Manager() ^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/context.py", line 57, in Manager m.start() File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/managers.py", line 567, in start self._address = reader.recv() ^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/connection.py", line 250, in recv buf = self._recv_bytes() ^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/connection.py", line 430, in _recv_bytes buf = self._recv(4) ^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/connection.py", line 399, in _recv raise EOFError EOFError
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/joyeed/promptflow/promptflow/groundedness/flowrun.py", line 16, in
Hi @rajib76 , does your code run outside of main scope? Coud you try wrap your code under main scope like this?
if __name__ == '__main__':
pf.run()
tried as below, still getting the error
from promptflow.client import PFClient
from promptflow.entities import Run
# Get a pf client to manage runs
pf = PFClient()
# # Initialize an Run object
run = Run(
flow="/Users/joyeed/promptflow/promptflow/groundedness/flow.dag.yaml",
# run flow against local data or existing run, only one of data & run can be specified.
data="/Users/joyeed/promptflow/promptflow/groundedness/data.jsonl",
column_mapping={"source": "${data.source}","statement": "${data.statement}"},
)
# inputs = [{"source": "TajMahal is a monument is Agra. it is situated in the bank of Yamuna","statement":"TajMahal is in the bank of Yanuma"},
# {"source": "Eiffel Tower is a iron tower and it is in Paris, France","statement":"Eiffel tower is made of iron"}]
#
# for input in inputs:
# source = input["source"]
# statement = input["statement"]
# result = pf.test(flow="/Users/joyeed/promptflow/promptflow/groundedness/flow.dag.yaml",inputs={"source":source,"statement":statement})
# print(result)
# Create the run
if __name__=="__main__":
result = pf.runs.create_or_update(run,stream=False)
print(result)
Starting prompt flow service... Start prompt flow service on port 54796, version: 1.9.0. You can stop the prompt flow service with the following command:'pf service stop'. Alternatively, if no requests are made within 1 hours, it will automatically stop. You can view the traces from local: http://localhost:54796/v1.0/ui/traces/?#run=groundedness_variant_0_20240421_204444_782556 [2024-04-21 20:44:49,583][promptflow._sdk._orchestrator.run_submitter][INFO] - Submitting run groundedness_variant_0_20240421_204444_782556, log path: /Users/joyeed/.promptflow/.runs/groundedness_variant_0_20240421_204444_782556/logs.txt [2024-04-21 20:44:57,261][promptflow._sdk._orchestrator.run_submitter][WARNING] - Run groundedness_variant_0_20240421_204444_782556 failed when executing in executor with exception Unexpected error occurred while executing the batch run. Error: (AssertionError) group argument must be None for now.. Traceback (most recent call last): File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/batch/_batch_engine.py", line 254, in run return async_run_allowing_running_loop( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/_utils/async_utils.py", line 40, in async_run_allowing_running_loop return asyncio.run(async_func(*args, **kwargs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/nest_asyncio.py", line 30, in run return loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/nest_asyncio.py", line 98, in run_until_complete return f.result() ^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/futures.py", line 203, in result raise self._exception.with_traceback(self._exception_tb) File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/tasks.py", line 277, in __step result = coro.send(None) ^^^^^^^^^^^^^^^ File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/batch/_batch_engine.py", line 406, in _exec_in_task return task.result() ^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/futures.py", line 203, in result raise self._exception.with_traceback(self._exception_tb) File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/tasks.py", line 277, in __step result = coro.send(None) ^^^^^^^^^^^^^^^ File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/batch/_batch_engine.py", line 465, in _exec results, is_timeout = await self._executor_proxy._exec_batch( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/_proxy/_python_executor_proxy.py", line 110, in _exec_batch with LineExecutionProcessPool( File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/executor/_line_execution_process_pool.py", line 146, in enter self.start() File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/promptflow/executor/_line_execution_process_pool.py", line 208, in start self._monitor_pool = ThreadPool( ^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/pool.py", line 930, in init Pool.init(self, processes, initializer, initargs) File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/pool.py", line 215, in init self._repopulate_pool() File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/pool.py", line 306, in _repopulate_pool return self._repopulate_pool_static(self._ctx, self.Process, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/pool.py", line 322, in _repopulate_pool_static w = Process(ctx, target=worker, ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/pool.py", line 927, in Process return Process(*args, **kwds) ^^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/multiprocessing/dummy/init.py", line 37, in init threading.Thread.init(self, group, target, name, args, kwargs) File "/Users/joyeed/promptflow/promptflow/venv/lib/python3.11/site-packages/trulens_eval/utils/threading.py", line 48, in init fThread.init( File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/threading.py", line 879, in init assert group is None, "group argument must be None for now" ^^^^^^^^^^^^^ AssertionError: group argument must be None for now
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/joyeed/promptflow/promptflow/groundedness/flowrun.py", line 26, in
Process finished with exit code 1
Hi @rajib76 , seems you installed nest-asyncio in your local environment and it changed python original threading's behavior. Could you uninstall it and try again? If not works, could you provide a repro of the bug?
pip uninstall nest-asyncio
Had the same problem on Mac OS using interactive Python in vscode. Added the if if __name__ == '__main__' and executed as a python script and it worked fine. I guess one can use the vscode extension to run things interactively instead.
Had the same problem on Mac OS using interactive Python in vscode. Added the if
if __name__ == '__main__'and executed as a python script and it worked fine. I guess one can use the vscode extension to run things interactively instead.
Yes, that's because we used multi-processing to execute flow and it's required to use if __name__ == '__main__': idiom to protect the code from executing unexpectedly.
Hi, we're sending this friendly reminder because we haven't heard back from you in 30 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 7 days of this comment, the issue will be automatically closed. Thank you!
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}