prefect icon indicating copy to clipboard operation
prefect copied to clipboard

When trying to run served flow (from_source) from Windows share folder it fails

Open kavvkon opened this issue 1 year ago • 3 comments

Bug summary

from prefect import flow
from pathlib import Path

mycode_path = Path(r'\\192.168.1.10\d\mycode\src')

if __name__ == "__main__":
    flow.from_source(
                     source=mycode_path , 
                     entrypoint="app.py:get_data"
                             ).serve(name="get_data_app", cron="0 4 * * *")

The above code is deployed successfully but when I try to run it from the server I get the following error message

12:14:41.064 | INFO    | prefect.flow_runs.runner - Runner 'get_data_app' submitting flow run '0e0cf219-b79e-44ef-892f-4dd8a94ca774'
12:14:41.121 | INFO    | prefect.flow_runs.runner - Opening process...
12:14:41.137 | INFO    | prefect.flow_runs.runner - Completed submission of flow run '0e0cf219-b79e-44ef-892f-4dd8a94ca774'
'\\192.168.1.10\d\mycode\src'
CMD.EXE was started with the above path as the current directory.
UNC paths are not supported.  Defaulting to Windows directory.
12:14:45.163 | INFO    | Flow run 'omicron616-denobula-h' - Downloading flow code from storage at '\\\\192.168.1.10\\d\\mycode\\src'
12:14:45.396 | ERROR   | prefect.engine - Engine execution of flow run '0e0cf219-b79e-44ef-892f-4dd8a94ca774' exited with unexpected exception
Traceback (most recent call last):
  File "C:\Users\user1\Anaconda3\envs\prefect_env\Lib\site-packages\prefect\engine.py", line 37, in <module>
    flow_run, flow = load_flow_and_flow_run(flow_run_id=flow_run_id)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user1\Anaconda3\envs\prefect_env\Lib\site-packages\prefect\flow_engine.py", line 104, in load_flow_and_flow_run
    flow = run_coro_as_sync(
           ^^^^^^^^^^^^^^^^^
  File "C:\Users\user1\Anaconda3\envs\prefect_env\Lib\site-packages\prefect\utilities\asyncutils.py", line 243, in run_coro_as_sync
    return call.result()
           ^^^^^^^^^^^^^
  File "C:\Users\user1\Anaconda3\envs\prefect_env\Lib\site-packages\prefect\_internal\concurrency\calls.py", line 312, in result
    return self.future.result(timeout=timeout)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user1\Anaconda3\envs\prefect_env\Lib\site-packages\prefect\_internal\concurrency\calls.py", line 182, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user1\Anaconda3\envs\prefect_env\Lib\concurrent\futures\_base.py", line 401, in __get_result
    raise self._exception
  File "C:\Users\user1\Anaconda3\envs\prefect_env\Lib\site-packages\prefect\_internal\concurrency\calls.py", line 383, in _run_async
    result = await coro
             ^^^^^^^^^^
  File "C:\Users\user1\Anaconda3\envs\prefect_env\Lib\site-packages\prefect\utilities\asyncutils.py", line 225, in coroutine_wrapper
    return await task
           ^^^^^^^^^^
  File "C:\Users\user1\Anaconda3\envs\prefect_env\Lib\site-packages\prefect\client\utilities.py", line 75, in wrapper    
    return await func(client, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user1\Anaconda3\envs\prefect_env\Lib\site-packages\prefect\flows.py", line 1932, in load_flow_from_flow_run
    await storage_block.get_directory(from_path=from_path, local_path=".")
  File "C:\Users\user1\Anaconda3\envs\prefect_env\Lib\site-packages\prefect\utilities\asyncutils.py", line 389, in ctx_call
    result = await async_fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user1\Anaconda3\envs\prefect_env\Lib\site-packages\prefect\filesystems.py", line 158, in get_directory
    copytree(from_path, local_path, dirs_exist_ok=True, ignore=ignore_func)
  File "C:\Users\user1\Anaconda3\envs\prefect_env\Lib\shutil.py", line 600, in copytree
    return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user1\Anaconda3\envs\prefect_env\Lib\shutil.py", line 554, in _copytree
    raise Error(errors)

Version info

Version:             3.1.1
API version:         0.8.4
Python version:      3.12.7
Git commit:          6b50a2b9
Built:               Fri, Nov 8, 2024 12:38 PM
OS/Arch:             win32/AMD64
Profile:             default
Server type:         server
Pydantic version:    2.9.2

Additional context

No response

kavvkon avatar Nov 11 '24 11:11 kavvkon

hi @kavvkon - thank you for the issue! can you give some context on why your path is set like this?

mycode_path = Path(r'\\192.168.1.10\d\mycode\src')

that will help us reproduce and fix this

zzstoatzz avatar Nov 11 '24 18:11 zzstoatzz

hi @kavvkon - thank you for the issue! can you give some context on why your path is set like this?

mycode_path = Path(r'\\192.168.1.10\d\mycode\src')

that will help us reproduce and fix this

Hi! It is a UNC path from a windows shared folder in our intranet. It is the folder where the source code lies.

kavvkon avatar Nov 13 '24 12:11 kavvkon

gotcha, hmm I see this now in the trace

CMD.EXE was started with the above path as the current directory.
UNC paths are not supported.  Defaulting to Windows directory.

@kavvkon can you try installing this branch and seeing if things are fixed there?

pip install git+https://github.com/prefecthq/prefect.git@path-to-unc

zzstoatzz avatar Nov 13 '24 19:11 zzstoatzz

Thanks! yes it seems to work fine

kavvkon avatar Nov 14 '24 13:11 kavvkon