workerd icon indicating copy to clipboard operation
workerd copied to clipboard

🐛 BUG:Python dependency packages cannot be correctly imported on worker

Open freelixilong opened this issue 1 year ago • 3 comments

Which Cloudflare product(s) does this pertain to?

Workers Runtime

What version(s) of the tool(s) are you using?

[email protected]

What version of Node are you using?

Python 3.10.12

What operating system and version are you using?

Ubuntu 22.04

Describe the Bug

Observed behavior

Python dependency packages cannot be correctly imported on worker, I want to imported requests package, but the requests depend on the urlib3, when i importing the failed happened. I had download the dependency package to the local.

Uncaught Error: PythonError: Traceback (most recent call last): File "/lib/python312.zip/_pyodide/_base.py", line 629, in pyimport_impl res = import(stem, fromlist=fromlist) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/session/metadata/entry.py", line 4, in import requests File "/session/metadata/requests/init.py", line 43, in import urllib3 File "/session/metadata/urllib3/init.py", line 209, in from .contrib.emscripten import inject_into_urllib3 # noqa: 401 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/session/metadata/urllib3/contrib/emscripten/init.py", line 6, in from .connection import EmscriptenHTTPConnection, EmscriptenHTTPSConnection File "/session/metadata/urllib3/contrib/emscripten/connection.py", line 17, in from .fetch import _RequestError, _TimeoutError, send_request, send_streaming_request File "/session/metadata/urllib3/contrib/emscripten/fetch.py", line 62, in .read_text(encoding="utf-8") ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/lib/python312.zip/pathlib.py", line 1027, in read_text with self.open(mode='r', encoding=encoding, errors=errors) as f: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/lib/python312.zip/pathlib.py", line 1013, in open return io.open(self, mode, buffering, encoding, errors, newline) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 44] No such file or directory: '/session/metadata/urllib3/contrib/emscripten/emscripten_fetch_worker.js'

at null.<anonymous> (pyodide-internal:generated/pyodide.asm:20:9998) in new_error
at [object Object] in $wrap_exception
at [object Object] in $pythonexc2js
at null.<anonymous> (pyodide-internal:generated/pyodide.asm:20:607956) in Module._pythonexc2js
at null.<anonymous> (pyodide-internal:generated/pyodide.asm:20:64098) in

Module.callPyObjectKwargs at null. (pyodide-internal:generated/pyodide.asm:20:65055) in Module.callPyObject at null. (pyodide-internal:generated/pyodide.asm:20:79132) in apply at null. (pyodide-internal:generated/pyodide.asm:20:77371) in apply at null. (pyodide-internal:generated/pyodide.asm:20:102862) in pyimport at null. (pyodide:python-entrypoint-helper:31:18) in pyimportMainModule [code: 10021]

Expected behavior

npx wrangler deploy

Steps to reproduce

The simple worker file

from js import Response

import requests async def on_fetch(request, env): return Response.new("Hello world!")

Please provide a link to a minimal reproduction

No response

Please provide any relevant error logs

Uncaught Error: PythonError: Traceback (most recent call last): File "/lib/python312.zip/_pyodide/_base.py", line 629, in pyimport_impl res = import(stem, fromlist=fromlist) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/session/metadata/entry.py", line 4, in import requests File "/session/metadata/requests/init.py", line 43, in import urllib3 File "/session/metadata/urllib3/init.py", line 209, in from .contrib.emscripten import inject_into_urllib3 # noqa: 401 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/session/metadata/urllib3/contrib/emscripten/init.py", line 6, in from .connection import EmscriptenHTTPConnection, EmscriptenHTTPSConnection File "/session/metadata/urllib3/contrib/emscripten/connection.py", line 17, in from .fetch import _RequestError, _TimeoutError, send_request, send_streaming_request File "/session/metadata/urllib3/contrib/emscripten/fetch.py", line 62, in .read_text(encoding="utf-8") ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/lib/python312.zip/pathlib.py", line 1027, in read_text with self.open(mode='r', encoding=encoding, errors=errors) as f: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/lib/python312.zip/pathlib.py", line 1013, in open return io.open(self, mode, buffering, encoding, errors, newline) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ FileNotFoundError: [Errno 44] No such file or directory: '/session/metadata/urllib3/contrib/emscripten/emscripten_fetch_worker.js'

at null.<anonymous> (pyodide-internal:generated/pyodide.asm:20:9998) in new_error
at [object Object] in $wrap_exception
at [object Object] in $pythonexc2js
at null.<anonymous> (pyodide-internal:generated/pyodide.asm:20:607956) in Module._pythonexc2js
at null.<anonymous> (pyodide-internal:generated/pyodide.asm:20:64098) in

Module.callPyObjectKwargs at null. (pyodide-internal:generated/pyodide.asm:20:65055) in Module.callPyObject at null. (pyodide-internal:generated/pyodide.asm:20:79132) in apply at null. (pyodide-internal:generated/pyodide.asm:20:77371) in apply at null. (pyodide-internal:generated/pyodide.asm:20:102862) in pyimport at null. (pyodide:python-entrypoint-helper:31:18) in pyimportMainModule [code: 10021]

freelixilong avatar Jun 17 '24 03:06 freelixilong

I'll ping the team working on Python support in Workers to take a look at this.

petebacondarwin avatar Jun 17 '24 13:06 petebacondarwin

Can you share your worker's full directory structure (including the file's contents)? Your Python script looks good, but are you also creating a requirements.txt file with requests in there?

dom96 avatar Jun 21 '24 16:06 dom96

Can you share your worker's full directory structure (including the file's contents)? Your Python script looks good, but are you also creating a requirements.txt file with requests in there?

The tutorial informs me that, for the time being, it does not support installing all dependencies through requirements.txt, so I have packaged all the dependencies locally. It is almost an empty project. The requirements.txt is also empty.

freelixilong avatar Jun 23 '24 02:06 freelixilong

I'm also having this issue, I'm using pywrangler

# pyproject.toml

[project]
name = "ai"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.12"
dependencies = ["langchain_core", "langchain_openai"]

[dependency-groups]
dev = ["workers-py"]

homerjam avatar Aug 15 '25 07:08 homerjam