poetry
poetry copied to clipboard
python-poetry
Tensorflow 2.13.0 cannot be imported after install
- Poetry version: Poetry (version 1.5.1)
- Python version: Python 3.11.4
-
OS version and name: macOS Ventura 13.3 M2 Chip
macOS-13.3-arm64-arm-64bit - pyproject.toml: pyproject.toml
- [x] I am on the latest stable Poetry version, installed using a recommended method.
- [x] I have searched the issues of this repo and believe that this is not a duplicate.
- [x] I have consulted the FAQ and blog for any relevant entries or release notes.
- [x] If an exception occurs when executing a command, I executed it again in debug mode (
-vvvoption) and have included the output below.
Issue
After installing tensorflow 2.13.0 with above pyproject.toml on macOs Apple Sillicon , importing fails with
ModuleNotFoundError: No module named 'tensorflow
Same method works under linux. HINT: in tensorflow 2.13 the authors introduce platform specific install for pip. Under macOs arm it is supposed to install tensorflow-macos . under linux there is another dependency tree
tensorflow's metadata does not express that dependency:
$ curl -s https://pypi.org/pypi/tensorflow/2.13.0/json | jq '.info.requires_dist'
[
"absl-py (>=1.0.0)",
"astunparse (>=1.6.0)",
"flatbuffers (>=23.1.21)",
"gast (<=0.4.0,>=0.2.1)",
"google-pasta (>=0.1.1)",
"h5py (>=2.9.0)",
"libclang (>=13.0.0)",
"numpy (<=1.24.3,>=1.22)",
"opt-einsum (>=2.3.2)",
"packaging",
"protobuf (!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3)",
"setuptools",
"six (>=1.12.0)",
"termcolor (>=1.1.0)",
"typing-extensions (<4.6.0,>=3.6.6)",
"wrapt (>=1.11.0)",
"grpcio (<2.0,>=1.24.3)",
"tensorboard (<2.14,>=2.13)",
"tensorflow-estimator (<2.14,>=2.13.0)",
"keras (<2.14,>=2.13.1)",
"tensorflow-io-gcs-filesystem (>=0.23.1) ; platform_machine != \"arm64\" or platform_system != \"Darwin\""
]
No mention here of tensorflow-macos.
I guess this is another variation on https://github.com/tensorflow/tensorflow/issues/58674
please encourage the tensorflow folk to publish consistent metadata in all of their wheels, with platform-specific variations described by markers
@dre-hh Have using environment markers to get around the issue?
If we take a look at tensorflow/tools/pip_package/setup.py, we see that they require the tensorflow-macos package where the environment marker has been satisfied.
It's generated, but would look something like this.
tensorflow-macos == 2.15.0 ;platform_system=="Darwin" and platform_machine=="arm64"
Following the Poetry documentation here, your dependency specification should look this.
[tool.poetry.dependencies]
python = ">=3.11,<3.12"
tensorflow-macos = { version = "^2.15.0", markers = "platform_system=='Darwin' and platform_machine=='arm64'" }
tensorflow = "^2.13.0"
I'd tend to agree with @dimbleby, the devs over at @tensorflow should improve the compatibility of their project with package managers, but that's another battle entirely.
For people still looking for a full combo workaround, this should work for tensorflow 2.11 in all situations (in reality I haven't tested everything). Don't hesitate to comment with your problems so that we improve on this. I've tested :
- new apple silicon
- docker in new apple silicon
- standard linux
This should work with windows, new mac, old mac, docker in new mac, amd and non-amd linux machines ... EDIT : Tested :
- windows by @Vuizur
Currently, I advise to copy the full code below in your pyproject.toml in the poetry's dependencies.
Using a poetry group, [tool.poetry.group.tensorflow.dependencies] section could be a good thing to be able to isolate it :
# Issue between poetry and tensorflow metadata since >=2.11
# This is a temporary workaround
# related to https://github.com/python-poetry/poetry/issues/8271
# Inspired from https://github.com/tensorflow/tensorflow/blob/adb39b04e9cb116df4659a7e2de9eea27e62f25c/tensorflow/tools/pip_package/setup.py#L148-L162
tensorflow = {version = "^2.13.0" }
tensorflow-macos = { version = "^2.13.0", platform = "darwin", markers = "platform_machine=='arm64'" }
tensorflow-intel = { version = "^2.13.0", platform = "win32" }
tensorflow-cpu = [
{ version = "^2.13.0", platform = "linux", markers = "platform_machine!='arm64' and platform_machine!='aarch64'" },
{ version = "^2.13.0", platform = "darwin", markers = "platform_machine!='arm64' and platform_machine!='aarch64'" },
]
tensorflow-cpu-aws = { version = "^2.13.0", platform = "linux", markers = "platform_machine=='arm64' or platform_machine=='aarch64'" }
# https://github.com/tensorflow/tensorflow/blob/adb39b04e9cb116df4659a7e2de9eea27e62f25c/tensorflow/tools/pip_package/setup.py#L107-L108
# https://github.com/python-poetry/poetry/issues/8271#issuecomment-1697740447
tensorflow-io-gcs-filesystem = [
{ version = ">= 0.23.1", markers = "platform_machine!='arm64' or platform_system!='Darwin'" },
{ version = "< 0.32.0", markers = "platform_system == 'Windows'" }
]
EDIT : I fixed above the tensorflow-io-gcs-filesystem issue from @RRiva EDIT 2 : adding @radoering answer EDIT 3 : fixing windows from @Vuizur
Thanks a lot for the suggestion. Poetry generated the lock file, but when I test my package on Windows the installation fails with
Installing tensorflow-io-gcs-filesystem (0.33.0)
RuntimeError
Unable to find installation candidates for tensorflow-io-gcs-filesystem (0.33.0)
It's not entirely Poetry's fault, as this package doesn't provide wheels for Windows on PyPi, but still...
Hi @RRiva ,
I went back to the tensorflow file and added what seems to be their way to specify "tensorflow-io-gcs-filesystem" Can you try again and confirm it works ?
Still a temporary fix. Wondering about publishing something like a package with just a pyproject.toml to avoid everybody having to fix it. I am afraid it will made both side lazy. :/
@dimbleby you seem to better grasp what's up. Do you have any documentation or an issue that will give an initial roadway on how to fix this ?
Thanks again 🙂 Unfortunately, the lock file is the same as before, and therefore the installation fails in the same way. Might be related to #1616.
#1616 is about different path dependencies with the same version (and resolved), thus, not related.
It's related to #2012 and #4046. If there is no wheel for Windows (and no sdist), you'll want to exclude that version for Windows:
tensorflow-io-gcs-filesystem = [
{ version = ">= 0.23.1", markers = "platform_machine!='arm64' or platform_system!='Darwin'" },
{ version = "< 0.32.0", markers = "platform_system == 'Windows'" }
]
@radoering adding it to my previous answering and citing this to have a standalone.
Unfortunately the workaround by Wirg and radoering still does not seem to work on Windows.
The pyproject.toml:
[tool.poetry]
name = "tt2"
version = "0.1.0"
description = ""
authors = ["Hannes Krumbiegel <[email protected]>"]
readme = "README.md"
[tool.poetry.dependencies]
python = "^3.10,<3.12"
# Issue between poetry and tensorflow metadata since >=2.11
# This is a temporary workaround
# related to https://github.com/python-poetry/poetry/issues/8271
# Inspired from https://github.com/tensorflow/tensorflow/blob/adb39b04e9cb116df4659a7e2de9eea27e62f25c/tensorflow/tools/pip_package/setup.py#L148-L162
tensorflow = {version = "^2.13.0" }
tensorflow-macos = { version = "^2.13.0", platform = "darwin", markers = "platform_machine=='arm64'" }
tensorflow-intel = { version = "^2.13.0", platform = "windows" }
tensorflow-cpu = [
{ version = "^2.13.0", platform = "linux", markers = "platform_machine!='arm64' and platform_machine!='aarch64'" },
{ version = "^2.13.0", platform = "darwin", markers = "platform_machine!='arm64' and platform_machine!='aarch64'" },
]
tensorflow-cpu-aws = { version = "^2.13.0", platform = "linux", markers = "platform_machine=='arm64' or platform_machine=='aarch64'" }
# https://github.com/tensorflow/tensorflow/blob/adb39b04e9cb116df4659a7e2de9eea27e62f25c/tensorflow/tools/pip_package/setup.py#L107-L108
# https://github.com/python-poetry/poetry/issues/8271#issuecomment-1697740447
tensorflow-io-gcs-filesystem = [
{ version = ">= 0.23.1", markers = "platform_machine!='arm64' or platform_system!='Darwin'" },
{ version = "< 0.32.0", markers = "platform_system == 'Windows'" }
]
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
causes an error with:
Creating virtualenv tt2 in D:\Programs\tt2\.venv
Updating dependencies
Resolving dependencies...
Package operations: 39 installs, 1 update, 0 removals
• Installing certifi (2023.7.22)
• Installing charset-normalizer (3.2.0)
• Installing idna (3.4)
• Installing pyasn1 (0.5.0)
• Installing urllib3 (1.26.16)
• Installing cachetools (5.3.1)
• Installing oauthlib (3.2.2)
• Installing rsa (4.9)
• Installing requests (2.31.0)
• Installing pyasn1-modules (0.3.0)
• Installing six (1.16.0)
• Installing google-auth (2.22.0)
• Installing markupsafe (2.1.3)
• Installing requests-oauthlib (1.3.1)
• Installing absl-py (1.4.0)
• Installing google-auth-oauthlib (1.0.0)
• Installing numpy (1.24.3)
• Installing markdown (3.4.4)
• Installing tensorboard-data-server (0.7.1)
• Updating setuptools (68.0.0 -> 68.2.0)
• Installing protobuf (4.24.3)
• Installing grpcio (1.58.0)
• Installing werkzeug (2.3.7)
• Installing astunparse (1.6.3)
• Installing flatbuffers (23.5.26)
• Installing h5py (3.9.0)
• Installing google-pasta (0.2.0)
• Installing gast (0.4.0)
• Installing keras (2.13.1)
• Installing tensorflow-io-gcs-filesystem (0.34.0)
• Installing packaging (23.1)
• Installing tensorflow-estimator (2.13.0)
• Installing libclang (16.0.6)
• Installing wrapt (1.15.0)
• Installing typing-extensions (4.5.0)
• Installing termcolor (2.3.0)
• Installing tensorboard (2.13.0)
• Installing opt-einsum (3.3.0)
• Installing tensorflow-io-gcs-filesystem (0.31.0)
RuntimeError
Unable to find installation candidates for tensorflow-io-gcs-filesystem (0.34.0)
at ~\.local\pipx\venvs\poetry\Lib\site-packages\poetry\installation\chooser.py:76 in choose_for
72│
73│ links.append(link)
74│
75│ if not links:
→ 76│ raise RuntimeError(f"Unable to find installation candidates for {package}")
77│
78│ # Get the best link
79│ chosen = max(links, key=lambda link: self._sort_key(package, link))
80│
(For both Python 3.10 and 3.11 I get an error like this.)
I got the idea to upgrade Poetry from 3.5 to 3.6 or so, and the install seems to work:
poetry install
Updating dependencies
Resolving dependencies...
Package operations: 1 install, 0 updates, 0 removals
• Installing tensorflow (2.13.0)
Writing lock file
Installing the current project: tt2 (0.1.0)
But when I try to execute a module with import tensorflow, I get:
File "d:\Programs\tt2\tt2\test.py", line 1, in <module>
import tensorflow
ModuleNotFoundError: No module named 'tensorflow'
I found the solution: Wirg's solution actually specified the wrong platform for pytorch-intel, one doesn't have to write windows, but win32. Here is a corrected version:
[tool.poetry.dependencies]
python = "^3.10,<3.12"
# Issue between poetry and tensorflow metadata since >=2.11
# This is a temporary workaround
# related to https://github.com/python-poetry/poetry/issues/8271
# Inspired from https://github.com/tensorflow/tensorflow/blob/adb39b04e9cb116df4659a7e2de9eea27e62f25c/tensorflow/tools/pip_package/setup.py#L148-L162
tensorflow = {version = "^2.13.0" }
tensorflow-macos = { version = "^2.13.0", platform = "darwin", markers = "platform_machine=='arm64'" }
tensorflow-intel = { version = "^2.13.0", platform = "win32" }
tensorflow-cpu = [
{ version = "^2.13.0", platform = "linux", markers = "platform_machine!='arm64' and platform_machine!='aarch64'" },
{ version = "^2.13.0", platform = "darwin", markers = "platform_machine!='arm64' and platform_machine!='aarch64'" },]
tensorflow-cpu-aws = { version = "^2.13.0", platform = "linux", markers = "platform_machine=='arm64' or platform_machine=='aarch64'" }
# https://github.com/tensorflow/tensorflow/blob/adb39b04e9cb116df4659a7e2de9eea27e62f25c/tensorflow/tools/pip_package/setup.py#L107-L108
# https://github.com/python-poetry/poetry/issues/8271#issuecomment-1697740447
tensorflow-io-gcs-filesystem = [
{ version = ">= 0.23.1", markers = "platform_machine!='arm64' or platform_system!='Darwin'" },
{ version = "< 0.32.0", markers = "platform_system == 'Windows'" }
]
For people still looking for a full combo workaround, this should work for tensorflow 2.11 in all situations (in reality I haven't tested everything). Don't hesitate to comment with your problems so that we improve on this. I've tested :
- new apple silicon
- docker in new apple silicon
- standard linux
This should work with windows, new mac, old mac, docker in new mac, amd and non-amd linux machines ... EDIT : Tested :
- windows by @Vuizur
Currently, I advise to copy the full code below in your
pyproject.tomlin the poetry's dependencies. Using a poetry group,[tool.poetry.group.tensorflow.dependencies]section could be a good thing to be able to isolate it :# Issue between poetry and tensorflow metadata since >=2.11 # This is a temporary workaround # related to https://github.com/python-poetry/poetry/issues/8271 # Inspired from https://github.com/tensorflow/tensorflow/blob/adb39b04e9cb116df4659a7e2de9eea27e62f25c/tensorflow/tools/pip_package/setup.py#L148-L162 tensorflow = {version = "^2.13.0" } tensorflow-macos = { version = "^2.13.0", platform = "darwin", markers = "platform_machine=='arm64'" } tensorflow-intel = { version = "^2.13.0", platform = "win32" } tensorflow-cpu = [ { version = "^2.13.0", platform = "linux", markers = "platform_machine!='arm64' and platform_machine!='aarch64'" }, { version = "^2.13.0", platform = "darwin", markers = "platform_machine!='arm64' and platform_machine!='aarch64'" }, ] tensorflow-cpu-aws = { version = "^2.13.0", platform = "linux", markers = "platform_machine=='arm64' or platform_machine=='aarch64'" } # https://github.com/tensorflow/tensorflow/blob/adb39b04e9cb116df4659a7e2de9eea27e62f25c/tensorflow/tools/pip_package/setup.py#L107-L108 # https://github.com/python-poetry/poetry/issues/8271#issuecomment-1697740447 tensorflow-io-gcs-filesystem = [ { version = ">= 0.23.1", markers = "platform_machine!='arm64' or platform_system!='Darwin'" }, { version = "< 0.32.0", markers = "platform_system == 'Windows'" } ]EDIT : I fixed above the tensorflow-io-gcs-filesystem issue from @RRiva EDIT 2 : adding @radoering answer EDIT 3 : fixing windows from @Vuizur
Just tried using this solution already taking into account the 3rd edit, and it's not working on windows, currently using Poetry version: 1.4.2, CUDA: 11.8 and windows 10. Adding this to the toml either keeps trying to install i/o package v0.34.0 which it shouldn't, and when forced to install 0.31.0, it tries also installing tensorflow-intel, and regardless of that, at the end tensorflow still cannot be imported. Has it worked for anyone else in windows? or any hints towards what I could do next?
currently using Poetry version: 1.4.2
Has it worked for anyone else in windows? or any hints towards what I could do next?
At first, please try the latest version of Poetry. 1.4.2 had already been out of date when this issue was created. (I don't know if it works with a more recent version, but it does not help to find bugs that may have already been fixed.)
currently using Poetry version: 1.4.2
Has it worked for anyone else in windows? or any hints towards what I could do next?
At first, please try the latest version of Poetry. 1.4.2 had already been out of date when this issue was created. (I don't know if it works with a more recent version, but it does not help to find bugs that may have already been fixed.)
Thank you for pointing this out, I didn't notice I was not working on the latest Poetry release. Already updated to Poetry v1.7.1, installed the project, which indeed now is able to install all depedencies smoothly, unlike when installing with 1.4.2, but still tensorflow cannot be imported. I will include my toml and the output from poetry list if it's of any help. Thanks beforehand for any suggestion.
toml:
[tool.poetry]
name = "tf-project"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]
readme = "README.md"
packages = [{include = "tf_project"}]
[tool.poetry.dependencies]
python = ">=3.9, <3.12"
[tool.poetry.group.dev.dependencies]
# tensorflow = ">=2.11.0, <2.12.0"
# tensorflow-io-gcs-filesystem = "^0.31.0"
opencv-python = "^4.7.0"
jupyter = "^1.0.0"
fs = "^2.4.16"
[tool.poetry.group.tensorflow.dependencies]
tensorflow = {version = "^2.13.0" }
tensorflow-macos = { version = "^2.13.0", platform = "darwin", markers = "platform_machine=='arm64'" }
tensorflow-intel = { version = "^2.13.0", platform = "win32" }
tensorflow-cpu = [
{ version = "^2.13.0", platform = "linux", markers = "platform_machine!='arm64' and platform_machine!='aarch64'" },
{ version = "^2.13.0", platform = "darwin", markers = "platform_machine!='arm64' and platform_machine!='aarch64'" },
]
tensorflow-cpu-aws = { version = "^2.13.0", platform = "linux", markers = "platform_machine=='arm64' or platform_machine=='aarch64'" }
tensorflow-io-gcs-filesystem = [
{ version = ">= 0.23.1", markers = "platform_machine!='arm64' or platform_system!='Darwin'" },
{ version = "< 0.32.0", markers = "platform_system == 'Windows'" }
]
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
poetry show output:
absl-py 2.0.0 Abseil Python Common Libraries, see https://github.com/abseil/abseil-py.
anyio 4.1.0 High level compatibility layer for multiple asynchronous event loop implementations
appdirs 1.4.4 A small Python module for determining appropriate platform-specific dirs, e.g. a "user d...
argon2-cffi 23.1.0 Argon2 for Python
argon2-cffi-bindings 21.2.0 Low-level CFFI bindings for Argon2
arrow 1.3.0 Better dates & times for Python
asttokens 2.4.1 Annotate AST trees with source code positions
astunparse 1.6.3 An AST unparser for Python
async-lru 2.0.4 Simple LRU cache for asyncio
attrs 23.1.0 Classes Without Boilerplate
babel 2.13.1 Internationalization utilities
beautifulsoup4 4.12.2 Screen-scraping library
bleach 6.1.0 An easy safelist-based HTML-sanitizing tool.
cachetools 5.3.2 Extensible memoizing collections and decorators
certifi 2023.11.17 Python package for providing Mozilla's CA Bundle.
cffi 1.16.0 Foreign Function Interface for Python calling C code.
charset-normalizer 3.3.2 The Real First Universal Charset Detector. Open, modern and actively maintained alternat...
colorama 0.4.6 Cross-platform colored terminal text.
comm 0.2.0 Jupyter Python Comm implementation, for usage in ipykernel, xeus-python etc.
debugpy 1.8.0 An implementation of the Debug Adapter Protocol for Python
decorator 5.1.1 Decorators for Humans
defusedxml 0.7.1 XML bomb protection for Python stdlib modules
exceptiongroup 1.2.0 Backport of PEP 654 (exception groups)
executing 2.0.1 Get the currently executing AST node of a frame, and other information
fastjsonschema 2.19.0 Fastest Python implementation of JSON schema
flatbuffers 23.5.26 The FlatBuffers serialization format for Python
fqdn 1.5.1 Validates fully-qualified domain names against RFC 1123, so that they are acceptable to ...
fs 2.4.16 Python's filesystem abstraction layer
gast 0.5.4 Python AST that abstracts the underlying Python version
google-auth 2.23.4 Google Authentication Library
google-auth-oauthlib 1.1.0 Google Authentication Library
google-pasta 0.2.0 pasta is an AST-based Python refactoring library
grpcio 1.59.3 HTTP/2-based RPC framework
h5py 3.10.0 Read and write HDF5 files from Python
idna 3.6 Internationalized Domain Names in Applications (IDNA)
importlib-metadata 6.8.0 Read metadata from Python packages
ipykernel 6.26.0 IPython Kernel for Jupyter
ipython 8.18.1 IPython: Productive Interactive Computing
ipywidgets 8.1.1 Jupyter interactive widgets
isoduration 20.11.0 Operations with ISO 8601 durations
jedi 0.19.1 An autocompletion tool for Python that can be used for text editors.
jinja2 3.1.2 A very fast and expressive template engine.
json5 0.9.14 A Python implementation of the JSON5 data format.
jsonpointer 2.4 Identify specific nodes in a JSON document (RFC 6901)
jsonschema 4.20.0 An implementation of JSON Schema validation for Python
jsonschema-specifications 2023.11.1 The JSON Schema meta-schemas and vocabularies, exposed as a Registry
jupyter 1.0.0 Jupyter metapackage. Install all the Jupyter components in one go.
jupyter-client 8.6.0 Jupyter protocol implementation and client libraries
jupyter-console 6.6.3 Jupyter terminal console
jupyter-core 5.5.0 Jupyter core package. A base package on which Jupyter projects rely.
jupyter-events 0.9.0 Jupyter Event System library
jupyter-lsp 2.2.1 Multi-Language Server WebSocket proxy for Jupyter Notebook/Lab server
jupyter-server 2.10.1 The backend—i.e. core services, APIs, and REST endpoints—to Jupyter web applications.
jupyter-server-terminals 0.4.4 A Jupyter Server Extension Providing Terminals.
jupyterlab 4.0.9 JupyterLab computational environment
jupyterlab-pygments 0.3.0 Pygments theme using JupyterLab CSS variables
jupyterlab-server 2.25.2 A set of server components for JupyterLab and JupyterLab like applications.
jupyterlab-widgets 3.0.9 Jupyter interactive widgets for JupyterLab
keras 2.15.0 Deep learning for humans.
libclang 16.0.6 Clang Python Bindings, mirrored from the official LLVM repo: https://github.com/llvm/llv...
markdown 3.5.1 Python implementation of John Gruber's Markdown.
markupsafe 2.1.3 Safely add untrusted strings to HTML/XML markup.
matplotlib-inline 0.1.6 Inline Matplotlib backend for Jupyter
mistune 3.0.2 A sane and fast Markdown parser with useful plugins and renderers
ml-dtypes 0.2.0
nbclient 0.9.0 A client library for executing notebooks. Formerly nbconvert's ExecutePreprocessor.
nbconvert 7.11.0 Converting Jupyter Notebooks
nbformat 5.9.2 The Jupyter Notebook format
nest-asyncio 1.5.8 Patch asyncio to allow nested event loops
notebook 7.0.6 Jupyter Notebook - A web-based notebook environment for interactive computing
notebook-shim 0.2.3 A shim layer for notebook traits and config
numpy 1.26.2 Fundamental package for array computing in Python
oauthlib 3.2.2 A generic, spec-compliant, thorough implementation of the OAuth request-signing logic
opencv-python 4.8.1.78 Wrapper package for OpenCV python bindings.
opt-einsum 3.3.0 Optimizing numpys einsum function
overrides 7.4.0 A decorator to automatically detect mismatch when overriding a method.
packaging 23.2 Core utilities for Python packages
pandocfilters 1.5.0 Utilities for writing pandoc filters in python
parso 0.8.3 A Python Parser
platformdirs 4.0.0 A small Python package for determining appropriate platform-specific dirs, e.g. a "user ...
prometheus-client 0.19.0 Python client for the Prometheus monitoring system.
prompt-toolkit 3.0.41 Library for building powerful interactive command lines in Python
protobuf 4.23.4
psutil 5.9.6 Cross-platform lib for process and system monitoring in Python.
pure-eval 0.2.2 Safely evaluate AST nodes without side effects
pyasn1 0.5.1 Pure-Python implementation of ASN.1 types and DER/BER/CER codecs (X.208)
pyasn1-modules 0.3.0 A collection of ASN.1-based protocols modules
pycparser 2.21 C parser in Python
pygments 2.17.2 Pygments is a syntax highlighting package written in Python.
python-dateutil 2.8.2 Extensions to the standard Python datetime module
python-json-logger 2.0.7 A python library adding a json log formatter
pywin32 306 Python for Window Extensions
pywinpty 2.0.12 Pseudo terminal support for Windows from Python.
pyyaml 6.0.1 YAML parser and emitter for Python
pyzmq 25.1.1 Python bindings for 0MQ
qtconsole 5.5.1 Jupyter Qt console
qtpy 2.4.1 Provides an abstraction layer on top of the various Qt bindings (PyQt5/6 and PySide2/6).
referencing 0.31.0 JSON Referencing + Python
requests 2.31.0 Python HTTP for Humans.
requests-oauthlib 1.3.1 OAuthlib authentication support for Requests.
rfc3339-validator 0.1.4 A pure python RFC3339 validator
rfc3986-validator 0.1.1 Pure python rfc3986 validator
rpds-py 0.13.1 Python bindings to Rust's persistent data structures (rpds)
rsa 4.9 Pure-Python RSA implementation
send2trash 1.8.2 Send file to trash natively under Mac OS X, Windows and Linux
setuptools 69.0.2 Easily download, build, install, upgrade, and uninstall Python packages
six 1.16.0 Python 2 and 3 compatibility utilities
sniffio 1.3.0 Sniff out which async library your code is running under
soupsieve 2.5 A modern CSS selector implementation for Beautiful Soup.
stack-data 0.6.3 Extract data from python stack frames and tracebacks for informative displays
tensorboard 2.15.1 TensorBoard lets you watch Tensors Flow
tensorboard-data-server 0.7.2 Fast data loading for TensorBoard
tensorflow 2.15.0 TensorFlow is an open source machine learning framework for everyone.
tensorflow-estimator 2.15.0 TensorFlow Estimator.
tensorflow-intel 2.15.0 TensorFlow is an open source machine learning framework for everyone.
tensorflow-io-gcs-filesystem 0.31.0 TensorFlow IO
termcolor 2.3.0 ANSI color formatting for output in terminal
terminado 0.18.0 Tornado websocket backend for the Xterm.js Javascript terminal emulator library.
tinycss2 1.2.1 A tiny CSS parser
tomli 2.0.1 A lil' TOML parser
tornado 6.3.3 Tornado is a Python web framework and asynchronous networking library, originally develo...
traitlets 5.13.0 Traitlets Python configuration system
types-python-dateutil 2.8.19.14 Typing stubs for python-dateutil
typing-extensions 4.8.0 Backported and Experimental Type Hints for Python 3.8+
uri-template 1.3.0 RFC 6570 URI Template Processor
urllib3 2.1.0 HTTP library with thread-safe connection pooling, file post, and more.
wcwidth 0.2.12 Measures the displayed width of unicode strings in a terminal
webcolors 1.13 A library for working with the color formats defined by HTML and CSS.
webencodings 0.5.1 Character encoding aliases for legacy web content
websocket-client 1.6.4 WebSocket client for Python with low level API options
werkzeug 3.0.1 The comprehensive WSGI web application library.
wheel 0.42.0 A built-package format for Python
widgetsnbextension 4.0.9 Jupyter interactive widgets for Jupyter Notebook
wrapt 1.14.1 Module for decorators, wrappers and monkey patching.
zipp 3.17.0 Backport of pathlib-compatible object wrapper for zip files
python3 -c 'import tensorflow as tf; print(tf.__version__)' error:
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'tensorflow'
looks like you simply haven't activated the virtual environment
no, I am activating the environment. I also tried this inside a jupyter notebook inside a poetry shell and the same happens, it's only tensorflow that seems to not work.
Edit: Actually I just noticed that apparently inside my jupyter notebook, tensorflow and all other libraries can now be imported, but in the ps prompt when I run python3 I cannot import any package, but this is the same for all other projects I have, although they all work inside the notebooks I launch within the poetry shell, I never noticed before. Is this supposed to happen?
I'm having this same issue with poetry and tensorflow only. poetry run pip list shows many things and those things import in the python REPL, but not tensorflow.
# poetry 1.7.1
# python 3.11.6
# tensorflow 2.15.0 (as resolved by poetry)
# mac M1
$ poetry pip list
# a bunch but let's just say pandas, pytest and tensorflow
$ poetry run python
>>> import pytest
# works
>>> import pandas
# works
>>> import tensorflow as tf
ModuleNotFoundError: No module named 'tensorflow'
I do not have any of these packages installed globally. So, I know my environment is fine. I even did poetry env remove [the name] and did a poetry install again. And poetry cache clear . --all. I have no idea what is going on. Some interaction between poetry and tensorflow?
ah, I see the issues linked at the top ... I should have followed those first
I've just tried Vuizur's solution. I can generate the lock file, and install my package on Linux and Windows, using Poetry 1.7.1. What I don't understand is why TensorFlow can be imported on my Windows 11 pc, but not on the CI machine with Docker container python:3.11-windowsservercore. Using poetry shell does not change anything.
tensorflow versions up to but not including 2.15.0.post1 fail to declare their windows-only dependency on tensorflow-intel.
$ curl -s https://pypi.org/pypi/tensorflow/2.15.0/json | jq '.info.requires_dist'
...
output that does not mention tensorflow-intel
tensorflow 2.15.0.post1 declares
tensorflow-intel (==2.15.0.post1) ; platform_system == \"Windows\"
which would be right, except that they have failed to publish a tensorflow-intel with that version.
y'all should be reporting these things to tensorflow, not poetry
Hi @dimbleby, thanks for looking into this 🙂 Unfortunately, we did report bugs to the TensorFlow folks, but so far they have not been addressed. I've added your finding to the pile.
- https://github.com/tensorflow/io/issues/1789
- https://github.com/tensorflow/io/issues/1815
- https://github.com/tensorflow/io/issues/1832
- https://github.com/tensorflow/io/issues/1881
- https://github.com/tensorflow/io/issues/1906
- https://github.com/tensorflow/tensorflow/issues/58674
- https://github.com/tensorflow/tensorflow/issues/61477
- https://github.com/tensorflow/tensorflow/issues/62000
- https://github.com/tensorflow/tensorflow/issues/62899
I guess that now that Keras supports PyTorch and JAX, it's a good time to move to a different machine learning library.
The issue has been resolved on Windows: please refer https://github.com/tensorflow/tensorflow/issues/62899#issuecomment-1935052936
Hi @dimbleby, can you please let me know what error are you getting and on which platform?
https://github.com/python-poetry/poetry/issues/8271#issuecomment-1928038659
Telling users that they should manually add the dependencies that tensorflow forgot to declare is at best a workaround, not a resolution.
Hi @dimbleby , TensorFlow supports pip as Python package manager: https://www.tensorflow.org/install/pip#windows-native. We use pip for our internal testing. Does the issue reported here happen only for Poetry?
tensorflow 2.15.0.post1 declares
tensorflow-intel (==2.15.0.post1) ; platform_system == \"Windows\"which would be right, except that they have failed to publish a tensorflow-intel with that version.
is obviously going to be true for everyone, I do not understand what testing you can be doing that does not find this
building cross-platform solutions and therefore wanting consistent metadata in all distributions is more poetry-specific, though you would have to ask all the other tools - rather than ask here - if you want to know whether all the other tools also mind.
Apart from the failure to publish 2.15.0.post1 versions of things it seems as though tensorflow is almost there...
For some reason, even though I can see tensorflow-macos v2.15.0 on PyPI, it doesn't seem to be available.
Even though it appears in poetry.lock file:
[[package]]
name = "tensorflow-macos"
version = "2.15.0"
When I install it in my virtual environment, the dependency doesn't seem to be there. When I attempt to download if directly via pip, I see the following error:
pip install tensorflow-macos==2.15.0
ERROR: Could not find a version that satisfies the requirement tensorflow-macos==2.15.0 (from versions: 2.9.0, 2.9.1, 2.9.2, 2.10.0, 2.11.0, 2.12.0)
ERROR: No matching distribution found for tensorflow-macos==2.15.0
PS: I made sure to upgrade pip to latest version (currently 24.0).
Regardless, even with v2.12.0, tensorflow-macos never gets installed on my machine (Apple M1 Pro) using the workaround proposed by @Wirg
Any thoughts anyone?
Update: In fact, I can get tensorflow-macos on my Apple M1 Pro, if I remove the markers from what @Wirg proposed:
tensorflow-macos = { version = "==2.12.0", platform = "darwin" }
Apart from the failure to publish 2.15.0.post1 versions of things it seems as though tensorflow is almost there...
this optimism seems to have been unwarranted, tensorflow have now rearranged things significantly - and they are once again providing completely different metadata in different distributions of their most recent release.
eg compare the output of curl -s https://files.pythonhosted.org/packages/71/a2/863a553d48f52d8f719626bb6d1c6f51d77e26d4bb9445a48fbf73f8e445/tensorflow-2.16.1-cp312-cp312-macosx_12_0_arm64.whl.metadata | grep Requires-Dist with curl -s https://files.pythonhosted.org/packages/76/4f/39ddae9fb07b8c039fa5a5f2b6623c6e0564199d82da33fcef62bcf93174/tensorflow-2.16.1-cp312-cp312-win_amd64.whl.metadata | grep Requires-Dist
to be clear: this variation is (unfortunately!) allowed by the packaging specifications, if being poetry-compatible is more trouble than it is worth to tensorflow users and contributors then so be it.
But this is not and almost certainly never will be compatible with installation through poetry.
Things should improve with the next TensorFlow release. Fingers crossed 🙂 https://github.com/tensorflow/tensorflow/issues/62899#issuecomment-2007774669
On windows, Also should do poetry.exe add tensorflow-intel operation seperately. And then import tensorflow will work fine.
