opencv-python icon indicating copy to clipboard operation
opencv-python copied to clipboard

Breaking changes in module loading v4.6.0.66

Open M3ssman opened this issue 2 years ago • 6 comments

Expected behaviour

With opencv-python-headless-4.5.5.64 I was able to import opencv module into Python alike:

from cv2 import (
     cv2
)

this way any related functionalities could be accessed, for example via cv2.putText.

Actual behavior

Since yesterday, opencv-python-headless-4.6.0.66 gets pulled. Any automated build pipelines now break with the error

$ pytest --cov=lib tests/ -v
ImportError while loading conftest '/home/gitlab-runner/builds/QWycMLvj/0/ulb/ulb-it-migration/tests/conftest.py'.
tests/conftest.py:15: in <module>
    from cv2 import (
E   ImportError: Bindings generation error. Submodule name should always start with a parent module name. Parent name: cv2.cv2. Submodule name: cv2

Steps to reproduce

  • example code: please see above
  • ubuntu 20.04 LTS
  • x86
  • opencv-python-headless-4.6.0.66
Issue submission checklist
  • [x] This is not a generic OpenCV usage question (looking for help for coding, other usage questions, homework etc.)
  • [x] I have read the README of this repository and understand that this repository provides only an automated build toolchain for OpenCV Python packages (there is no actual OpenCV code here)
  • [ ] The issue is related to the build scripts in this repository, to the pre-built binaries or is a feature request (such as "please enable this additional dependency")
  • [x] I'm using the latest version of opencv-python

M3ssman avatar Jun 10 '22 05:06 M3ssman

Any suggestions welcome!

M3ssman avatar Jun 10 '22 05:06 M3ssman

Good day, why do you try to import the internal implementation module instead of the using cv2 module directly: import cv2?

$ python3 --version
Python 3.9.12
$ python3 -m venv venv
$ source venv/bin/activate
$ pip install opencv-python-headless==4.6.0.66
$ python -c "import cv2; print(cv2.__version__)"
4.6.0
$ python -c "import cv2; print(cv2.putText)"
<built-in function putText>

it is the same thing to do the following: from csv import _csv internal implementation - it is not possible and raises the error.

VadimLevin avatar Jun 10 '22 07:06 VadimLevin

I don't know how the cv2 is organized, by when you use

Good day, why do you try to import the internal implementation module instead of the using cv2 module directly: import cv2?

$ python3 --version
Python 3.9.12
$ python3 -m venv venv
$ source venv/bin/activate
$ pip install opencv-python-headless==4.6.0.66
$ python -c "import cv2; print(cv2.__version__)"
4.6.0
$ python -c "import cv2; print(cv2.putText)"
<built-in function putText>

it is the same thing to do the following: from csv import _csv internal implementation - it is not possible and raises the error.

I don't know how the cv2 is organized, but when you use "import cv2", the PyCharm (and pylint) is not able to detect the members of cv2 , so it is giving warning for any operation of cv2. The solution is to use 'from cv2 import cv2' at the beginning of the code.

Looks like the new version (4.6.066) changes the packaging of cv2, making 'from cv2 import cv2' no longer valid.

BenLin0 avatar Jun 18 '22 05:06 BenLin0

Chiming in to improve searchability! I'm getting new pylint errors as of the June 8 version 4.6.0.66 release:

No name 'VideoCapture' in module 'cv2' (no-name-in-module)

I have allow-listed cv2 as an extension package and still get a pylint error with the new version:

[tool.pylint.master]
extension-pkg-allow-list = ["cv2"]

This issue was not present in opencv-python-headless version 4.5.5.64. I'll downgrade to 4.5.5.64 until the fix is released. Thanks!

gregorybchris avatar Jun 21 '22 20:06 gregorybchris

Another pycharm user has to pin the version to 4.5.5.64

semaphore-egg avatar Jul 19 '22 11:07 semaphore-egg

Potentially related: https://github.com/opencv/opencv-python/issues/689

asmorkalov avatar Jul 28 '22 11:07 asmorkalov

This issue is also bothering me

kevinlinxc avatar Oct 10 '22 21:10 kevinlinxc

The best way to fix autocomplete for a compiled package is to distribute a pyi type stubs file with the package. Then autocomplete could work on just cv2 instead of using the hack that you used previously.

It's possible that type stubs could be generated automatically by mypy's stubgen module.

virtuald avatar Nov 26 '22 23:11 virtuald

There is activity on type stubs generation for OpenCV, but it's frozen today: they guys are very busy with main job: https://github.com/opencv/opencv/pull/20370

asmorkalov avatar Nov 28 '22 08:11 asmorkalov

This problem is still valid for opencv 4.7.0 if build it from source

I have a folder cv2 with cv2.so in it

If I call import cv2 within the folder -- all ok

If I call from without from cv2 import cv2 nothing will work with above described error

banderlog avatar Jan 01 '23 22:01 banderlog

crossposts:

  • https://forum.opencv.org/t/custom-built-opencv-4-7-0-python-import-problem/11469
  • https://stackoverflow.com/questions/74977869/custom-built-opencv-4-7-0-python-import-problem

this "from cv2 import cv2" nonsense is wrong. only pycharm users do it. stop doing it. it's wrong.

crackwitz avatar Jan 02 '23 12:01 crackwitz

@crackwitz you are posting the same 0 useful answer everywhere.

import foo from foo is a proper way of python importing if that corresponds to folder/file structure of your module

Let’s take a simple example with some dumb python module with only one function:

bkaba@bkaba:~$ tree some_python_module/
some_python_module/
├── __init__.py
├── __pycache__
│   ├── __init__.cpython-310.pyc
│   └── some_python_module.cpython-310.pyc
└── some_python_module.py

Contents of module:

bkaba@bkaba:~$ cat some_python_module/some_python_module.py
def foo():
    print('Oh no, it works')

Now let’s run it with the same python import rules:

bkaba@bkaba:~$ python3
Python 3.10.6 (main, Nov 14 2022, 16:10:14) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from some_python_module import some_python_module
>>> some_python_module.foo()
Oh no, it works

Now, please, tell me, why it supposed to fail and why it worked with opencv 4.5.5?

How I supposed to properly import cv2.so compiled library in python module?

With 4.5.5 working solution was

# __init__.py
import importlib
from .cv2 import *

# wildcard import above does not import "private" variables like __version__
# this makes them available
globals().update(importlib.import_module('cv2.cv2').__dict__)

banderlog avatar Jan 02 '23 15:01 banderlog

@crackwitz you are posting the same 0 useful answer everywhere. [...]

To add as to why it's a problem in the first place and we can't "just import cv2". Not being able to import cv2.cv2 also breaks non-static library analyzing tools. An obvious example is mypy's stubgen/stubtest, but it wouldn't be the only one.

Avasam avatar Jan 02 '23 19:01 Avasam

I was able to import opencv module into Python alike:

from cv2 import (
     cv2
)

Just invalid usage. Agreed with @crackwitz

So initial request and this issue are "invalid".


@banderlog Your example is not correct as you can't do that for any(!) python module. There are no such obligations on the internal structure from OpenCV Python Bindings which you assumed.

alalek avatar Jan 02 '23 21:01 alalek

ok, the workaround is:

# __init__.py of `foo_cv2` module with `cv2.so` within
import sys
import importlib
from importlib import resources


# get module path
path = resources.path(package='foo_cv2', resource="").__enter__()
# add it to PYTHONPATH
sys.path.insert(0, str(path))

# load all stuff
from cv2 import *
# wildcard import above does not import "private" variables like __version__
globals().update(importlib.import_module('cv2').__dict__)

And usage will be import foo_cv2 as cv2

@alalek ok, how it supposed to be imported correctly?

banderlog avatar Jan 02 '23 22:01 banderlog

how it supposed to be imported correctly?

I believe one week is enough to find OpenCV Python samples code:

  • https://github.com/opencv/opencv/tree/4.x/samples/python
  • https://github.com/opencv/opencv/tree/4.x/samples/dnn

to see import cv2 as cv everywhere.

It this doesn't work, then there is a problem with used Python environment.


Besides of strange approach of hacking...

sys.path.insert(0, str(path))

You should not modify the first entry of sys.path as it is reserved.


If you want just to import cv2.so / cv2.pyd / etc, then just put that file into site-packages location directly and drop "other" modules with cv2 name (including egg-like files). But this is not equal to opencv-python package anymore as some parts of OpenCV Python binding are missing in .so file.

alalek avatar Jan 11 '23 14:01 alalek

Fixed in:

  • https://github.com/opencv/opencv/pull/20370
  • https://github.com/opencv/opencv-python/pull/853

asmorkalov avatar Jun 23 '23 07:06 asmorkalov

please help me, how can fix this in jupyter import cv2 output: ~\anaconda3\lib\site-packages\cv2_init_.py in 179 180 --> 181 bootstrap()

~\anaconda3\lib\site-packages\cv2_init_.py in bootstrap() 173 174 for submodule in __collect_extra_submodules(DEBUG): --> 175 if __load_extra_py_code_for_module("cv2", submodule, DEBUG): 176 if DEBUG: print("Extra Python code for", submodule, "is loaded") 177

~\anaconda3\lib\site-packages\cv2_init_.py in __load_extra_py_code_for_module(base, name, enable_debug_print) 26 native_module = sys.modules.pop(module_name, None) 27 try: ---> 28 py_module = importlib.import_module(module_name) 29 except ImportError as err: 30 if enable_debug_print:

~\anaconda3\lib\importlib_init_.py in import_module(name, package) 125 break 126 level += 1 --> 127 return _bootstrap._gcd_import(name[level:], package, level) 128 129

~\anaconda3\lib\site-packages\cv2\typing_init_.py in 67 68 if numpy.lib.NumpyVersion(numpy.version) > "1.20.0" and sys.version_info >= (3, 9): ---> 69 NumPyArrayGeneric = numpy.ndarray[typing.Any, numpy.dtype[numpy.generic]] 70 else: 71 NumPyArrayGeneric = numpy.ndarray

TypeError: 'numpy._DTypeMeta' object is not subscriptable

alirezahamd254 avatar Nov 24 '23 03:11 alirezahamd254

@alirezahamd254 That is a separate issue, unrelated to this issue.

crackwitz avatar Nov 24 '23 08:11 crackwitz