pyinstaller icon indicating copy to clipboard operation
pyinstaller copied to clipboard

Numpy Import Issue

Open ankushraut008 opened this issue 1 year ago • 34 comments

Facing the numpy Import issue

:\Development\ANKUSH\arc\Signals>main.exeTraceback (most recent call last):File "main.py", line 10, in import pandas as pdFile "PyInstaller\loader\pyimod02_importers.py", line 419, in exec_module File "pandas_init_.py", line 32, in ImportError: Unable to import required dependencies:numpy: Error importing numpy: you should not try to import numpy fromits source directory; please exit the numpy source tree, and relaunchyour python interpreter from there.[10864] Failed to execute script 'main' due to unhandled exception!

ankushraut008 avatar Aug 20 '24 06:08 ankushraut008

What version of python are you using? What version of numpy? And what version of PyInstaller?

rokm avatar Aug 20 '24 07:08 rokm

Python version: 3.11.4 NumPy version: 2.0.1 PyInstaller version: 6.10.0

ankushraut008 avatar Aug 20 '24 08:08 ankushraut008

That combination seems to work as expected on my system.

Can you try freezing the following minimal program

import numpy
print(numpy)

Does it raise the same error? If it does, paste the PyInstaller command that you used and the complete build log here.

rokm avatar Aug 20 '24 09:08 rokm

tried with your command not found any issue

below is the command we are using

def create_exe(script_path, extra_files=None):
    try:
        script_directory = Path(script_path).parent
        extra_files = extra_files or []

        if len(extra_files) > 0:
            # Format extra files correctly for PyInstaller
            # Format extra files correctly for PyInstaller
            extra_files_formatted = [f"{file};." for file in extra_files]

            # Construct the command with proper syntax
            command = [
                'python',
                '-m', 'PyInstaller',
                '--distpath', str(script_directory),
                *[f"--add-data={file}" for file in extra_files_formatted],
                script_path
            ]

        else:
            # Construct the command with proper syntax
            command = [
                'python',
                '-m', 'PyInstaller',
                '--distpath', str(script_directory),
                script_path
            ]

        subprocess.run(command, check=True)

        exe_name = os.path.join(script_directory, 'dist', os.path.basename(script_path)[:-3] + '.exe')
        print(f"Conversion successful! Executable created at {exe_name}")
    except subprocess.CalledProcessError as e:
        print(f"Conversion failed: {e}")
    except Exception as e:
        print(f"An error occurred: {e}")

we are generating the multiple exe at a time using this function

ankushraut008 avatar Aug 20 '24 09:08 ankushraut008

tried with your command not found any issue

Then either this is specific to your executable, or the custom build system that you have in place.

What if you try to build one of the executables manually, without this build script?

Also, what are you passing via extra_files in the case of the failing build?

rokm avatar Aug 20 '24 09:08 rokm

the same issue was getting generated while running the command manually

The extra_files I am passing was supportable json files required

ankushraut008 avatar Aug 20 '24 10:08 ankushraut008

Hmm, can you come up with a minimal example that triggers the error? If basic import numpy program did not, how about import pandas?

rokm avatar Aug 20 '24 10:08 rokm

Well can you have the most basic command you can give us that still shows the error. The less unknowns you give us, the less likely we are to have to write this off as user is probably just doing something weird.

bwoodsend avatar Aug 20 '24 10:08 bwoodsend

when the exe was importing the pandas at that time this error was getting triggred

ankushraut008 avatar Aug 20 '24 10:08 ankushraut008

when the exe was importing the pandas at that time this error was getting triggred

So if you build an executable from the following minimal program:

import pandas
print(pandas)

it also triggers the error?

rokm avatar Aug 20 '24 10:08 rokm

['python', '-m', 'PyInstaller', '--distpath', 'D:\Application\arc\DataRunners\ExchangeEODRecorder', '--add-data=D:\Application\arc\DataRunners\ExchangeEODRecorder\ExchangesEODRecorder.json;.', 'D:\Application\arc\DataRunners\ExchangeEODRecorder\ExchangesEODRecorder.py']

this was the basic command i am paassing

ankushraut008 avatar Aug 20 '24 10:08 ankushraut008

when the exe was importing the pandas at that time this error was getting triggred

So if you build an executable from the following minimal program:

import pandas
print(pandas)

it also triggers the error?

NO

ankushraut008 avatar Aug 20 '24 10:08 ankushraut008

What if you add a --clean argument to the PyInstaller command in your build script to force a full clean rebuild of your executable. Does that make any difference?

rokm avatar Aug 20 '24 10:08 rokm

i have used --clean i have also tried with previous versions as well

ankushraut008 avatar Aug 20 '24 10:08 ankushraut008

Then I guess we cannot help you without a complete minimal example that reproduces the problem. Especially as minimal numpy/pandas import programs seem to work for you, but your actual program does not.

rokm avatar Aug 20 '24 11:08 rokm

see the program was written before 6 Months at that time the code was working correctly but after resetting the system the code was not working that's why we have contacted you

ankushraut008 avatar Aug 20 '24 13:08 ankushraut008

That doesn't actually tell us anything.

bwoodsend avatar Aug 20 '24 13:08 bwoodsend

OK

ankushraut008 avatar Aug 20 '24 13:08 ankushraut008

what type of info do you need

ankushraut008 avatar Aug 20 '24 13:08 ankushraut008

Just fill out the issue template.

bwoodsend avatar Aug 20 '24 15:08 bwoodsend

The fastest way to determine what the issue is if you can upload the faulty build (i.e., both the build and dist directory) somewhere so I can look at it. If you cannot do that (proprietary application), then you will need to create a minimal reproducer, so that you can create a faulty build that you can share.

rokm avatar Aug 20 '24 15:08 rokm

Ok

ankushraut008 avatar Aug 21 '24 14:08 ankushraut008

This issue might only occur when running pyinstaller on Windows platforms.

I've encountered the same issue (or perhaps it was a very similar issue) with numpy >=1.26, with pandas >=2, and with confluent_kafka in the past. But I've held off on creating an issue for this as I haven't yet had time to reproduce with minimal programs nor to test who the issues were more likely to lie with.

eickr avatar Aug 21 '24 19:08 eickr

@rokm Waiting for your responce

ankushraut008 avatar Aug 26 '24 07:08 ankushraut008

@rokm Waiting for your responce

I thought I told you to provide a minimal reproducer, or a complete artifact of a faulty build (the build and dist directories). Otherwise, you're on your own.

rokm avatar Aug 26 '24 08:08 rokm

https://drive.google.com/file/d/1ebL5BY5BiR7PL7_Qvs0XP9dhWeiFVtEV/view?usp=sharing

PFA Faulty Build I Have use one file method for provided build

ankushraut008 avatar Aug 26 '24 09:08 ankushraut008

https://drive.google.com/file/d/1ebL5BY5BiR7PL7_Qvs0XP9dhWeiFVtEV/view?usp=sharing

PFA Faulty Build I Have use one file method for provided build

What is the error you get with this build?

If I try to run that main.pkg with bootloader from v6.10.0 that has PKG sideload enabled, I get

[PYI-3552:ERROR] Module object for pyimod02_importers is NULL!
Traceback (most recent call last):
  File "PyInstaller\loader\pyimod02_importers.py", line 22, in <module>
  File "pathlib.py", line 14, in <module>
  File "urllib\parse.py", line 40, in <module>
ModuleNotFoundError: No module named 'ipaddress'

(Which looks like what you had in #8746).

The last version of PyInstaller that imported pathlib in pyimod02_importers on line 22 was v5.13.2.

So if you are indeed using PyInstaller 6.10.0, then you have old version of PyInstaller's bootstrap modules in cache. Forcing clean build with --clean should fix this; or removing D:\Development\ANKUSH\arc\Custom_Build\Config\build\main\localpycs and then rebuilding.

rokm avatar Aug 26 '24 10:08 rokm

PFA Update Link

https://drive.google.com/drive/folders/1W6CfclEOLCxBby1ib0lv7dbuJ2joWxvt?usp=sharing

ankushraut008 avatar Aug 26 '24 10:08 ankushraut008

Hmm, looks like numpy.libs directory is not collected in your build. If I copy it from a test build (made with same python and numpy version), it gets past that point.

Can you run a full re-build (with added --clean flag) and added --log-level DEBUG, and attach the complete build log so that we can see what is going on during the build...

rokm avatar Aug 26 '24 11:08 rokm

Here is the updated files https://drive.google.com/drive/folders/15ufIiRU_9WCUbbnReTQeTBBnlj7n9Kpq?usp=sharing

ankushraut008 avatar Aug 27 '24 12:08 ankushraut008