Can't build pandas using: sudo toolchain pip install pandas

[jcrreis]

Hello, I'm trying to run a Kivy application in Xcode and I need to have pandas in order to run my app, but I can't install this package. I'm using Xcode 12.5 with MacOS BigSur running on a virtual machine. I could install kivymd with this command though, only fails with pandas library and I don't know why.

This is the error I get when I try to install pandas package:

[DEBUG ] Exception in thread background thread for pid 19775: Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 954, in _bootstrap_inner self.run() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/threading.py", line 892, in run self._target(*self._args, **self._kwargs) File "/Users/joaoreis/Documents/venv/lib/python3.9/site-packages/sh.py", line 1683, in wrap fn(*rgs, **kwargs) File "/Users/joaoreis/Documents/venv/lib/python3.9/site-packages/sh.py", line 2662, in background_thread handle_exit_code(exit_code) File "/Users/joaoreis/Documents/venv/lib/python3.9/site-packages/sh.py", line 2349, in fn return self.command.handle_command_exit_code(exit_code) File "/Users/joaoreis/Documents/venv/lib/python3.9/site-packages/sh.py", line 905, in handle_command_exit_code raise exc sh.ErrorReturnCode_1:

RAN: /Users/joaoreis/Documents/dist/hostpython3/bin/pip3 install --isolated --prefix /Users/joaoreis/Documents/dist/root/python3 pandas

STDOUT: Collecting pandas Using cached https://files.pythonhosted.org/packages/e8/81/f7be049fe887865200a0450b137f2c574647b9154503865502cfd720ab5d/pandas-1.2.4.tar.gz Installing build dependencies ... error ERROR: Command errored out with exit status 1: command: /Users/joaoreis/Documents/dist/hostpython3/bin/python3.8 /Users/joaoreis/Documents/dist/hostpython3/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /private/tmp/pip-build-env-jmj_9m12/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel 'Cython>=0.29.21,<3' 'numpy==1.16.5; python_version=='"'"'3.7'"'"' and platform_system!='"'"'AIX'"'"'... (25131 more, please see e.stdout)

STDERR:

Traceback (most recent call last): File "/Users/joaoreis/Documents/venv/bin/toolchain", line 8, in sys.exit(main()) File "/Users/joaoreis/Documents/venv/lib/python3.9/site-packages/kivy_ios/toolchain.py", line 1519, in main ToolchainCL() File "/Users/joaoreis/Documents/venv/lib/python3.9/site-packages/kivy_ios/toolchain.py", line 1276, in init getattr(self, args.command)() File "/Users/joaoreis/Documents/venv/lib/python3.9/site-packages/kivy_ios/toolchain.py", line 1478, in pip _pip(sys.argv[2:]) File "/Users/joaoreis/Documents/venv/lib/python3.9/site-packages/kivy_ios/toolchain.py", line 1171, in _pip shprint(pip_cmd, *args, _env=pip_env) File "/Users/joaoreis/Documents/venv/lib/python3.9/site-packages/kivy_ios/toolchain.py", line 55, in shprint for line in cmd: File "/Users/joaoreis/Documents/venv/lib/python3.9/site-packages/sh.py", line 953, in next self.wait() File "/Users/joaoreis/Documents/venv/lib/python3.9/site-packages/sh.py", line 879, in wait self.handle_command_exit_code(exit_code) File "/Users/joaoreis/Documents/venv/lib/python3.9/site-packages/sh.py", line 905, in handle_command_exit_code raise exc sh.ErrorReturnCode_1:

RAN: /Users/joaoreis/Documents/dist/hostpython3/bin/pip3 install --isolated --prefix /Users/joaoreis/Documents/dist/root/python3 pandas

STDOUT: Collecting pandas Using cached https://files.pythonhosted.org/packages/e8/81/f7be049fe887865200a0450b137f2c574647b9154503865502cfd720ab5d/pandas-1.2.4.tar.gz Installing build dependencies ... error ERROR: Command errored out with exit status 1: command: /Users/joaoreis/Documents/dist/hostpython3/bin/python3.8 /Users/joaoreis/Documents/dist/hostpython3/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /private/tmp/pip-build-env-jmj_9m12/overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- setuptools wheel 'Cython>=0.29.21,<3' 'numpy==1.16.5; python_version=='"'"'3.7'"'"' and platform_system!='"'"'AIX'"'"'... (25131 more, please see e.stdout)

STDERR:

Thank you!

[rirons]

From my scouring the internet it doesn't seem like this will be possible until someone dedicates the time to creating a recipe for pandas but i've seen one post suggesting he was able to get it to work messing with the version of numpy but no such luck for me.please let me know if you find a solution.

[jcrreis]

From my scouring the internet it doesn't seem like this will be possible until someone dedicates the time to creating a recipe for pandas but i've seen one post suggesting he was able to get it to work messing with the version of numpy but no such luck for me.please let me know if you find a solution.

I didn't find a solution, but I looked at python-for-android and they support pandas (they have a recipe for pandas).. So my application only runs on android unfortunately. I think this library should have a tutorial on how recipes actually work, so people can easily build their own recipes. Just a suggestion.

[pythonsus]

can you copy the pandas recipe from python-for-android to the kivy-ios/recipes folder inside the kivy-ios main folder

[subhashsharma]

Hi @misl6 - I am happy to dedicate the time to learn and build Pandas recipe and contribute here. I have already spend two days trying building it using the ./toolchain.py. I have progressed further but stuck here:

[DEBUG   ] creating build/lib.macosx-13.2-x86_64-cpython-310/pandas/_libs/window
[DEBUG   ] /var/folders/_z/2wq_30nj20g7207df29p270r0000gn/T/tmplqqre5bt -arch x86_64 -L/Users/subhash/Development/personal/ios/Kivy/kivy-ios-build/kivy-ios/dist/lib -L/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator16.4.sdk/usr/lib -miphoneos-version-min=9.0 -O3 -miphoneos-version-min=9.0 -I/Users/subhash/Development/personal/ios/Kivy/kivy-ios-build/kivy-ios/dist/include/x86_64/ffi -I/Users/subhash/Development/personal/ios/Kivy/kivy-ios-build/kivy-ios/dist/include/x86_64/openssl -I/Users/subhash/Development/personal/ios/Kivy/kivy-ios-build/kivy-ios/dist/include/x86_64 build/temp.macosx-13.2-x86_64-cpython-310/pandas/_libs/window/aggregations.o -o build/lib.macosx-13.2-x86_64-cpython-310/pandas/_libs/window/aggregations.cpython-310-darwin.so
[DEBUG   ] Undefined symbols for architecture x86_64:
[DEBUG   ]   "_PyBaseObject_Type", referenced from:
[DEBUG   ]       __Pyx_setup_reduce(_object*) in aggregations.o
[DEBUG   ]       __pyx_memoryview_is_slice(__pyx_memoryview_obj*, _object*) in aggregations.o
[DEBUG   ]       __pyx_memoryview_setitem_slice_assignment(__pyx_memoryview_obj*, _object*, _object*) in aggregations.o
[DEBUG   ]       __pyx_tp_new_array(_typeobject*, _object*, _object*) in aggregations.o
[DEBUG   ]       __Pyx__ArgTypeTest(_object*, _typeobject*, char const*, int) in aggregations.o
[DEBUG   ]       __Pyx_PyObject_GetAttrStr_ClearAttributeError() in aggregations.o
[DEBUG   ]       __Pyx_inner_PyErr_GivenExceptionMatches2(_object*, _object*, _object*) in aggregations.o
[DEBUG   ]       ...

And then the error is:

  RAN: /Users/subhash/Development/personal/ios/Kivy/kivy-ios-build/kivy-ios/dist/hostpython3/bin/python setup.py build_ext --inplace --force

  STDOUT:
/Users/subhash/Development/personal/ios/Kivy/kivy-ios-build/kivy-ios/dist/hostpython3/lib/python3.10/site-packages/setuptools/config/pyprojecttoml.py:108: _BetaConfiguration: Support for `[tool.setuptools]` in `pyproject.toml` is still *beta*.
  warnings.warn(msg, _BetaConfiguration)
running build_ext
building 'pandas._libs.algos' extension
creating build/temp.macosx-13.2-x86_64-cpython-310
creating build/temp.macosx-13.2-x86_64-cpython-310/pandas
creating build/temp.macosx-13.2-x86_64-cpython-310/pandas/_libs
/var/folders/_z/2wq_30nj20g7207df29p270r0000gn/T/tmpij7qiwit -O3 -miphoneos-version-min=9.0 -I/Users/subhash/Development/personal/ios/Kivy/kivy-ios-build/kivy-ios/dist/include/x86_64/ffi -I/Users/subhash/Development/personal/ios/Kivy/... (169109 more, please see e.stdout)

  STDERR:

I have tried various things including building the pandas standalone using the Pandas way (not using toolchain), and I can build that but using the Kivy-ios Recipe I didn't have any luck. See the Recipe I am using on Mac M1 one:

from kivy_ios.toolchain import CythonRecipe
from os.path import join
import sh
import shutil


class PandasRecipe(CythonRecipe):
    custom_dir = '/Users/subhash/Development/personal/ios/Kivy/kivy-ios-build/python-packages/pandas-2.0.3'
    depends = ['numpy']
    python_depends = ['python-dateutil', 'pytz']
    library = "libpandas.a"
    patches = ['fix_numpy_includes.patch']
    
    hostpython_prerequisites = ["Cython"]
    cythonize = False
   
    def get_recipe_env(self, arch):
        env = super().get_recipe_env(arch)
        # we need the includes from our installed numpy at site packages
        # because we need some includes generated at numpy's compile time
        print("Print Arch")
        print(arch)
        
        env['NUMPY_INCLUDES'] = '/Users/subhash/Development/personal/ios/Kivy/kivy-ios-build/kivy-io-38/kivy-ios-master/kivy_ios/dist/include/common/numpy/numpy'
        return env
    
recipe = PandasRecipe()

I need this package, please help :)

[misl6]

Hi @subhashsharma !

We have a #dev channel on Discord where you can ask for help about "Developing Kivy project itself"

[pythonsus]

try this fork: https://github.com/pythonsus/kivy-ios/tree/master

[subhashsharma]

@pythonsus thanks for making the fork and putting a recipe together :) I have been trying to build it but for some reason it's going to my conda env during the build step rather than staying in the virtual env which I created to do this exercise. See here:

$ python ./toolchain.py build pandas
[INFO    ] Building with 10 processes, where supported
[INFO    ] Want to build ['pandas']
[INFO    ] Using the bundled version for recipe 'pandas'
[INFO    ] Loaded recipe pandas (depends of ['cython', 'numpy', 'libbz2', 'liblzma'], optional are [])
[INFO    ] Using the bundled version for recipe 'cython'
Traceback (most recent call last):
  File "/Users/subhash/Development/personal/ios/Kivy/kivy-ios-build/kivy-ios-pandas/./toolchain.py", line 3, in <module>
    main()
  File "/Users/subhash/Development/personal/ios/Kivy/kivy-ios-build/kivy-ios-pandas/kivy_ios/toolchain.py", line 1609, in main
    ToolchainCL()
  File "/Users/subhash/Development/personal/ios/Kivy/kivy-ios-build/kivy-ios-pandas/kivy_ios/toolchain.py", line 1353, in __init__
    getattr(self, args.command)()
  File "/Users/subhash/Development/personal/ios/Kivy/kivy-ios-build/kivy-ios-pandas/kivy_ios/toolchain.py", line 1422, in build
    build_recipes(args.recipe, ctx)
  File "/Users/subhash/Development/personal/ios/Kivy/kivy-ios-build/kivy-ios-pandas/kivy_ios/toolchain.py", line 1157, in build_recipes
    recipe = Recipe.get_recipe(name, ctx)
  File "/Users/subhash/Development/personal/ios/Kivy/kivy-ios-build/kivy-ios-pandas/kivy_ios/toolchain.py", line 990, in get_recipe
    mod = importlib.import_module(f"kivy_ios.recipes.{name}")
  File "/Users/subhash/opt/anaconda3/envs/kivy/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'kivy_ios.recipes.cython'

I am trying to workout that. If you know any quick reason then please share. If not then I will come back with the updates after I fix this intermediate issue. Thanks Again!!