python-for-android icon indicating copy to clipboard operation
python-for-android copied to clipboard

p4a Doesn't Pick Up Latest Installed Java Command Line Tools, Resulting in avdmanager exception

Open cedarbob opened this issue 3 years ago • 13 comments
trafficstars

Versions

  • Python: 3.9.7
  • OS: Ubuntu 20.04
  • Kivy: 2.0
  • Cython: 0.29.25
  • OpenJDK: 11.0.13

Description

I'm trying to build an apk with p4a for the first time. Using IntellijIdea, I installed the latest sdk command line tools. But p4a used the avdmanager in ../Sdk/tools/bin, instead of the new one which was installed in the default location .../Sdk/cmdline-tools/latest/bin. The avdmanager then threw a ClassNotFoundException for javax.xml.bind.annotation.XmlSchema.

Logs

$ p4a apk --private $HOME/Test/Python/kivydemo-for-android/showcase --package=com.rhrapps.showcase --name "Kivy Showcase" --version 0.1 --bootstrap=sdl2 --requirements=Python,kivy
[WARNING]: $ANDROIDNDKVER is deprecated and no longer necessary, the value you set is ignored
[INFO]:    Will compile for the following archs: armeabi-v7a
[INFO]:    Found Android API target in $ANDROIDAPI: 32
Traceback (most recent call last):
  File "/home/rhr/Tools/miniconda3/envs/kivy_env/bin/p4a", line 8, in <module>
    sys.exit(main())
  File "/home/rhr/Tools/miniconda3/envs/kivy_env/lib/python3.9/site-packages/pythonforandroid/entrypoints.py", line 18, in main
    ToolchainCL()
  File "/home/rhr/Tools/miniconda3/envs/kivy_env/lib/python3.9/site-packages/pythonforandroid/toolchain.py", line 725, in __init__
    getattr(self, command)(args)
  File "/home/rhr/Tools/miniconda3/envs/kivy_env/lib/python3.9/site-packages/pythonforandroid/toolchain.py", line 143, in wrapper_func
    ctx.prepare_build_environment(user_sdk_dir=self.sdk_dir,
  File "/home/rhr/Tools/miniconda3/envs/kivy_env/lib/python3.9/site-packages/pythonforandroid/build.py", line 300, in prepare_build_environment
    apis = get_available_apis(self.sdk_dir)
  File "/home/rhr/Tools/miniconda3/envs/kivy_env/lib/python3.9/site-packages/pythonforandroid/build.py", line 74, in get_available_apis
    targets = get_targets(sdk_dir)
  File "/home/rhr/Tools/miniconda3/envs/kivy_env/lib/python3.9/site-packages/pythonforandroid/build.py", line 62, in get_targets
    targets = avdmanager('list', 'target').stdout.decode('utf-8').split('\n')
  File "/home/rhr/Tools/miniconda3/envs/kivy_env/lib/python3.9/site-packages/sh.py", line 1566, in __call__
    return RunningCommand(cmd, call_args, stdin, stdout, stderr)
  File "/home/rhr/Tools/miniconda3/envs/kivy_env/lib/python3.9/site-packages/sh.py", line 822, in __init__
    self.wait()
  File "/home/rhr/Tools/miniconda3/envs/kivy_env/lib/python3.9/site-packages/sh.py", line 879, in wait
    self.handle_command_exit_code(exit_code)
  File "/home/rhr/Tools/miniconda3/envs/kivy_env/lib/python3.9/site-packages/sh.py", line 905, in handle_command_exit_code
    raise exc
sh.ErrorReturnCode_1: 

  RAN: /home/rhr/Android/Sdk/tools/bin/avdmanager list target

  STDOUT:


  STDERR:
Exception in thread "main" java.lang.NoClassDefFoundError: javax/xml/bind/annotation/XmlSchema
	at com.android.repository.api.SchemaModule$SchemaModuleVersion.<init>(SchemaModule.java:156)
	at com.android.repository.api.SchemaModule.<init>(SchemaModule.java:75)
	at com.android.sdklib.repository.AndroidSdkHandler.<clinit>(AndroidSdkHandler.java:81)
	at com.android.sdklib.tool.AvdManagerCli.run(AvdManagerCli.java:213)
	at com.android.sdklib.tool.AvdManagerCli.main(AvdManagerCli.java:200)
Caused by: java.lang.ClassNotFoundException: javax.xml.bind.annotation.XmlSchema
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	... 5 more

cedarbob avatar Jan 16 '22 22:01 cedarbob

Yeah, recently tools became cmdline-tools.

In buildozer we're still relying on an old version of the cmdline tools, which are located at tools, but I agree that should be updated to support the newest ones (or at least we should support both locations)

misl6 avatar Apr 06 '22 20:04 misl6

I have this same problem, I've followed instructions at:

https://stackoverflow.com/a/67413427 and https://python-for-android.readthedocs.io/en/latest/quickstart/#installing-android-sdk

Then when building I get:

~/Documents/codings/kivy> p4a apk --private $HOME/Documents/codings/kivy/ --package=org.example.myapp --name "My application" --version 0.1 --requirements=python3,kivy --arch=armeabi-v7a
[WARNING]: $ANDROIDNDKVER is deprecated and no longer necessary, the value you set is ignored
[INFO]:    Will compile for the following archs: armeabi-v7a
[INFO]:    Found Android API target in $ANDROIDAPI: 27
Traceback (most recent call last):
  File "/usr/bin/p4a", line 8, in <module>
    sys.exit(main())
  File "/usr/lib/python3.10/site-packages/pythonforandroid/entrypoints.py", line 18, in main
    ToolchainCL()
  File "/usr/lib/python3.10/site-packages/pythonforandroid/toolchain.py", line 728, in __init__
    getattr(self, command)(args)
  File "/usr/lib/python3.10/site-packages/pythonforandroid/toolchain.py", line 141, in wrapper_func
    ctx.prepare_build_environment(user_sdk_dir=self.sdk_dir,
  File "/usr/lib/python3.10/site-packages/pythonforandroid/build.py", line 358, in prepare_build_environment
    apis = get_available_apis(self.sdk_dir)
  File "/usr/lib/python3.10/site-packages/pythonforandroid/build.py", line 130, in get_available_apis
    targets = get_targets(sdk_dir)
  File "/usr/lib/python3.10/site-packages/pythonforandroid/build.py", line 118, in get_targets
    targets = avdmanager('list', 'target').stdout.decode('utf-8').split('\n')
  File "/usr/lib/python3.10/site-packages/sh.py", line 1566, in __call__
    return RunningCommand(cmd, call_args, stdin, stdout, stderr)
  File "/usr/lib/python3.10/site-packages/sh.py", line 822, in __init__
    self.wait()
  File "/usr/lib/python3.10/site-packages/sh.py", line 879, in wait
    self.handle_command_exit_code(exit_code)
  File "/usr/lib/python3.10/site-packages/sh.py", line 905, in handle_command_exit_code
    raise exc
sh.ErrorReturnCode_1: 

  RAN: /opt/android-sdk/tools/bin/avdmanager list target

  STDOUT:


  STDERR:
Exception in thread "main" java.lang.NoClassDefFoundError: javax/xml/bind/annotation/XmlSchema
	at com.android.repository.api.SchemaModule$SchemaModuleVersion.<init>(SchemaModule.java:156)
	at com.android.repository.api.SchemaModule.<init>(SchemaModule.java:75)
	at com.android.sdklib.repository.AndroidSdkHandler.<clinit>(AndroidSdkHandler.java:81)
	at com.android.sdklib.tool.AvdManagerCli.run(AvdManagerCli.java:213)
	at com.android.sdklib.tool.AvdManagerCli.main(AvdManagerCli.java:200)
Caused by: java.lang.ClassNotFoundException: javax.xml.bind.annotation.XmlSchema
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	... 5 more

I don't know about you, but to me this sounds very acute, because I don't understand any way around.

mavavilj avatar Apr 10 '22 15:04 mavavilj

I can proceed by editing /usr/lib/python3.10/site-packages/pythonforandroid/build.py lines 116 and 117 to:

    if exists(join(sdk_dir, 'cmdline-tools', 'bin', 'avdmanager')):
        avdmanager = sh.Command(join(sdk_dir, 'cmdline-tools', 'bin', 'avdmanager'))

But now the error becomes:

p4a apk --private $HOME/Documents/codings/kivy/ --package=org.example.myapp --name "My application" --version 0.1 --requirements=python3,kivy --arch=armeabi-v7a
[WARNING]: $ANDROIDNDKVER is deprecated and no longer necessary, the value you set is ignored
[INFO]:    Will compile for the following archs: armeabi-v7a
[INFO]:    Found Android API target in $ANDROIDAPI: 27
Traceback (most recent call last):
  File "/usr/bin/p4a", line 8, in <module>
    sys.exit(main())
  File "/usr/lib/python3.10/site-packages/pythonforandroid/entrypoints.py", line 18, in main
    ToolchainCL()
  File "/usr/lib/python3.10/site-packages/pythonforandroid/toolchain.py", line 728, in __init__
    getattr(self, command)(args)
  File "/usr/lib/python3.10/site-packages/pythonforandroid/toolchain.py", line 141, in wrapper_func
    ctx.prepare_build_environment(user_sdk_dir=self.sdk_dir,
  File "/usr/lib/python3.10/site-packages/pythonforandroid/build.py", line 358, in prepare_build_environment
    apis = get_available_apis(self.sdk_dir)
  File "/usr/lib/python3.10/site-packages/pythonforandroid/build.py", line 130, in get_available_apis
    targets = get_targets(sdk_dir)
  File "/usr/lib/python3.10/site-packages/pythonforandroid/build.py", line 121, in get_targets
    targets = android('list').stdout.decode('utf-8').split('\n')
  File "/usr/lib/python3.10/site-packages/sh.py", line 1566, in __call__
    return RunningCommand(cmd, call_args, stdin, stdout, stderr)
  File "/usr/lib/python3.10/site-packages/sh.py", line 822, in __init__
    self.wait()
  File "/usr/lib/python3.10/site-packages/sh.py", line 879, in wait
    self.handle_command_exit_code(exit_code)
  File "/usr/lib/python3.10/site-packages/sh.py", line 905, in handle_command_exit_code
    raise exc
sh.ErrorReturnCode_2: 

  RAN: /opt/android-sdk/tools/android list

  STDOUT:
*************************************************************************
The "android" command is deprecated.
For manual SDK, AVD, and project management, please use Android Studio.
For command-line tools, use tools/bin/sdkmanager and tools/bin/avdmanager
*************************************************************************
Invalid or unsupported command "list"

Supported commands are:
android list target
android list avd
android list device
android create avd
android move avd
android delete avd
android list sdk
android update sdk


  STDERR:

mavavilj avatar Apr 10 '22 15:04 mavavilj

As a workaround, until we update python-for-android to support the latest Android command-line-tools version, please rely on the following links:

macOS: https://dl.google.com/android/repository/commandlinetools-mac-6514223_latest.zip Linux: https://dl.google.com/android/repository/commandlinetools-linux-6514223_latest.zip

misl6 avatar Apr 10 '22 16:04 misl6

I can also proceed from the previous edit of /usr/lib/python3.10/site-packages/pythonforandroid/build.py lines 116 and 117 to:

    if exists(join(sdk_dir, 'cmdline-tools', 'bin', 'avdmanager')):
        avdmanager = sh.Command(join(sdk_dir, 'cmdline-tools', 'bin', 'avdmanager'))
        targets = avdmanager('list', 'target').stdout.decode('utf-8').split('\n')
    elif exists(join(sdk_dir, 'cmdline-tools/latest/bin', 'sdkmanager')):
        android = sh.Command(join(sdk_dir, 'cmdline-tools/latest/bin', 'sdkmanager'))
        targets = android('--list').stdout.decode('utf-8').split('\n')

Then the error becomes:

p4a apk --private $HOME/Documents/codings/kivy/ --package=org.example.myapp --name "My application" --version 0.1 --requirements=python3,kivy --arch=arm64-v8a
[WARNING]: $ANDROIDNDKVER is deprecated and no longer necessary, the value you set is ignored
[INFO]:    Will compile for the following archs: arm64-v8a
[INFO]:    Found Android API target in $ANDROIDAPI: 27
[INFO]:    Available Android APIs are ()
[ERROR]:   Build failed: Requested API target 27 is not available, install it with the SDK android tool.

mavavilj avatar Apr 10 '22 16:04 mavavilj

Did you installed the requested API? See: https://python-for-android.readthedocs.io/en/latest/quickstart/#platform-and-build-tools

misl6 avatar Apr 10 '22 16:04 misl6

Yes, but by using /opt/android-sdk/cmdline-tools/latest/bin/sdkmanager as given in:

https://stackoverflow.com/questions/65262340/cmdline-tools-could-not-determine-sdk-root/67413427#67413427

my ~/.bashrc:

# Adjust the paths!
export ANDROIDSDK="/opt/android-sdk"
export ANDROIDNDK="$HOME/Developer/android-ndk-r23b"
export ANDROIDAPI="27"  # Target API version of your application
export NDKAPI="21"  # Minimum supported API version of your application
export ANDROIDNDKVER="r23b"  # Version of the NDK you installed

The installation created a new /tools/ for example, but:

/opt/android-sdk/tools/bin/sdkmanager
Exception in thread "main" java.lang.NoClassDefFoundError: javax/xml/bind/annotation/XmlSchema
	at com.android.repository.api.SchemaModule$SchemaModuleVersion.<init>(SchemaModule.java:156)
	at com.android.repository.api.SchemaModule.<init>(SchemaModule.java:75)
	at com.android.sdklib.repository.AndroidSdkHandler.<clinit>(AndroidSdkHandler.java:81)
	at com.android.sdklib.tool.sdkmanager.SdkManagerCli.main(SdkManagerCli.java:73)
	at com.android.sdklib.tool.sdkmanager.SdkManagerCli.main(SdkManagerCli.java:48)
Caused by: java.lang.ClassNotFoundException: javax.xml.bind.annotation.XmlSchema
	at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:581)
	at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
	at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:522)
	... 5 more

At line 358 of /usr/lib/python3.10/site-packages/pythonforandroid/build.py we find:

apis = get_available_apis(self.sdk_dir)

The self.sdk_dir prints to /opt/android-sdk, which indeed should be the sdk_root.

mavavilj avatar Apr 10 '22 16:04 mavavilj

The reason it doesn't return any available APIs is caused by:

apis = [s for s in targets if re.match(r'^ *API level: ', s)]

at line 131

resulting to apis = [].

Even when the line before it

targets = get_targets(sdk_dir)

has the full output of /opt/android-sdk/cmdline-tools/latest/bin/sdkmanager --list. This list actually does contain:

system-images;android-27;default;arm64-v8a | 1 | ARM 64 v8a System Image

mavavilj avatar Apr 10 '22 17:04 mavavilj

I can proceed by commenting out the lines that check for the API (358-367):

        """
        apis = get_available_apis(self.sdk_dir)
        info('Available Android APIs are ({})'.format(
            ', '.join(map(str, apis))))
        if android_api in apis:
            info(('Requested API target {} is available, '
                  'continuing.').format(android_api))
        else:
            raise BuildInterruptingException(
                ('Requested API target {} is not available, install '
                 'it with the SDK android tool.').format(android_api))
        """

Now the error becomes:

p4a apk --private $HOME/Documents/codings/kivy/ --package=org.example.myapp --name "My application" --version 0.1 --requirements=python3,kivy --arch=arm64-v8a
[INFO]:    Will compile for the following archs: arm64-v8a
[INFO]:    Found Android API target in $ANDROIDAPI: 27
[INFO]:    Found NDK dir in $ANDROIDNDK: /home/myuser/Developer/android-ndk-r23b
[INFO]:    Found NDK version 23b
[WARNING]: Maximum recommended NDK version is 19c, but newer versions may work.
[WARNING]: Newer NDKs may not be fully supported by p4a.
[INFO]:    Found Android API target in $NDKAPI
[INFO]:    ccache is missing, the build will not be optimized in the future.
/usr/bin/python3: No module named cython
[WARNING]: Cython for python3 missing. If you are building for  a python 3 target (which is the default) then THINGS WILL BREAK.
[WARNING]: Could not find any toolchain for aarch64-linux-android!
Traceback (most recent call last):
  File "/usr/bin/p4a", line 8, in <module>
    sys.exit(main())
  File "/usr/lib/python3.10/site-packages/pythonforandroid/entrypoints.py", line 18, in main
    ToolchainCL()
  File "/usr/lib/python3.10/site-packages/pythonforandroid/toolchain.py", line 728, in __init__
    getattr(self, command)(args)
  File "/usr/lib/python3.10/site-packages/pythonforandroid/toolchain.py", line 141, in wrapper_func
    ctx.prepare_build_environment(user_sdk_dir=self.sdk_dir,
  File "/usr/lib/python3.10/site-packages/pythonforandroid/build.py", line 447, in prepare_build_environment
    self.toolchain_version = select_and_check_toolchain_version(
  File "/usr/lib/python3.10/site-packages/pythonforandroid/build.py", line 90, in select_and_check_toolchain_version
    toolchain_version=toolchain_version,
UnboundLocalError: local variable 'toolchain_version' referenced before assignment


mavavilj avatar Apr 10 '22 17:04 mavavilj

I can proceed by modifying line 430 to use the python3 that I really use:

"python3.10", "-m", "cython", "--help",

Now error becomes:

p4a apk --private $HOME/Documents/codings/kivy/ --package=org.example.myapp --name "My application" --version 0.1 --requirements=python3,kivy --arch=arm64-v8a
[INFO]:    Will compile for the following archs: arm64-v8a
[INFO]:    Found Android API target in $ANDROIDAPI: 27
[INFO]:    Found NDK dir in $ANDROIDNDK: /home/myuser/Developer/android-ndk-r23b
[INFO]:    Found NDK version 23b
[WARNING]: Maximum recommended NDK version is 19c, but newer versions may work.
[WARNING]: Newer NDKs may not be fully supported by p4a.
[INFO]:    Found Android API target in $NDKAPI
[INFO]:    ccache is missing, the build will not be optimized in the future.
[WARNING]: Could not find any toolchain for aarch64-linux-android!
Traceback (most recent call last):
  File "/usr/bin/p4a", line 8, in <module>
    sys.exit(main())
  File "/usr/lib/python3.10/site-packages/pythonforandroid/entrypoints.py", line 18, in main
    ToolchainCL()
  File "/usr/lib/python3.10/site-packages/pythonforandroid/toolchain.py", line 728, in __init__
    getattr(self, command)(args)
  File "/usr/lib/python3.10/site-packages/pythonforandroid/toolchain.py", line 141, in wrapper_func
    ctx.prepare_build_environment(user_sdk_dir=self.sdk_dir,
  File "/usr/lib/python3.10/site-packages/pythonforandroid/build.py", line 447, in prepare_build_environment
    self.toolchain_version = select_and_check_toolchain_version(
  File "/usr/lib/python3.10/site-packages/pythonforandroid/build.py", line 90, in select_and_check_toolchain_version
    toolchain_version=toolchain_version,
UnboundLocalError: local variable 'toolchain_version' referenced before assignment

mavavilj avatar Apr 10 '22 17:04 mavavilj

Finally,

changing to android-ndk-r19c, rather than android-ndk-r23b, the build starts and goes on for quite a long time until I hit something which looks like a python3 name mismatch again:

[INFO]:    Building pyjnius for arm64-v8a
[INFO]:    jnius apparently isn't already in site-packages
[INFO]:    Cythonizing anything necessary in pyjnius
[INFO]:    -> directory context /home/myuser/.local/share/python-for-android/build/other_builds/pyjnius-sdl2/arm64-v8a__ndk_target_21/pyjnius
[INFO]:    -> running python3 -c import sys; print(sys.path)
[INFO]:    Trying first build of pyjnius to get cython files: this is expected to fail                                       
[INFO]:    -> running python3 setup.py build_ext -v
           working: subprocess.CalledProcessError: Command '['which', 'javac']' returned non-zero exit status 1.             Exception in thread background thread for pid 10702:
Traceback (most recent call last):
  File "/usr/lib64/python3.10/threading.py", line 1009, in _bootstrap_inner
    self.run()
  File "/usr/lib64/python3.10/threading.py", line 946, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.10/site-packages/sh.py", line 1683, in wrap
    fn(*rgs, **kwargs)
  File "/usr/lib/python3.10/site-packages/sh.py", line 2662, in background_thread
    handle_exit_code(exit_code)
  File "/usr/lib/python3.10/site-packages/sh.py", line 2349, in fn
    return self.command.handle_command_exit_code(exit_code)
  File "/usr/lib/python3.10/site-packages/sh.py", line 905, in handle_command_exit_code
    raise exc
sh.ErrorReturnCode_1: 

  RAN: /home/myuser/.local/share/python-for-android/build/other_builds/hostpython3/desktop/hostpython3/native-build/python3 setup.py build_ext -v

  STDOUT:
which: no javac in (/home/myuser/Developer/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/bin:/home/myuser/Developer/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/bin:/home/myuser/Developer/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/bin:/home/myuser/Developer/android-ndk-r19c/toolchains/llvm/prebuilt/linux-x86_64/bin:/home/myuser/Developer/android-ndk-r19c/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86/bin/:/home/myuser/Developer/android-ndk-r19c/toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/:/home/myuser/Developer/android-ndk-r19c:/opt/android-sdk/tools:/home/myuser/.local/bin:/home/myuser/bin:/usr/local/bin:/usr/bin:/bin:/snap/bin)
Traceback (most recent call last):
  File "setup.py", line 95, in <module>
    compile_native_invocation_handler(*get_possible_homes(PLATFORM))
  File "jnius/env.py", line 172, in get_possible_homes
    get_jdk_home(platform),
  File "jnius/env.py", line 136, in get_jdk_home
    check_output(
  File "/home/myuser/.local/share/python-for-android/build/other_builds/hostpython3/desktop/hostpython3/Lib/subprocess.py", line 415, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/home/myuser/.local/share/python-for-android/build/other_builds/hostpython3/desktop/hostpython3/Lib/subprocess.py", line 516, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['which', 'javac']' returned non-zero exit status 1.


  STDERR:

                                                                                                                             
[INFO]:    pyjnius first build failed (as expected)
[INFO]:    Running cython where appropriate
[INFO]:    Cythonize jnius/jnius.pyx
[INFO]:    -> running python3 -cimport sys; from Cython.Compiler.Main import setuptools_main; sys....(and 42 more)
           working: ModuleNotFoundError: No module named 'Cython'                                                            Exception in thread background thread for pid 10710:
Traceback (most recent call last):
  File "/usr/lib64/python3.10/threading.py", line 1009, in _bootstrap_inner
    self.run()
  File "/usr/lib64/python3.10/threading.py", line 946, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.10/site-packages/sh.py", line 1683, in wrap
    fn(*rgs, **kwargs)
  File "/usr/lib/python3.10/site-packages/sh.py", line 2662, in background_thread
    handle_exit_code(exit_code)
  File "/usr/lib/python3.10/site-packages/sh.py", line 2349, in fn
    return self.command.handle_command_exit_code(exit_code)
  File "/usr/lib/python3.10/site-packages/sh.py", line 905, in handle_command_exit_code
    raise exc
sh.ErrorReturnCode_1: 

  RAN: /usr/bin/python3 '-cimport sys; from Cython.Compiler.Main import setuptools_main; sys.exit(setuptools_main());' ./jnius/jnius.pyx

  STDOUT:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'Cython'


  STDERR:

Traceback (most recent call last):                                                                                           
  File "/usr/bin/p4a", line 8, in <module>
    sys.exit(main())
  File "/usr/lib/python3.10/site-packages/pythonforandroid/entrypoints.py", line 18, in main
    ToolchainCL()
  File "/usr/lib/python3.10/site-packages/pythonforandroid/toolchain.py", line 728, in __init__
    getattr(self, command)(args)
  File "/usr/lib/python3.10/site-packages/pythonforandroid/toolchain.py", line 151, in wrapper_func
    build_dist_from_args(ctx, dist, args)
  File "/usr/lib/python3.10/site-packages/pythonforandroid/toolchain.py", line 210, in build_dist_from_args
    build_recipes(build_order, python_modules, ctx,
  File "/usr/lib/python3.10/site-packages/pythonforandroid/build.py", line 597, in build_recipes
    recipe.build_arch(arch)
  File "/usr/lib/python3.10/site-packages/pythonforandroid/recipe.py", line 1059, in build_arch
    self.build_cython_components(arch)
  File "/usr/lib/python3.10/site-packages/pythonforandroid/recipe.py", line 1084, in build_cython_components
    self.cythonize_build(env=env)
  File "/usr/lib/python3.10/site-packages/pythonforandroid/recipe.py", line 1132, in cythonize_build
    self.cythonize_file(env, build_dir, join(root, filename))
  File "/usr/lib/python3.10/site-packages/pythonforandroid/recipe.py", line 1121, in cythonize_file
    shprint(python_command, "-c"
  File "/usr/lib/python3.10/site-packages/pythonforandroid/logger.py", line 167, in shprint
    for line in output:
  File "/usr/lib/python3.10/site-packages/sh.py", line 953, in next
    self.wait()
  File "/usr/lib/python3.10/site-packages/sh.py", line 879, in wait
    self.handle_command_exit_code(exit_code)
  File "/usr/lib/python3.10/site-packages/sh.py", line 905, in handle_command_exit_code
    raise exc
sh.ErrorReturnCode_1: 

  RAN: /usr/bin/python3 '-cimport sys; from Cython.Compiler.Main import setuptools_main; sys.exit(setuptools_main());' ./jnius/jnius.pyx

  STDOUT:
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'Cython'


  STDERR:

mavavilj avatar Apr 10 '22 17:04 mavavilj

Modifying line 1118 of recipe.py to:

python_command = sh.Command("python3.10")

ends up to:

[INFO]:    pyjnius first build failed (as expected)
[INFO]:    Running cython where appropriate
[INFO]:    Cythonize jnius/jnius.pyx
[INFO]:    -> running python3.10 -cimport sys; from ...(and 91 more)
           working: jnius/jnius.pyx:113:3: Compile-time name '...(and 26 more) Exception in thread background thread for pid 15352:
Traceback (most recent call last):
  File "/usr/lib64/python3.10/threading.py", line 1009, in _bootstrap_inner
    self.run()
  File "/usr/lib64/python3.10/threading.py", line 946, in run
    self._target(*self._args, **self._kwargs)
  File "/usr/lib/python3.10/site-packages/sh.py", line 1683, in wrap
    fn(*rgs, **kwargs)
  File "/usr/lib/python3.10/site-packages/sh.py", line 2662, in background_thread
    handle_exit_code(exit_code)
  File "/usr/lib/python3.10/site-packages/sh.py", line 2349, in fn
    return self.command.handle_command_exit_code(exit_code)
  File "/usr/lib/python3.10/site-packages/sh.py", line 905, in handle_command_exit_code
    raise exc
sh.ErrorReturnCode_1: 

  RAN: /usr/bin/python3.10 '-cimport sys; from Cython.Compiler.Main import setuptools_main; sys.exit(setuptools_main());' ./jnius/jnius.pyx

  STDOUT:
/home/myuser/.local/lib/python3.10/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /home/myuser/.local/share/python-for-android/build/other_builds/pyjnius-sdl2/arm64-v8a__ndk_target_21/pyjnius/jnius/jnius.pyx
  tree = Parsing.p_module(s, pxd, full_module_name)

Error compiling Cython file:
------------------------------------------------------------
...
import sys
import traceback

include "jnius_compat.pxi"
include "jni.pxi"
include "config.pxi"
^
------------------------------------------------------------

jnius/jnius.pyx:100:0: 'config.pxi' not found

Error compiling Cython file:
------------------------------------------------------------
...

include "jnius_compat.pxi"
include "jni.pxi"
include "config.pxi"

IF JNIUS_PLATFORM == "android":
  ^
------------------------------------------------------------

jnius/jnius.pyx:102:3: Compile-time name 'JNIUS_PLATFORM' not defined

Error compiling Cython file:
------------------------------------------------------------
...
include "jni.pxi"
include "config.pxi"

IF JNIUS_PLATFORM == "android":
    include "jnius_jvm_android.pxi"
ELIF JNIUS_PLATFORM == "win32":
    ^
------------------------------------------------------------

jnius/jnius.pyx:104:5: Compile-time name 'JNIUS_PLATFORM' not defined

Error compiling Cython file:
------------------------------------------------------------
...
include "config.pxi"
^
------------------------------------------------------------

jnius/jnius_jvm_dlopen.pxi:1:0: 'config.pxi' not found

Error compiling Cython file:
------------------------------------------------------------
...
    JAVA_HOME = os.getenv('JAVA_HOME') or find_java_home()
    if JAVA_HOME is None or JAVA_HOME == '':
        raise SystemError("JAVA_HOME is not set, and unable to guess JAVA_HOME")
    cdef str JNIUS_LIB_SUFFIX = get_jnius_lib_location(JNIUS_PLATFORM)

    IF JNIUS_PYTHON3:
      ^
------------------------------------------------------------

jnius/jnius_jvm_dlopen.pxi:89:7: Compile-time name 'JNIUS_PYTHON3' not defined

Error compiling Cython file:
------------------------------------------------------------
...

include "jnius_env.pxi"
include "jnius_utils.pxi"
include "jnius_conversion.pxi"
include "jnius_localref.pxi"
IF JNIUS_PYTHON3:
  ^
------------------------------------------------------------

jnius/jnius.pyx:113:3: Compile-time name 'JNIUS_PYTHON3' not defined



  STDERR:

Traceback (most recent call last):                                             
  File "/usr/bin/p4a", line 8, in <module>
    sys.exit(main())
  File "/usr/lib/python3.10/site-packages/pythonforandroid/entrypoints.py", line 18, in main
    ToolchainCL()
  File "/usr/lib/python3.10/site-packages/pythonforandroid/toolchain.py", line 728, in __init__
    getattr(self, command)(args)
  File "/usr/lib/python3.10/site-packages/pythonforandroid/toolchain.py", line 151, in wrapper_func
    build_dist_from_args(ctx, dist, args)
  File "/usr/lib/python3.10/site-packages/pythonforandroid/toolchain.py", line 210, in build_dist_from_args
    build_recipes(build_order, python_modules, ctx,
  File "/usr/lib/python3.10/site-packages/pythonforandroid/build.py", line 597, in build_recipes
    recipe.build_arch(arch)
  File "/usr/lib/python3.10/site-packages/pythonforandroid/recipe.py", line 1059, in build_arch
    self.build_cython_components(arch)
  File "/usr/lib/python3.10/site-packages/pythonforandroid/recipe.py", line 1084, in build_cython_components
    self.cythonize_build(env=env)
  File "/usr/lib/python3.10/site-packages/pythonforandroid/recipe.py", line 1130, in cythonize_build
    self.cythonize_file(env, build_dir, join(root, filename))
  File "/usr/lib/python3.10/site-packages/pythonforandroid/recipe.py", line 1119, in cythonize_file
    shprint(python_command, "-c"
  File "/usr/lib/python3.10/site-packages/pythonforandroid/logger.py", line 167, in shprint
    for line in output:
  File "/usr/lib/python3.10/site-packages/sh.py", line 953, in next
    self.wait()
  File "/usr/lib/python3.10/site-packages/sh.py", line 879, in wait
    self.handle_command_exit_code(exit_code)
  File "/usr/lib/python3.10/site-packages/sh.py", line 905, in handle_command_exit_code
    raise exc
sh.ErrorReturnCode_1: 

  RAN: /usr/bin/python3.10 '-cimport sys; from Cython.Compiler.Main import setuptools_main; sys.exit(setuptools_main());' ./jnius/jnius.pyx

  STDOUT:
/home/myuser/.local/lib/python3.10/site-packages/Cython/Compiler/Main.py:369: FutureWarning: Cython directive 'language_level' not set, using 2 for now (Py2). This will change in a later release! File: /home/myuser/.local/share/python-for-android/build/other_builds/pyjnius-sdl2/arm64-v8a__ndk_target_21/pyjnius/jnius/jnius.pyx
  tree = Parsing.p_module(s, pxd, full_module_name)

Error compiling Cython file:
------------------------------------------------------------
...
import sys
import traceback

include "jnius_compat.pxi"
include "jni.pxi"
include "config.pxi"
^
------------------------------------------------------------

jnius/jnius.pyx:100:0: 'config.pxi' not found

Error compiling Cython file:
------------------------------------------------------------
...

include "jnius_compat.pxi"
include "jni.pxi"
include "config.pxi"

IF JNIUS_PLATFORM == "android":
  ^
------------------------------------------------------------

jnius/jnius.pyx:102:3: Compile-time name 'JNIUS_PLATFORM' not defined

Error compiling Cython file:
------------------------------------------------------------
...
include "jni.pxi"
include "config.pxi"

IF JNIUS_PLATFORM == "android":
    include "jnius_jvm_android.pxi"
ELIF JNIUS_PLATFORM == "win32":
    ^
------------------------------------------------------------

jnius/jnius.pyx:104:5: Compile-time name 'JNIUS_PLATFORM' not defined

Error compiling Cython file:
------------------------------------------------------------
...
include "config.pxi"
^
------------------------------------------------------------

jnius/jnius_jvm_dlopen.pxi:1:0: 'config.pxi' not found

Error compiling Cython file:
------------------------------------------------------------
...
    JAVA_HOME = os.getenv('JAVA_HOME') or find_java_home()
    if JAVA_HOME is None or JAVA_HOME == '':
        raise SystemError("JAVA_HOME is not set, and unable to guess JAVA_HOME")
    cdef str JNIUS_LIB_SUFFIX = get_jnius_lib_location(JNIUS_PLATFORM)

    IF JNIUS_PYTHON3:
      ^
------------------------------------------------------------

jnius/jnius_jvm_dlopen.pxi:89:7: Compile-time name 'JNIUS_PYTHON3' not defined

Error compiling Cython file:
------------------------------------------------------------
...

include "jnius_env.pxi"
include "jnius_utils.pxi"
include "jnius_conversion.pxi"
include "jnius_localref.pxi"
IF JNIUS_PYTHON3:
  ^
------------------------------------------------------------

jnius/jnius.pyx:113:3: Compile-time name 'JNIUS_PYTHON3' not defined



  STDERR:

Which may suggest that altering "python3" is not robust.

Also:

https://github.com/kivy/buildozer/issues/1422

mavavilj avatar Apr 10 '22 17:04 mavavilj

In #2593 I changed p4a to prefer the newer avdmanager in cmdline-tools/latest/bin. I don't know if that fixes buildozer, though.

dbnicholson avatar May 31 '22 15:05 dbnicholson

Just coming back of a fdroidata merge. And agiain trapped into the avdmanager ClassNonFoundException. f-droid too seems to have its own problems with the new command line tools. But why does p4a use that call? Time to dig into the code.

The only call to avdmanager is in build.py/get_targets() as 'avdmanager list targts'. It delivers a list of parameters for each targent.

get_targets() is called only from get_available_apis(). The only relevant information extracted in there are the api numbers. Isn't it not an overkill?

Targets api information in android-sdk resides in the subdir 'platforms' So why not simply do:

def get_available_apis(sdk_dir): apis = [] path = join(sdk_dir, 'platforms') if exists(path): targets = os.listdir(path) apis = [s for s in targets if re.match(r'^ *android-', s)] apis = [re.findall(r'[0-9]+', s) for s in apis] apis = [int(s[0]) for s in apis if s] else: raise BuildInterruptingException( 'Could not find platfrom information in Android SDK', instructions='Make sure the path to the Android SDK is correct') return apis

This would deliver the same information and eliminate a critical dependency.

lufebe16 avatar Mar 04 '23 16:03 lufebe16

Addendum (correctly formatted now):

def get_available_apis(sdk_dir):
   apis = []
   path = join(sdk_dir, 'platforms')
   if exists(path):
       targets = os.listdir(path)
       apis = [s for s in targets if re.match(r'^ *android-', s)]
       apis = [re.findall(r'[0-9]+', s) for s in apis]
       apis = [int(s[0]) for s in apis if s]
   else:
       raise BuildInterruptingException(
           'Could not find platfrom information in Android SDK',
           instructions='Make sure the path to the Android SDK is correct')
   return apis

lufebe16 avatar Mar 04 '23 16:03 lufebe16