termux-packages icon indicating copy to clipboard operation
termux-packages copied to clipboard
verified publisher icon termux

bump(main/python-torch{,audio}): 2.5.0

Open twaik opened this issue 1 year ago • 1 comments

Needs testing on device. Also needs testing python-torchvision because I have no idea if updating pytorch breaks pytorchvision or not. Also I tried to update torchvision but failed. python-torchvision.tar.gz I have a weird log

twaik@twaikpc:~/termux-packages$ ./build-package.sh -I -f python-torchvision -s
termux - building python-torchvision for arch aarch64...
Requirement already satisfied: setuptools==67.6.1 in /home/twaik/.termux-build/python3.12-crossenv-prefix-bionic-aarch64/build/lib/python3.12/site-packages (67.6.1)
Requirement already satisfied: wheel in /home/twaik/.termux-build/python3.12-crossenv-prefix-bionic-aarch64/build/lib/python3.12/site-packages (0.44.0)
Requirement already satisfied: setuptools==67.6.1 in /home/twaik/.termux-build/python3.12-crossenv-prefix-bionic-aarch64/build/lib/python3.12/site-packages (67.6.1)
Requirement already satisfied: wheel in /home/twaik/.termux-build/python3.12-crossenv-prefix-bionic-aarch64/build/lib/python3.12/site-packages (0.44.0)
Applying patch: setup.py.patch
Using pip 24.0 from /home/twaik/.termux-build/python3.12-crossenv-prefix-bionic-aarch64/build/lib/python3.12/site-packages/pip (python 3.12)
Processing /home/twaik/.termux-build/python-torchvision/src
  Running command Preparing metadata (pyproject.toml)
  Torchvision build configuration:
  FORCE_CUDA = False
  FORCE_MPS = False
  DEBUG = False
  USE_PNG = True
  USE_JPEG = True
  USE_WEBP = True
  USE_HEIC = False
  USE_AVIF = False
  USE_NVJPEG = True
  NVCC_FLAGS = None
  USE_CPU_VIDEO_DECODER = False
  USE_GPU_VIDEO_DECODER = False
  TORCHVISION_INCLUDE = []
  TORCHVISION_LIBRARY = []
  IS_ROCM = False
  BUILD_CUDA_SOURCES = False
  Building wheel torchvision-0.20.0
  Building _C extension
  Building image extension
  Building torchvision with PNG support
  png_include_dir = '/data/data/com.termux/files/usr/include/libpng16'
  png_library_dir = '/data/data/com.termux/files/usr/lib'
  Searching for jpeglib.h. Didn't find in TORCHVISION_INCLUDE.
  Searching for jpeglib.h. Didn't find in BUILD_PREFIX.
  Searching for jpeglib.h. Didn't find in CONDA_PREFIX.
  Searching for jpeglib.h. Found in /usr/include.
  Building torchvision with JPEG support
  jpeg_include_dir = None
  jpeg_library_dir = None
  Searching for webp/decode.h. Didn't find in TORCHVISION_INCLUDE.
  Searching for webp/decode.h. Didn't find in BUILD_PREFIX.
  Searching for webp/decode.h. Didn't find in CONDA_PREFIX.
  Searching for webp/decode.h. Found in /usr/include.
  Building torchvision with WEBP support
  webp_include_dir = None
  webp_library_dir = None
  Traceback (most recent call last):
    File "/home/twaik/.termux-build/python3.12-crossenv-prefix-bionic-aarch64/build/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/home/twaik/.termux-build/python3.12-crossenv-prefix-bionic-aarch64/build/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/twaik/.termux-build/python3.12-crossenv-prefix-bionic-aarch64/build/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel
      return hook(metadata_directory, config_settings)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/twaik/.termux-build/python3.12-crossenv-prefix-bionic-aarch64/build/lib/python3.12/site-packages/setuptools/build_meta.py", line 377, in prepare_metadata_for_build_wheel
      self.run_setup()
    File "/home/twaik/.termux-build/python3.12-crossenv-prefix-bionic-aarch64/build/lib/python3.12/site-packages/setuptools/build_meta.py", line 485, in run_setup
      self).run_setup(setup_script=setup_script)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/home/twaik/.termux-build/python3.12-crossenv-prefix-bionic-aarch64/build/lib/python3.12/site-packages/setuptools/build_meta.py", line 335, in run_setup
      exec(code, locals())
    File "<string>", line 566, in <module>
    File "<string>", line 364, in make_image_extension
  NameError: name 'torch' is not defined
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /home/twaik/.termux-build/python3.12-crossenv-prefix-bionic-aarch64/cross/bin/python3.12 /home/twaik/.termux-build/python3.12-crossenv-prefix-bionic-aarch64/build/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py prepare_metadata_for_build_wheel /tmp/tmpksd1g6ud
  cwd: /home/twaik/.termux-build/python-torchvision/src
  Preparing metadata (pyproject.toml) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

It is weird because last version has torch module in pyproject.toml and it did not cause this error. Ping @Biswa96 @fornwall @licy183 @landfillbaby because you commited to the python-torchvision package.

twaik avatar Oct 18 '24 10:10 twaik

Ok. So. Pythorch has issue with prctl call failing.

import torch
import math


dtype = torch.float
device = torch.device("cpu")
# device = torch.device("cuda:0") # Uncomment this to run on GPU

# Create random input and output data
x = torch.linspace(-math.pi, math.pi, 2000, device=device, dtype=dtype)
y = torch.sin(x)

# Randomly initialize weights
a = torch.randn((), device=device, dtype=dtype)
b = torch.randn((), device=device, dtype=dtype)
c = torch.randn((), device=device, dtype=dtype)
d = torch.randn((), device=device, dtype=dtype)

learning_rate = 1e-6
for t in range(2000):
    # Forward pass: compute predicted y
    y_pred = a + b * x + c * x ** 2 + d * x ** 3

    # Compute and print loss
    loss = (y_pred - y).pow(2).sum().item()
    if t % 100 == 99:
        print(t, loss)

    # Backprop to compute gradients of a, b, c, d with respect to loss
    grad_y_pred = 2.0 * (y_pred - y)
    grad_a = grad_y_pred.sum()
    grad_b = (grad_y_pred * x).sum()
    grad_c = (grad_y_pred * x ** 2).sum()
    grad_d = (grad_y_pred * x ** 3).sum()

    # Update weights using gradient descent
    a -= learning_rate * grad_a
    b -= learning_rate * grad_b
    c -= learning_rate * grad_c
    d -= learning_rate * grad_d


print(f'Result: y = {a.item()} + {b.item()} x + {c.item()} x^2 + {d.item()} x^3')

~ $ python pytorch.py 
Error in cpuinfo: prctl(PR_SVE_GET_VL) failed
99 1176.34765625
199 786.20703125
299 526.6643676757812
399 353.9311218261719
499 238.9255828857422
599 162.32208251953125
699 111.27428436279297
799 77.24101257324219
899 54.53923416137695
999 39.38843536376953
1099 29.271270751953125
1199 22.511442184448242
1299 17.992034912109375
1399 14.968706130981445
1499 12.944772720336914
1599 11.588927268981934
1699 10.679987907409668
1799 10.070181846618652
1899 9.660752296447754
1999 9.3856201171875
Result: y = 0.013475478626787663 + 0.876336932182312 x + -0.002324742265045643 x^2 + -0.09611776471138 x^3

So this release is problematic.

~ $ python3
Python 3.12.7 (main, Oct  6 2024, 15:06:14) [Clang 18.0.2 (https://android.googlesource.com/toolchain/llvm-project d8003a456 on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
Error in cpuinfo: prctl(PR_SVE_GET_VL) failed
>>> import torchaudio
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/data/data/com.termux/files/home/torchaudio.py", line 8, in <module>
    import librosa
ModuleNotFoundError: No module named 'librosa'
>>> import torchvision
>>> torchvision.version.git_version
'afc54f754c734d903a06194e416495e20d920ff6'

librosa fails to be compiled because of missing fortran/flang.

I am not sure I can fix this.

twaik avatar Oct 19 '24 23:10 twaik

Error in cpuinfo: prctl(PR_SVE_GET_VL) failed

This does not actually affect the functionality of pytorch. pytorch/cpuinfo has downgraded it from error to warning in commit https://github.com/pytorch/cpuinfo/commit/f0d67c613e1a859fdc41238ed5848545e8e0e102.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/data/data/com.termux/files/home/torchaudio.py", line 8, in <module>
    import librosa
ModuleNotFoundError: No module named 'librosa'

It seems that you started your python in $HOME, and there is a file named torchaudio.py in $HOME. Python will load $HOME/torchaudio.py as module torchaudio, and it imports libsora.

licy183 avatar Oct 20 '24 08:10 licy183

It seems that you started your python in $HOME, and there is a file named torchaudio.py in $HOME.

Correct. I am not good pythonist, sorry.

twaik avatar Oct 20 '24 09:10 twaik

LGTM.

twaik avatar Oct 20 '24 12:10 twaik