termux-packages [Bug]: build-package.sh: cannot compile subpackages

Problem description

Now I'm setting up build-bootstraps.sh to support compiling bootstrap which will be based on pacman. During testing, a bug was noticed, it appears when the compiler reaches the bzip2 package.

Error message (testing happens in docker):

[*] Building 'bzip2'...
ERROR: No package bzip2 found in any of the enabled repositories. Are you trying to set up a custom repository?
[*] Building 'bzip2' exited with exit code 1
Failed to build package 'bzip2' for arch 'aarch64' in format 'pacman'

In fact, this can be fixed in build-bootstraps.sh, but I don't know if this will be correct.

What steps will reproduce the bug?

Command:

./scripts/build-bootstraps.sh --pm pacman --architectures aarch64

What is the expected behavior?

No response

System information

On all devices

May 02 '22 20:05 Maxython

I will try to fix this bug in build-bootstraps.sh.

May 03 '22 20:05 Maxython

The issue exists because build-bootstraps.sh currently doesn't support building subpackages and the three subpackages bzip2, curl and xz-utils builds were ignored but it wasn't an issue at the time of the prototype since their parent packages were dependencies of apt and so would automatically get included.

There was a note for a related issue as well at https://github.com/termux/termux-packages/blob/44572292816c5eca4eca32ec66068af5456255a3/scripts/build-bootstraps.sh#L80

Previously, the builds would complete because No build.sh script at package dir errors were ignored but recently the monorepo commit also added a precheck for the TERMUX_PKG_BUILDER_DIR itself, which changed the error thrown so it wasn't ignored and building failed.

https://github.com/termux/termux-packages/commit/f0ea9b922bf2b4a642ba78136cd01d46b5e0f624#diff-731444686e10066f1aeb6193b5c45aa13844d82e0f54de5aec3c325402972f2bR472

However, at this stage, this will need to fixed, since parent packages may not always be dependencies of other packages.

So firstly the build-bootstrap.sh will need to check if build.sh is not found, then try checking if its a subpackage, and then build its parent, basically, automatically build libbz2 instead of bzip2. But that creates an issue, we actually don't want the debs of parent packages to be added to bootstrap and only the subpackage and their dependencies. So we need to know exactly what package dependencies will be built for the parent package and the subpackage and only include the debs of subpackage and its dependencies in the bootstrap. The parent package and their dependencies could also be included if parent package is itself a dependency of another package, like libbz2 is of apt. I need to refactor the build-boostraps.sh and generate-bootstraps.sh scripts, will look into fixing this too. It may be possible to use buildorder.py somehow.

May 06 '22 08:05 agnostic-apollo

Ok, I see

May 06 '22 08:05 Maxython

This issue/PR has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Jun 20 '22 09:06 stale[bot]

This also affects normal apt based packaging fyi

I think temporarily patching build-bootstraps.sh for now is better than having it broken for a long time...

Aug 04 '22 10:08 truboxl

This also affects normal apt based packaging fyi

The bootstraps generated for apps are done with generate-bootstraps.sh instead.

https://github.com/termux/termux-packages/wiki/For-maintainers#bootstraps

Also check https://github.com/termux/termux-app/issues/2892#issuecomment-1186482737

I plan on working more on it soon before next termux-app release. Both scripts require lot of changes.

Aug 04 '22 18:08 agnostic-apollo

Actually I have been experimenting build-bootstraps.sh for arm vfpv3-d16 for some time so I can't use generate-bootsraps.sh

I kept retrying (by updating packages that have been outdated) until finally reach this error only to see its been reported a long time ago (but its definitely not related to using pacman)

Aug 04 '22 21:08 truboxl

You can use changes in #10540 for now.

Aug 05 '22 08:08 agnostic-apollo

This issue/PR has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Sep 19 '22 08:09 stale[bot]

You can use changes in #10540 for now.

I have tested #10540, it shows:

$ ./build-bootstraps.sh --architectures aarch64

[*] Building 'apt'...
[email protected] built - skipping (rm /data/data/.built-packages/apt to force rebuild)
[*] Building 'apt' exited with exit code 0

[*] Building 'bash'...
[email protected] built - skipping (rm /data/data/.built-packages/bash to force rebuild)
[*] Building 'bash' exited with exit code 0

[*] Building 'libbz2'...
[email protected] built - skipping (rm /data/data/.built-packages/libbz2 to force rebuild)
[*] Building 'libbz2' exited with exit code 0

[*] Building 'command-not-found'...
termux - building command-not-found for arch aarch64...
Downloading https://github.com/termux/command-not-found/archive/2.1.0.tar.gz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0
100  8184    0  8184    0     0   1950      0 --:--:--  0:00:04 --:--:--  4019
Downloading https://nodejs.org/dist/v16.14.0/node-v16.14.0-linux-x64.tar.xz
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 20.9M  100 20.9M    0     0  26109      0  0:13:59  0:13:59 --:--:-- 10593
-- Android: Targeting API '24' with architecture 'arm64', ABI 'arm64-v8a', and processor 'aarch64'
-- The C compiler identification is Clang 14.0.6
-- The CXX compiler identification is Clang 14.0.6
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /home/builder/.termux-build/_cache/android-r25b-api-24-v0/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /home/builder/.termux-build/_cache/android-r25b-api-24-v0/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
CMake Warning:
  Manually-specified variables were not used by the project:

    BUILD_TESTING
    CMAKE_FIND_ROOT_PATH
    CMAKE_FIND_ROOT_PATH_MODE_LIBRARY
    CMAKE_INSTALL_LIBDIR
    CMAKE_USE_SYSTEM_LIBRARIES
    DOXYGEN_EXECUTABLE


-- Build files have been written to: /home/builder/.termux-build/command-not-found/build
[1/3] cd /home/builder/.termux-build/command-not-found/src && ./generate-db.js
[2/3] Building CXX object CMakeFiles/command-not-found.dir/command-not-found.cpp.o
FAILED: CMakeFiles/command-not-found.dir/command-not-found.cpp.o 
/home/builder/.termux-build/_cache/android-r25b-api-24-v0/bin/clang++ --target=aarch64-none-linux-android --gcc-toolchain=/home/builder/.termux-build/_cache/android-r25b-api-24-v0 --sysroot=/home/builder/.termux-build/_cache/android-r25b-api-24-v0/sysroot   -fstack-protector-strong -Oz --target=aarch64-linux-android24  -I/data/data/com.termux/files/usr/include -O3 -DNDEBUG -fPIE -Wall -Wextra -Werror -pedantic -MD -MT CMakeFiles/command-not-found.dir/command-not-found.cpp.o -MF CMakeFiles/command-not-found.dir/command-not-found.cpp.o.d -o CMakeFiles/command-not-found.dir/command-not-found.cpp.o -c /home/builder/.termux-build/command-not-found/src/command-not-found.cpp
/home/builder/.termux-build/command-not-found/src/command-not-found.cpp:30:11: fatal error: 'commands-aarch64-termux-main.h' file not found
# include "commands-aarch64-termux-main.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 error generated.
ninja: build stopped: subcommand failed.
[*] Building 'command-not-found' exited with exit code 1
Failed to build package 'command-not-found' for arch 'aarch64' in format 'debian'
builder@86fd6add4010:~/termux-packages/scripts$ ls /home/builder/.termux-build/command-not-found/src/command-not-found.cpp
/home/builder/.termux-build/command-not-found/src/command-not-found.cpp

Any idea?

Sep 21 '22 00:09 diyism

https://github.com/termux/command-not-found/blob/e187c1099fd6ef22524e2ca2ce7314f12065a593/command-not-found.cpp#L30

cc: @thunder-coding

Sep 21 '22 00:09 agnostic-apollo

I can't reproduce the behaviour, atleast when building just command-not-found. Can you try building just command-not-found? Also, make sure you're able to ping packages-cf.termux.org, and make HTTPS connections with the site.

Sep 21 '22 01:09 thunder-coding

I can't reproduce the behaviour, atleast when building just command-not-found. Can you try building just command-not-found? Also, make sure you're able to ping packages-cf.termux.org, and make HTTPS connections with the site.

You're right, it's my network problem, I tried again, it can continue now.

Sep 21 '22 01:09 diyism

Sorry for bothering you again, with: TERMUX_PKG_ESSENTIAL=true ./termux-packages/scripts/run-docker.sh ./scripts/build-bootstraps.10540.sh --architectures aarch64 I have successfully built the "./termux-packages/bootstrap-aarch64.zip", but its size is about 242MB, while the bootstrap-aarch64.zip in releases page(https://github.com/termux/termux-packages/releases/tag/bootstrap-2022.09.18-r1%2Bapt-android-7) is only 25MB,

Is there a special building parameter for the slim recipe?

Sep 22 '22 17:09 diyism

@diyism well what does ./scripts/build-bootstraps.10540.sh contain? Which packages have you added? Possibly added packages (and their dependencies) bring bootstraps to 242 mb

Sep 22 '22 20:09 Grimler91

We use generate-bootstraps.sh which only packs the essential packages

https://github.com/termux/termux-packages/blob/9381ffe8b5c0d6c66386504c6d52c3751ec7e3e0/.github/workflows/bootstrap_archives.yml#L30

https://github.com/termux/termux-packages/blob/9381ffe8b5c0d6c66386504c6d52c3751ec7e3e0/scripts/generate-bootstraps.sh#L429-L472

But generate-bootstraps.sh pulls packages from server instead of locally built packages, that's where #11491 comes in However generate-bootstraps.sh does not keep Unix permission so some packages do not work, https://github.com/termux/termux-packages/issues/11844#issuecomment-1239713251

Sep 23 '22 10:09 truboxl

@diyism I encountered both of your problems. Maybe I have to give up previous progress to fix the missing "commands-aarch64-termux-main.h" file? ~I still have no idea - it would be helpful if I know exactly what went wrong. Tried to fix this in https://github.com/termux/command-not-found/pull/17~ Not a real fix, still thinking about how to handle this.

I added nothing but nodejs - which then seems to bring many extra packages - however it seems quite weird that some seemingly unrelated packages like "tor" are also included. Have no idea what went wrong either.

Oct 23 '22 02:10 segfault-bilibili

By the way I'm using the patch from #10540 as well.

Oct 23 '22 03:10 segfault-bilibili

This issue/PR has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

Dec 11 '22 17:12 stale[bot]

Is any plan to address these bugs both issues are still there with https://github.com/termux/termux-packages/pull/10540 also

/home/builder/.termux-build/command-not-found/src/command-not-found.cpp:30:11: fatal error: 'commands-aarch64-termux-main.h' file not found
# include "commands-aarch64-termux-main.h"
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

and clang and tor like packages are still building when running build-bootstraps.sh

Jan 18 '23 15:01 hnmn

docker run --name=unruffled_williamson --hostname=ded0bdd038e4 --user=builder:builder --mac-address=02:42:ac:11:00:06 --volume=/:/all --volume=/root/docker/termux-package-builder/home_builder:/home/builder --volume=/home/qgb/github/termux_python3.9.7:/home/builder/termux-packages --volume=/root/docker/termux-package-builder/usr:/usr --workdir=/home/builder/termux-packages -p 397:397 --restart=no --runtime=runc -t termux/package-builder /bin/bash

May 16 '23 15:05 QGB

./build-package.sh -a arm ncurses

May 16 '23 19:05 QGB

@QGB please don't spam random shell commands here.

May 17 '23 03:05 thunder-coding

@agnostic-apollo the error still showing (lastest repo clone) , and i still cannot build using build-bootstraps.sh as it show:

[*] Building 'bzip2'...
ERROR: No package bzip2 found in any of the enabled repositories. Are you trying to set up a custom repository?
[*] Building 'bzip2' exited with exit code 1
Failed to build package 'bzip2' for arch 'aarch64'

and yeah i am trying to build package locally. if i use generate-bootstraps.sh , it generate a small bootstrap.zip file (150KB) it contain var/lib/dpkg/info/grep.list and more like that.... is this because this warning

curl: (22) The requested URL returned error: 404
[!] Skipping architecture-independent package list as not available...
[*] Downloading package list for architecture 'arm'...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--
...... some downloads .......
[*] Extracting 'unzip'...
[*] Creating 'bootstrap-arm.zip'...
  adding: etc/ (stored 0%)
  adding: etc/apt/ (stored 0%)
  adding: etc/apt/preferences.d/ (stored 0%)
  adding: etc/apt/apt.conf.d/ (stored 0%)
  adding: tmp/ (stored 0%)
  adding: var/ (stored 0%)
  adding: var/log/ (stored 0%)
  adding: var/log/apt/ (stored 0%)
  adding: var/lib/ (stored 0%)
  adding: var/lib/dpkg/ (stored 0%)
  adding: var/lib/dpkg/status (deflated 77%)
  adding: var/lib/dpkg/updates/ (stored 0%)
  adding: var/lib/dpkg/triggers/ (stored 0%)
  adding: var/lib/dpkg/info/ (stored 0%)
  adding: var/lib/dpkg/info/util-linux.list (deflated 91%)
  adding: var/lib/dpkg/info/findutils.md5sums (deflated 61%)
  adding: var/lib/dpkg/info/bzip2.list (deflated 86%)
  adding: var/lib/dpkg/info/command-not-found.md5sums (deflated 12%)
  adding: var/lib/dpkg/info/termux-licenses.list (deflated 87%)
  adding: var/lib/dpkg/info/readline.conffiles (deflated 5%)
  adding: var/lib/dpkg/info/libgmp.list (deflated 84%)
  adding: var/lib/dpkg/info/dos2unix.md5sums (deflated 66%)
  adding: var/lib/dpkg/info/libunistring.md5sums (deflated 72%)
  adding: var/lib/dpkg/info/inetutils.md5sums (deflated 66%)
  adding: var/lib/dpkg/info/termux-tools.list (deflated 88%)
  adding: var/lib/dpkg/info/libssh2.list (deflated 91%)
  adding: var/lib/dpkg/info/libsmartcols.list (deflated 79%)
  adding: var/lib/dpkg/info/libtirpc.list (deflated 90%)
  adding: var/lib/dpkg/info/net-tools.md5sums (deflated 67%)
  adding: var/lib/dpkg/info/nano.conffiles (deflated 5%)
  adding: var/lib/dpkg/info/libnghttp2.md5sums (deflated 54%)
  adding: var/lib/dpkg/info/zstd.md5sums (deflated 61%)
  adding: var/lib/dpkg/info/lsof.md5sums (deflated 42%)
  adding: var/lib/dpkg/info/libtirpc.md5sums (deflated 70%)
  adding: var/lib/dpkg/info/tar.list (deflated 84%)
  adding: var/lib/dpkg/info/libandroid-support.md5sums (deflated 48%)
  adding: var/lib/dpkg/info/resolv-conf.list (deflated 78%)
  adding: var/lib/dpkg/info/libnettle.list (deflated 90%)
  adding: var/lib/dpkg/info/procps.md5sums (deflated 68%)
  adding: var/lib/dpkg/info/gawk.md5sums (deflated 68%)
  adding: var/lib/dpkg/info/nano.list (deflated 90%)
  adding: var/lib/dpkg/info/ncurses.list (deflated 92%)
  adding: var/lib/dpkg/info/patch.list (deflated 79%)
  adding: var/lib/dpkg/info/libgmp.md5sums (deflated 61%)
  adding: var/lib/dpkg/info/dialog.list (deflated 81%)
  adding: var/lib/dpkg/info/termux-am.md5sums (deflated 30%)
  adding: var/lib/dpkg/info/procps.list (deflated 88%)
  adding: var/lib/dpkg/info/gawk.list (deflated 88%)
  adding: var/lib/dpkg/info/sed.md5sums (deflated 43%)
  adding: var/lib/dpkg/info/net-tools.list (deflated 87%)
  adding: var/lib/dpkg/info/tar.md5sums (deflated 58%)
  adding: var/lib/dpkg/info/sed.list (deflated 82%)
  adding: var/lib/dpkg/info/liblzma.list (deflated 89%)
  adding: var/lib/dpkg/info/termux-am-socket.list (deflated 81%)
  adding: var/lib/dpkg/info/grep.md5sums (deflated 53%)
  adding: var/lib/dpkg/info/termux-exec.list (deflated 79%)
  adding: var/lib/dpkg/info/libgpg-error.list (deflated 85%)
  adding: var/lib/dpkg/info/libcap-ng.md5sums (deflated 68%)
  adding: var/lib/dpkg/info/coreutils.md5sums (deflated 71%)
  adding: var/lib/dpkg/info/liblz4.md5sums (deflated 61%)
  adding: var/lib/dpkg/info/gzip.md5sums (deflated 67%)
  adding: var/lib/dpkg/info/libidn2.list (deflated 88%)
  adding: var/lib/dpkg/info/ca-certificates.list (deflated 78%)
  adding: var/lib/dpkg/info/libassuan.list (deflated 82%)
  adding: var/lib/dpkg/info/libnpth.list (deflated 80%)
  adding: var/lib/dpkg/info/nano.md5sums (deflated 69%)
  adding: var/lib/dpkg/info/bash.list (deflated 89%)
  adding: var/lib/dpkg/info/libidn2.md5sums (deflated 70%)
  adding: var/lib/dpkg/info/libandroid-glob.md5sums (deflated 40%)
  adding: var/lib/dpkg/info/diffutils.md5sums (deflated 61%)
  adding: var/lib/dpkg/info/libc++.md5sums (deflated 8%)
  adding: var/lib/dpkg/info/openssl.md5sums (deflated 71%)
  adding: var/lib/dpkg/info/termux-licenses.md5sums (deflated 68%)
  adding: var/lib/dpkg/info/dpkg.md5sums (deflated 66%)
  adding: var/lib/dpkg/info/libmd.list (deflated 92%)
  adding: var/lib/dpkg/info/bash.md5sums (deflated 68%)
  adding: var/lib/dpkg/info/zlib.list (deflated 82%)
  adding: var/lib/dpkg/info/resolv-conf.md5sums (deflated 32%)
  adding: var/lib/dpkg/info/diffutils.list (deflated 85%)
  adding: var/lib/dpkg/info/liblz4.list (deflated 84%)
  adding: var/lib/dpkg/info/less.md5sums (deflated 58%)
  adding: var/lib/dpkg/info/patch.md5sums (deflated 31%)
  adding: var/lib/dpkg/info/command-not-found.list (deflated 79%)
  adding: var/lib/dpkg/info/openssl.list (deflated 91%)
  adding: var/lib/dpkg/info/gzip.list (deflated 88%)
  adding: var/lib/dpkg/info/libassuan.md5sums (deflated 54%)
  adding: var/lib/dpkg/info/libgpg-error.md5sums (deflated 64%)
  adding: var/lib/dpkg/info/psmisc.list (deflated 85%)
  adding: var/lib/dpkg/info/less.list (deflated 84%)
  adding: var/lib/dpkg/info/lsof.list (deflated 81%)
  adding: var/lib/dpkg/info/libmpfr.list (deflated 85%)
  adding: var/lib/dpkg/info/pcre2.md5sums (deflated 74%)
  adding: var/lib/dpkg/info/libunbound.md5sums (deflated 69%)
  adding: var/lib/dpkg/info/curl.md5sums (deflated 31%)
  adding: var/lib/dpkg/info/libcurl.list (deflated 90%)
  adding: var/lib/dpkg/info/dialog.md5sums (deflated 32%)
  adding: var/lib/dpkg/info/dpkg.list (deflated 87%)
  adding: var/lib/dpkg/info/libgcrypt.list (deflated 84%)
  adding: var/lib/dpkg/info/libmpfr.md5sums (deflated 64%)
  adding: var/lib/dpkg/info/inetutils.list (deflated 87%)
  adding: var/lib/dpkg/info/libunbound.list (deflated 88%)
  adding: var/lib/dpkg/info/bash.conffiles (deflated 42%)
  adding: var/lib/dpkg/info/readline.list (deflated 88%)
  adding: var/lib/dpkg/info/libgnutls.list (deflated 87%)
  adding: var/lib/dpkg/info/libmd.md5sums (deflated 66%)
  adding: var/lib/dpkg/info/dash.md5sums (deflated 39%)
  adding: var/lib/dpkg/info/xxhash.md5sums (deflated 53%)
  adding: var/lib/dpkg/info/unzip.list (deflated 85%)
  adding: var/lib/dpkg/info/termux-am.list (deflated 81%)
  adding: var/lib/dpkg/info/libunistring.list (deflated 91%)
  adding: var/lib/dpkg/info/liblzma.md5sums (deflated 70%)
  adding: var/lib/dpkg/info/libssh2.md5sums (deflated 74%)
  adding: var/lib/dpkg/info/pcre2.list (deflated 91%)
  adding: var/lib/dpkg/info/zlib.md5sums (deflated 52%)
  adding: var/lib/dpkg/info/apt.md5sums (deflated 69%)
  adding: var/lib/dpkg/info/curl.list (deflated 79%)
  adding: var/lib/dpkg/info/findutils.list (deflated 84%)
  adding: var/lib/dpkg/info/xxhash.list (deflated 86%)
  adding: var/lib/dpkg/info/libnpth.md5sums (deflated 46%)
  adding: var/lib/dpkg/info/gpgv.md5sums (deflated 30%)
  adding: var/lib/dpkg/info/libevent.md5sums (deflated 69%)
  adding: var/lib/dpkg/info/debianutils.list (deflated 83%)
  adding: var/lib/dpkg/info/apt.conffiles (deflated 4%)
  adding: var/lib/dpkg/info/libandroid-support.list (deflated 80%)
  adding: var/lib/dpkg/info/termux-keyring.md5sums (deflated 63%)
  adding: var/lib/dpkg/info/termux-keyring.list (deflated 88%)
  adding: var/lib/dpkg/info/gpgv.list (deflated 79%)
  adding: var/lib/dpkg/info/readline.md5sums (deflated 67%)
  adding: var/lib/dpkg/info/libgnutls.md5sums (deflated 68%)
  adding: var/lib/dpkg/info/dash.list (deflated 83%)
  adding: var/lib/dpkg/info/libgcrypt.md5sums (deflated 61%)
  adding: var/lib/dpkg/info/termux-exec.md5sums (deflated 9%)
  adding: var/lib/dpkg/info/zstd.list (deflated 86%)
  adding: var/lib/dpkg/info/libsmartcols.md5sums (deflated 46%)
  adding: var/lib/dpkg/info/libiconv.md5sums (deflated 67%)
  adding: var/lib/dpkg/info/libevent.list (deflated 89%)
  adding: var/lib/dpkg/info/termux-tools.conffiles (deflated 87%)
  adding: var/lib/dpkg/info/util-linux.md5sums (deflated 71%)
  adding: var/lib/dpkg/info/xz-utils.md5sums (deflated 65%)
  adding: var/lib/dpkg/info/termux-tools.md5sums (deflated 70%)
  adding: var/lib/dpkg/info/libbz2.list (deflated 81%)
  adding: var/lib/dpkg/info/ed.md5sums (deflated 48%)
  adding: var/lib/dpkg/info/xz-utils.list (deflated 91%)
  adding: var/lib/dpkg/info/libnettle.md5sums (deflated 69%)
  adding: var/lib/dpkg/info/dos2unix.list (deflated 87%)
  adding: var/lib/dpkg/info/libnghttp2.list (deflated 82%)
  adding: var/lib/dpkg/info/nano.postinst (deflated 54%)
  adding: var/lib/dpkg/info/libiconv.list (deflated 87%)
  adding: var/lib/dpkg/info/libc++.list (deflated 78%)
  adding: var/lib/dpkg/info/libandroid-glob.list (deflated 79%)
  adding: var/lib/dpkg/info/unzip.md5sums (deflated 62%)
  adding: var/lib/dpkg/info/coreutils.list (deflated 91%)
  adding: var/lib/dpkg/info/psmisc.md5sums (deflated 63%)
  adding: var/lib/dpkg/info/openssl.conffiles (deflated 4%)
  adding: var/lib/dpkg/info/debianutils.md5sums (deflated 59%)
  adding: var/lib/dpkg/info/libcap-ng.list (deflated 87%)
  adding: var/lib/dpkg/info/nano.prerm (deflated 46%)
  adding: var/lib/dpkg/info/apt.list (deflated 89%)
  adding: var/lib/dpkg/info/libbz2.md5sums (deflated 38%)
  adding: var/lib/dpkg/info/libcurl.md5sums (deflated 72%)
  adding: var/lib/dpkg/info/termux-am-socket.md5sums (deflated 35%)
  adding: var/lib/dpkg/info/resolv-conf.conffiles (deflated 38%)
  adding: var/lib/dpkg/info/ca-certificates.md5sums (deflated 7%)
  adding: var/lib/dpkg/info/ed.list (deflated 84%)
  adding: var/lib/dpkg/info/grep.list (deflated 83%)
  adding: var/lib/dpkg/info/bzip2.md5sums (deflated 64%)
  adding: var/lib/dpkg/info/ncurses.md5sums (deflated 71%)
  adding: var/lib/dpkg/available (stored 0%)
[*] Finished successfully (arm).

what is my problem?

Jul 03 '23 01:07 Agono0

@Agono0 This is because build-bootstraps script is broken. There indeed no bzip2 package because it is a sub package of libbz2. Same for curl and few others. (btw this is what the current issue about)

On other hand generate-bootstraps script works because it operates on already built packages installable by apt, i.e. not their recipes in the termux-packages repository.

See this pull request https://github.com/termux/termux-packages/pull/17031, but anyway built bootstrap still doesn't match generated by generate-bootstraps script.

Jul 03 '23 07:07 sylirre

@sylirre Thanks for response. I try generate-bootstrap.sh to build bootstrap but like i said above, it create (useless) bootstrap.zip 150KB which contains only folders with packages name and here is the output(from generate-bootstrap.sh ):

curl: (22) The requested URL returned error: 404
[!] Skipping architecture-independent package list as not available...
[*] Downloading package list for architecture 'arm'...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--
...... some downloads .......
[*] Extracting 'unzip'...
[*] Creating 'bootstrap-arm.zip'...
  adding: etc/ (stored 0%)
  adding: etc/apt/ (stored 0%)
  adding: etc/apt/preferences.d/ (stored 0%)
  adding: etc/apt/apt.conf.d/ (stored 0%)
  adding: tmp/ (stored 0%)
  adding: var/ (stored 0%)
  adding: var/log/ (stored 0%)
  adding: var/log/apt/ (stored 0%)
  adding: var/lib/ (stored 0%)
  adding: var/lib/dpkg/ (stored 0%)
  adding: var/lib/dpkg/status (deflated 77%)
  adding: var/lib/dpkg/updates/ (stored 0%)
  adding: var/lib/dpkg/triggers/ (stored 0%)
  adding: var/lib/dpkg/info/ (stored 0%)
  adding: var/lib/dpkg/info/util-linux.list (deflated 91%)
  adding: var/lib/dpkg/info/findutils.md5sums (deflated 61%)
  adding: var/lib/dpkg/info/bzip2.list (deflated 86%)
  adding: var/lib/dpkg/info/command-not-found.md5sums (deflated 12%)
  adding: var/lib/dpkg/info/termux-licenses.list (deflated 87%)
  adding: var/lib/dpkg/info/readline.conffiles (deflated 5%)
  adding: var/lib/dpkg/info/libgmp.list (deflated 84%)
  adding: var/lib/dpkg/info/dos2unix.md5sums (deflated 66%)
  adding: var/lib/dpkg/info/libunistring.md5sums (deflated 72%)
  adding: var/lib/dpkg/info/inetutils.md5sums (deflated 66%)
  adding: var/lib/dpkg/info/termux-tools.list (deflated 88%)
  adding: var/lib/dpkg/info/libssh2.list (deflated 91%)
  adding: var/lib/dpkg/info/libsmartcols.list (deflated 79%)
  adding: var/lib/dpkg/info/libtirpc.list (deflated 90%)
  adding: var/lib/dpkg/info/net-tools.md5sums (deflated 67%)
  adding: var/lib/dpkg/info/nano.conffiles (deflated 5%)
  adding: var/lib/dpkg/info/libnghttp2.md5sums (deflated 54%)
  adding: var/lib/dpkg/info/zstd.md5sums (deflated 61%)
  adding: var/lib/dpkg/info/lsof.md5sums (deflated 42%)
  adding: var/lib/dpkg/info/libtirpc.md5sums (deflated 70%)
  adding: var/lib/dpkg/info/tar.list (deflated 84%)
  adding: var/lib/dpkg/info/libandroid-support.md5sums (deflated 48%)
  adding: var/lib/dpkg/info/resolv-conf.list (deflated 78%)
  adding: var/lib/dpkg/info/libnettle.list (deflated 90%)
  adding: var/lib/dpkg/info/procps.md5sums (deflated 68%)
  adding: var/lib/dpkg/info/gawk.md5sums (deflated 68%)
  adding: var/lib/dpkg/info/nano.list (deflated 90%)
  adding: var/lib/dpkg/info/ncurses.list (deflated 92%)
  adding: var/lib/dpkg/info/patch.list (deflated 79%)
  adding: var/lib/dpkg/info/libgmp.md5sums (deflated 61%)
  adding: var/lib/dpkg/info/dialog.list (deflated 81%)
  adding: var/lib/dpkg/info/termux-am.md5sums (deflated 30%)
  adding: var/lib/dpkg/info/procps.list (deflated 88%)
  adding: var/lib/dpkg/info/gawk.list (deflated 88%)
  adding: var/lib/dpkg/info/sed.md5sums (deflated 43%)
  adding: var/lib/dpkg/info/net-tools.list (deflated 87%)
  adding: var/lib/dpkg/info/tar.md5sums (deflated 58%)
  adding: var/lib/dpkg/info/sed.list (deflated 82%)
  adding: var/lib/dpkg/info/liblzma.list (deflated 89%)
  adding: var/lib/dpkg/info/termux-am-socket.list (deflated 81%)
  adding: var/lib/dpkg/info/grep.md5sums (deflated 53%)
  adding: var/lib/dpkg/info/termux-exec.list (deflated 79%)
  adding: var/lib/dpkg/info/libgpg-error.list (deflated 85%)
  adding: var/lib/dpkg/info/libcap-ng.md5sums (deflated 68%)
  adding: var/lib/dpkg/info/coreutils.md5sums (deflated 71%)
  adding: var/lib/dpkg/info/liblz4.md5sums (deflated 61%)
  adding: var/lib/dpkg/info/gzip.md5sums (deflated 67%)
  adding: var/lib/dpkg/info/libidn2.list (deflated 88%)
  adding: var/lib/dpkg/info/ca-certificates.list (deflated 78%)
  adding: var/lib/dpkg/info/libassuan.list (deflated 82%)
  adding: var/lib/dpkg/info/libnpth.list (deflated 80%)
  adding: var/lib/dpkg/info/nano.md5sums (deflated 69%)
  adding: var/lib/dpkg/info/bash.list (deflated 89%)
  adding: var/lib/dpkg/info/libidn2.md5sums (deflated 70%)
  adding: var/lib/dpkg/info/libandroid-glob.md5sums (deflated 40%)
  adding: var/lib/dpkg/info/diffutils.md5sums (deflated 61%)
  adding: var/lib/dpkg/info/libc++.md5sums (deflated 8%)
  adding: var/lib/dpkg/info/openssl.md5sums (deflated 71%)
  adding: var/lib/dpkg/info/termux-licenses.md5sums (deflated 68%)
  adding: var/lib/dpkg/info/dpkg.md5sums (deflated 66%)
  adding: var/lib/dpkg/info/libmd.list (deflated 92%)
  adding: var/lib/dpkg/info/bash.md5sums (deflated 68%)
  adding: var/lib/dpkg/info/zlib.list (deflated 82%)
  adding: var/lib/dpkg/info/resolv-conf.md5sums (deflated 32%)
  adding: var/lib/dpkg/info/diffutils.list (deflated 85%)
  adding: var/lib/dpkg/info/liblz4.list (deflated 84%)
  adding: var/lib/dpkg/info/less.md5sums (deflated 58%)
  adding: var/lib/dpkg/info/patch.md5sums (deflated 31%)
  adding: var/lib/dpkg/info/command-not-found.list (deflated 79%)
  adding: var/lib/dpkg/info/openssl.list (deflated 91%)
  adding: var/lib/dpkg/info/gzip.list (deflated 88%)
  adding: var/lib/dpkg/info/libassuan.md5sums (deflated 54%)
  adding: var/lib/dpkg/info/libgpg-error.md5sums (deflated 64%)
  adding: var/lib/dpkg/info/psmisc.list (deflated 85%)
  adding: var/lib/dpkg/info/less.list (deflated 84%)
  adding: var/lib/dpkg/info/lsof.list (deflated 81%)
  adding: var/lib/dpkg/info/libmpfr.list (deflated 85%)
  adding: var/lib/dpkg/info/pcre2.md5sums (deflated 74%)
  adding: var/lib/dpkg/info/libunbound.md5sums (deflated 69%)
  adding: var/lib/dpkg/info/curl.md5sums (deflated 31%)
  adding: var/lib/dpkg/info/libcurl.list (deflated 90%)
  adding: var/lib/dpkg/info/dialog.md5sums (deflated 32%)
  adding: var/lib/dpkg/info/dpkg.list (deflated 87%)
  adding: var/lib/dpkg/info/libgcrypt.list (deflated 84%)
  adding: var/lib/dpkg/info/libmpfr.md5sums (deflated 64%)
  adding: var/lib/dpkg/info/inetutils.list (deflated 87%)
  adding: var/lib/dpkg/info/libunbound.list (deflated 88%)
  adding: var/lib/dpkg/info/bash.conffiles (deflated 42%)
  adding: var/lib/dpkg/info/readline.list (deflated 88%)
  adding: var/lib/dpkg/info/libgnutls.list (deflated 87%)
  adding: var/lib/dpkg/info/libmd.md5sums (deflated 66%)
  adding: var/lib/dpkg/info/dash.md5sums (deflated 39%)
  adding: var/lib/dpkg/info/xxhash.md5sums (deflated 53%)
  adding: var/lib/dpkg/info/unzip.list (deflated 85%)
  adding: var/lib/dpkg/info/termux-am.list (deflated 81%)
  adding: var/lib/dpkg/info/libunistring.list (deflated 91%)
  adding: var/lib/dpkg/info/liblzma.md5sums (deflated 70%)
  adding: var/lib/dpkg/info/libssh2.md5sums (deflated 74%)
  adding: var/lib/dpkg/info/pcre2.list (deflated 91%)
  adding: var/lib/dpkg/info/zlib.md5sums (deflated 52%)
  adding: var/lib/dpkg/info/apt.md5sums (deflated 69%)
  adding: var/lib/dpkg/info/curl.list (deflated 79%)
  adding: var/lib/dpkg/info/findutils.list (deflated 84%)
  adding: var/lib/dpkg/info/xxhash.list (deflated 86%)
  adding: var/lib/dpkg/info/libnpth.md5sums (deflated 46%)
  adding: var/lib/dpkg/info/gpgv.md5sums (deflated 30%)
  adding: var/lib/dpkg/info/libevent.md5sums (deflated 69%)
  adding: var/lib/dpkg/info/debianutils.list (deflated 83%)
  adding: var/lib/dpkg/info/apt.conffiles (deflated 4%)
  adding: var/lib/dpkg/info/libandroid-support.list (deflated 80%)
  adding: var/lib/dpkg/info/termux-keyring.md5sums (deflated 63%)
  adding: var/lib/dpkg/info/termux-keyring.list (deflated 88%)
  adding: var/lib/dpkg/info/gpgv.list (deflated 79%)
  adding: var/lib/dpkg/info/readline.md5sums (deflated 67%)
  adding: var/lib/dpkg/info/libgnutls.md5sums (deflated 68%)
  adding: var/lib/dpkg/info/dash.list (deflated 83%)
  adding: var/lib/dpkg/info/libgcrypt.md5sums (deflated 61%)
  adding: var/lib/dpkg/info/termux-exec.md5sums (deflated 9%)
  adding: var/lib/dpkg/info/zstd.list (deflated 86%)
  adding: var/lib/dpkg/info/libsmartcols.md5sums (deflated 46%)
  adding: var/lib/dpkg/info/libiconv.md5sums (deflated 67%)
  adding: var/lib/dpkg/info/libevent.list (deflated 89%)
  adding: var/lib/dpkg/info/termux-tools.conffiles (deflated 87%)
  adding: var/lib/dpkg/info/util-linux.md5sums (deflated 71%)
  adding: var/lib/dpkg/info/xz-utils.md5sums (deflated 65%)
  adding: var/lib/dpkg/info/termux-tools.md5sums (deflated 70%)
  adding: var/lib/dpkg/info/libbz2.list (deflated 81%)
  adding: var/lib/dpkg/info/ed.md5sums (deflated 48%)
  adding: var/lib/dpkg/info/xz-utils.list (deflated 91%)
  adding: var/lib/dpkg/info/libnettle.md5sums (deflated 69%)
  adding: var/lib/dpkg/info/dos2unix.list (deflated 87%)
  adding: var/lib/dpkg/info/libnghttp2.list (deflated 82%)
  adding: var/lib/dpkg/info/nano.postinst (deflated 54%)
  adding: var/lib/dpkg/info/libiconv.list (deflated 87%)
  adding: var/lib/dpkg/info/libc++.list (deflated 78%)
  adding: var/lib/dpkg/info/libandroid-glob.list (deflated 79%)
  adding: var/lib/dpkg/info/unzip.md5sums (deflated 62%)
  adding: var/lib/dpkg/info/coreutils.list (deflated 91%)
  adding: var/lib/dpkg/info/psmisc.md5sums (deflated 63%)
  adding: var/lib/dpkg/info/openssl.conffiles (deflated 4%)
  adding: var/lib/dpkg/info/debianutils.md5sums (deflated 59%)
  adding: var/lib/dpkg/info/libcap-ng.list (deflated 87%)
  adding: var/lib/dpkg/info/nano.prerm (deflated 46%)
  adding: var/lib/dpkg/info/apt.list (deflated 89%)
  adding: var/lib/dpkg/info/libbz2.md5sums (deflated 38%)
  adding: var/lib/dpkg/info/libcurl.md5sums (deflated 72%)
  adding: var/lib/dpkg/info/termux-am-socket.md5sums (deflated 35%)
  adding: var/lib/dpkg/info/resolv-conf.conffiles (deflated 38%)
  adding: var/lib/dpkg/info/ca-certificates.md5sums (deflated 7%)
  adding: var/lib/dpkg/info/ed.list (deflated 84%)
  adding: var/lib/dpkg/info/grep.list (deflated 83%)
  adding: var/lib/dpkg/info/bzip2.md5sums (deflated 64%)
  adding: var/lib/dpkg/info/ncurses.md5sums (deflated 71%)
  adding: var/lib/dpkg/available (stored 0%)
[*] Finished successfully (arm).

Am i doing something wrong? Or i miss something to do with script?

Jul 03 '23 07:07 Agono0

@sylirre Can i just use your modified script you provide it on the pull request? Or still not working?

Jul 03 '23 07:07 Agono0

generate-bootstraps is only for official packaging which means it basically useless for Termux derivatives. The Termux application package name should be com.termux, if you want to use it.

Or still not working?

It works, but the generated bootstrap archive is much bigger than it should be. Lots of "junk" included which mostly come from subpackage dependencies.

Jul 03 '23 07:07 sylirre

So , generate-bootstrap.sh will not working if i change the prefix (the package name) idk , i just guess. Anyway, thanks so much for help, I really appreciate that

Jul 03 '23 07:07 Agono0

still have problem with generate-bootstrap.sh i cannot make a vaild bootstrap.zip with SYMLINKS.txt and there is no lib folder i just use sudo ./scripts/run-docker.sh ./scripts/generate-bootstraps.sh what i am missing?? here is produced file : bootstrap-arm.zip

Jul 03 '23 08:07 Agono0

termux-packages termux-packages copied to clipboard

[Bug]: build-package.sh: cannot compile subpackages

Problem description

What steps will reproduce the bug?

What is the expected behavior?

System information

termux-packages
termux-packages copied to clipboard