build fails

[glennra]

This is on Ubuntu 18.04.2. Running in a python3 virtualenv. I have followed the setup instructions in the readme.

`$ pydo :build --- Logging error --- Traceback (most recent call last): File "/home/glenn/devel/rpi-ramdisk/init.py", line 14, in jobs = int(os.environ['PYDOJOBS'], 10) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/os.py", line 669, in getitem raise KeyError(key) from None KeyError: 'PYDOJOBS'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 994, in emit msg = self.format(record) File "/usr/lib/python3.6/logging/init.py", line 840, in format return fmt.format(record) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 52, in format return f'{colors.color(command, **l2c(record.levelno))}: {colors.color(record.msg, fg="white")}' AttributeError: 'bool' object has no attribute 'color' Call stack: File "/home/glenn/devel/rpi-ramdisk/venv/bin/pydo", line 10, in sys.exit(main()) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/main.py", line 84, in main importlib.import_module(commands.project_root.name) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/importlib/init.py", line 126, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 994, in _gcd_import File "", line 971, in _find_and_load File "", line 955, in _find_and_load_unlocked File "", line 665, in _load_unlocked File "", line 678, in exec_module File "", line 219, in _call_with_frames_removed File "/home/glenn/devel/rpi-ramdisk/init.py", line 18, in log.warning(f'Setting jobs to {jobs}.') Message: 'Setting jobs to 4.' Arguments: () Don't forget to set up apt-cacher-ng Don't forget to set up apt-cacher-ng Don't forget to set up apt-cacher-ng --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 994, in emit msg = self.format(record) File "/usr/lib/python3.6/logging/init.py", line 840, in format return fmt.format(record) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 52, in format return f'{colors.color(command, **l2c(record.levelno))}: {colors.color(record.msg, fg="white")}' AttributeError: 'bool' object has no attribute 'color' Call stack: File "/home/glenn/devel/rpi-ramdisk/venv/bin/pydo", line 10, in sys.exit(main()) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/main.py", line 106, in main f() File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 80, in _consider_cmd_and_deps f() File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 60, in _run_cmd_if_necessary return f() File "/home/glenn/devel/rpi-ramdisk/kernel/init.py", line 61, in _build ], env=self.env, shell=True) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 18, in _inner return f(logger, *args, **kwargs) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/utils.py", line 49, in call logger.info(c) Message: 'git -C /home/glenn/devel/rpi-ramdisk/kernel/linux clean -dfxq' Arguments: () --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 994, in emit msg = self.format(record) File "/usr/lib/python3.6/logging/init.py", line 840, in format return fmt.format(record) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 52, in format return f'{colors.color(command, **l2c(record.levelno))}: {colors.color(record.msg, fg="white")}' AttributeError: 'bool' object has no attribute 'color' Call stack: File "/home/glenn/devel/rpi-ramdisk/venv/bin/pydo", line 10, in sys.exit(main()) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/main.py", line 106, in main f() File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 80, in _consider_cmd_and_deps f() File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 60, in _run_cmd_if_necessary return f() File "/home/glenn/devel/rpi-ramdisk/kernel/init.py", line 61, in _build ], env=self.env, shell=True) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 18, in _inner return f(logger, *args, **kwargs) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/utils.py", line 49, in call logger.info(c) Message: 'rm -rf --one-file-system /home/glenn/devel/rpi-ramdisk/kernel/stage' Arguments: () --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 994, in emit msg = self.format(record) File "/usr/lib/python3.6/logging/init.py", line 840, in format return fmt.format(record) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 52, in format return f'{colors.color(command, **l2c(record.levelno))}: {colors.color(record.msg, fg="white")}' AttributeError: 'bool' object has no attribute 'color' Call stack: File "/home/glenn/devel/rpi-ramdisk/venv/bin/pydo", line 10, in sys.exit(main()) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/main.py", line 106, in main f() File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 80, in _consider_cmd_and_deps f() File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 60, in _run_cmd_if_necessary return f() File "/home/glenn/devel/rpi-ramdisk/kernel/init.py", line 61, in _build ], env=self.env, shell=True) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 18, in _inner return f(logger, *args, **kwargs) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/utils.py", line 49, in call logger.info(c) Message: 'cp /home/glenn/devel/rpi-ramdisk/kernel/kernel.config /home/glenn/devel/rpi-ramdisk/kernel/linux/.config' Arguments: () --- Logging error --- Traceback (most recent call last): File "/usr/lib/python3.6/logging/init.py", line 994, in emit msg = self.format(record) File "/usr/lib/python3.6/logging/init.py", line 840, in format return fmt.format(record) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 52, in format return f'{colors.color(command, **l2c(record.levelno))}: {colors.color(record.msg, fg="white")}' AttributeError: 'bool' object has no attribute 'color' Call stack: File "/home/glenn/devel/rpi-ramdisk/venv/bin/pydo", line 10, in sys.exit(main()) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/main.py", line 106, in main f() File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 80, in _consider_cmd_and_deps f() File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 60, in _run_cmd_if_necessary return f() File "/home/glenn/devel/rpi-ramdisk/kernel/init.py", line 61, in _build ], env=self.env, shell=True) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 18, in _inner return f(logger, *args, **kwargs) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/utils.py", line 49, in call logger.info(c) Message: 'make -j4 -C /home/glenn/devel/rpi-ramdisk/kernel/linux zImage modules dtbs' Arguments: () make: *** No rule to make target 'zImage'. Stop. Traceback (most recent call last): File "/home/glenn/devel/rpi-ramdisk/venv/bin/pydo", line 10, in sys.exit(main()) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/main.py", line 106, in main f() File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 80, in _consider_cmd_and_deps f() File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 60, in _run_cmd_if_necessary return f() File "/home/glenn/devel/rpi-ramdisk/kernel/init.py", line 61, in _build ], env=self.env, shell=True) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 18, in _inner return f(logger, *args, **kwargs) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/utils.py", line 55, in call stderr=subprocess.DEVNULL if verbosity < 1 else None, File "/usr/lib/python3.6/subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command 'make -j4 -C /home/glenn/devel/rpi-ramdisk/kernel/linux zImage modules dtbs' returned non-zero exit status 2. `

[ali1234]

Should be fixed in https://github.com/ali1234/pydo/commit/dee856ebfe68bc76475ee29d9dc13f6819ccf587

It will now fall back to not colouring logs instead of crashing when the colors module is not available.

[glennra]

That works now, but there is another issue: $ pydo :build ::18: Setting jobs to 4. Don't forget to set up apt-cacher-ng Don't forget to set up apt-cacher-ng Don't forget to set up apt-cacher-ng [7/15] kernel:_build:61: git -C /home/glenn/devel/rpi-ramdisk/kernel/linux clean -dfxq [7/15] kernel:_build:61: rm -rf --one-file-system /home/glenn/devel/rpi-ramdisk/kernel/stage [7/15] kernel:_build:61: cp /home/glenn/devel/rpi-ramdisk/kernel/kernel.config /home/glenn/devel/rpi-ramdisk/kernel/linux/.config [7/15] kernel:_build:61: make -j4 -C /home/glenn/devel/rpi-ramdisk/kernel/linux zImage modules dtbs make: *** No rule to make target 'zImage'. Stop. Traceback (most recent call last): File "/home/glenn/devel/rpi-ramdisk/venv/bin/pydo", line 10, in sys.exit(main()) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/main.py", line 106, in main f() File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 80, in _consider_cmd_and_deps f() File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/commands.py", line 60, in _run_cmd_if_necessary return f() File "/home/glenn/devel/rpi-ramdisk/kernel/init.py", line 61, in _build ], env=self.env, shell=True) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/loghelper.py", line 18, in _inner return f(logger, *args, **kwargs) File "/home/glenn/devel/rpi-ramdisk/venv/lib/python3.6/site-packages/pydo/utils.py", line 55, in call stderr=subprocess.DEVNULL if verbosity < 1 else None, File "/usr/lib/python3.6/subprocess.py", line 418, in run output=stdout, stderr=stderr) subprocess.CalledProcessError: Command 'make -j4 -C /home/glenn/devel/rpi-ramdisk/kernel/linux zImage modules dtbs' returned non-zero exit status 2.

[ali1234]

That sounds like you did not recursively clone the submodules.

[glennra]

You are right, the submodule update failed:

git submodule update --recursive remote: Enumerating objects: 29, done. remote: Counting objects: 100% (29/29), done. remote: Compressing objects: 100% (6/6), done. remote: Total 16 (delta 13), reused 13 (delta 10), pack-reused 0 Unpacking objects: 100% (16/16), done. From git://github.com/raspberrypi/linux 97b8d85c9..e20d30ad9 rpi-4.19.y -> origin/rpi-4.19.y error: Server does not allow request for unadvertised object 3c35cb554e1f0333af3622773afa4013131ba994 Fetched in submodule path 'kernel/linux', but it did not contain 3c35cb554e1f0333af3622773afa4013131ba994. Direct fetching of that commit failed.

[glennra]

I did a shallow clone to save time/space, I guess it needs some commit that I didn't get.

[ali1234]

Ugh that again. Yes, github won't let you shallow clone an arbitrary commit. Only if it is the tip of a branch or a tag. So you have to clone the whole repository. Is this something I need to fix? Because I tried to implement shallow cloning by default before but had to back it out due to this reason. Still looking for a solution (one would be to mirror the upstream repos and just add tags to them).

[glennra]

I think you just need to add a note that shallow cloning won't work for the kernel submodule. An alternative may be to provide a tarball of the necessary kernel version. With a normal DSL connection (~10Mbps) it still takes a while to download the kernel repo.

[ali1234]

It won't work for most of them unfortunately. I think GNOME were considering enabling it on their gitlab. What I could do is keep the master branch always pointing to an upstream tag. I don't really want to get in the business of mirroring tarballs.