script fetch configs from different kernel images for current architecture

[o8opi]

This script now tries to fetch and/or build the different kernel images for current architecture and derive the kernel configs from them

[o8opi]

This might resolve #63

[codecov-commenter]

Codecov Report

Merging #64 (86b6b08) into master (0d5c56f) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #64   +/-   ##
=======================================
  Coverage   98.08%   98.08%           
=======================================
  Files           3        3           
  Lines         625      625           
  Branches      139      139           
=======================================
  Hits          613      613           
  Misses          5        5           
  Partials        7        7           

Flag	Coverage Δ
functional_test	98.08% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

[a13xp0p0v]

Hello @o8opi,

I tried your version of this script in a Docker container with Ubuntu 20.04.2.

It failed with the error:

...
copying path '/nix/store/l920bx9bw37jd681pk98dfra0j3lanva-libarchive-3.6.1-lib' from 'https://cache.nixos.org'...
copying path '/nix/store/km0c80plib16fp76prmhcdwbag9iqnvf-nix-2.9.1' from 'https://cache.nixos.org'...
copying path '/nix/store/0szyscpg632p7vlj9if5gadwlvwcb91d-nix-2.9.1-dev' from 'https://cache.nixos.org'...
building '/nix/store/yz1y19d71lp53jymd51h4qw9c2663x6a-builder.pl.drv'...
building '/nix/store/c539pzdghlrfcik2qymswm30ycbdj3yz-python3-3.9.13-env.drv'...
created 226 symlinks in user environment
Traceback (most recent call last):
  File "/home/a13x/src/kconfig-hardened-check/contrib/./get-nix-kconfig.py", line 61, in <module>
    main()
  File "/home/a13x/src/kconfig-hardened-check/contrib/./get-nix-kconfig.py", line 16, in main
    data = json.loads(proc.stdout)
  File "/nix/store/553d7c4xcwp9j1a1gb9cb1s9ry3x1pi9-python3-3.9.13/lib/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/nix/store/553d7c4xcwp9j1a1gb9cb1s9ry3x1pi9-python3-3.9.13/lib/python3.9/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/nix/store/553d7c4xcwp9j1a1gb9cb1s9ry3x1pi9-python3-3.9.13/lib/python3.9/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

It looks like json.loads() didn't manage to handle the output of nix search.

[o8opi]

this should work better now

[o8opi]

I have tested in an Ubuntu-20.04 container and it worked for me, can share Dockerfile if needed :)

[a13xp0p0v]

Hello @o8opi,

Now it works better, but gives a bunch of other errors:

created 223 symlinks in user environment
extract-vmlinux: Cannot find vmlinux.
Usage: extract-ikconfig <kernel-image>
failed to extract config from legacyPackages.x86_64-linux.linuxPackages_5_10_hardened.kernel
extract-vmlinux: Cannot find vmlinux.
Usage: extract-ikconfig <kernel-image>
failed to extract config from legacyPackages.x86_64-linux.linuxPackages_5_15_hardened.kernel
extract-vmlinux: Cannot find vmlinux.
Usage: extract-ikconfig <kernel-image>
failed to extract config from legacyPackages.x86_64-linux.linuxPackages_5_18_hardened.kernel
extract-vmlinux: Cannot find vmlinux.
Usage: extract-ikconfig <kernel-image>
failed to extract config from legacyPackages.x86_64-linux.linuxPackages_hardened.kernel
error: Package ‘linux-4.14.180-176’ in /nix/store/xcba8ikxvdzw7ycg5ncnfq37w9491cn9-source/pkgs/os-specific/linux/kernel/linux-hardkernel-4.14.nix:4 is not supported on ‘x86_64-linux’, refusing to evaluate.

       a) To temporarily allow packages that are unsupported for this system, you can use an environment variable
          for a single invocation of the nix tools.

            $ export NIXPKGS_ALLOW_UNSUPPORTED_SYSTEM=1

        Note: For `nix shell`, `nix build`, `nix develop` or any other Nix 2.4+
        (Flake) command, `--impure` must be passed in order to read this
        environment variable.

       b) For `nixos-rebuild` you can set
         { nixpkgs.config.allowUnsupportedSystem = true; }
       in configuration.nix to override this.

       c) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
         { allowUnsupportedSystem = true; }
       to ~/.config/nixpkgs/config.nix.
(use '--show-trace' to show detailed location information)
failed to build legacyPackages.x86_64-linux.linuxPackages_hardkernel_latest.kernel
extract-vmlinux: Cannot find vmlinux.
Usage: extract-ikconfig <kernel-image>
failed to extract config from legacyPackages.x86_64-linux.linuxPackages_latest.kernel
extract-vmlinux: Cannot find vmlinux.
Usage: extract-ikconfig <kernel-image>
failed to extract config from legacyPackages.x86_64-linux.linuxPackages_latest-libre.kernel
extract-vmlinux: Cannot find vmlinux.
Usage: extract-ikconfig <kernel-image>
failed to extract config from legacyPackages.x86_64-linux.linuxPackages_lqx.kernel
error: Package ‘linux-5.18.12-bcachefs-unstable-2022-04-25’ in /nix/store/xcba8ikxvdzw7ycg5ncnfq37w9491cn9-source/pkgs/os-specific/linux/kernel/linux-testing-bcachefs.nix:15 is marked as broken, refusing to evaluate.

       a) To temporarily allow broken packages, you can use an environment variable
          for a single invocation of the nix tools.

            $ export NIXPKGS_ALLOW_BROKEN=1

        Note: For `nix shell`, `nix build`, `nix develop` or any other Nix 2.4+
        (Flake) command, `--impure` must be passed in order to read this
        environment variable.

       b) For `nixos-rebuild` you can set
         { nixpkgs.config.allowBroken = true; }
       in configuration.nix to override this.

       c) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
         { allowBroken = true; }
       to ~/.config/nixpkgs/config.nix.
(use '--show-trace' to show detailed location information)
failed to build legacyPackages.x86_64-linux.linuxPackages_testing_bcachefs.kernel
extract-vmlinux: Cannot find vmlinux.
Usage: extract-ikconfig <kernel-image>
failed to extract config from legacyPackages.x86_64-linux.linuxPackages_xanmod.kernel
extract-vmlinux: Cannot find vmlinux.
Usage: extract-ikconfig <kernel-image>
failed to extract config from legacyPackages.x86_64-linux.linuxPackages_xanmod_latest.kernel
extract-vmlinux: Cannot find vmlinux.
Usage: extract-ikconfig <kernel-image>
failed to extract config from legacyPackages.x86_64-linux.linuxPackages_zen.kernel

I see at least three different kinds of errors here. Could you have a look?

I would also ask you to rebase your branch over origin/master.

Thanks!