Build with Nix

[quapka]

Building with Nix is one way of allowing multiple versions of tested libs to be running at the same time. Currently, few versions of OpenSSL are possible, check the commit message for more details.

More work is needed before merging this.

[J08nY]

Oh and one more thing. Can you make the version of the library propagate as an environment variable to the build and run environment? Then in the Makefile we can pass it on as a define to the compiler and thus get it in the shim code so that we can do ifdefs based on it.

[quapka]

Oh and one more thing. Can you make the version of the library propagate as an environment variable to the build and run environment? Then in the Makefile we can pass it on as a define to the compiler and thus get it in the shim code so that we can do ifdefs based on it.

Yes. Both should be somewhat simple and straightforward.

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 90.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 62.66%. Comparing base (65ddb49) to head (8d54fae).

Files	Patch %	Lines
...crcs/ectester/standalone/libs/NativeECLibrary.java	75.00%	0 Missing and 1 partial :warning:

Additional details and impacted files

@@             Coverage Diff              @@
##             master      #29      +/-   ##
============================================
+ Coverage     62.65%   62.66%   +0.01%     
+ Complexity     1349     1348       -1     
============================================
  Files           133      133              
  Lines          9358     9348      -10     
  Branches       1288     1287       -1     
============================================
- Hits           5863     5858       -5     
+ Misses         2882     2877       -5     
  Partials        613      613              

Components	Coverage Δ
Applet	70.27% <ø> (ø)
Common	57.41% <ø> (ø)
Reader	55.44% <ø> (ø)
Standalone	71.05% <90.00%> (+0.07%)	:arrow_up:

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[J08nY]

How do I actually run this? I have nix installed, I did something like: nix build --extra-experimental-features nix-command --extra-experimental-features flakes Is that the right incantation?

I got:

error: hash mismatch in fixed-output derivation '/nix/store/k8g01jg7kbcn7al58i5dzydwb32g6s8z-openssl-3.3.1.tar.gz.drv':
         specified: sha256-Ezvzm40WNayUqEgwQsxEglG3cKDRLHrwwF6old3Zjx0=
            got:    sha256-d3zVlihMiDN1oqehG/XSeG/FQTJV76sgxQ1v/m0CC34=
error: 1 dependencies of derivation '/nix/store/4c17vwxxwlsj3jk0k7la25lh1xxll1r8-openssl-3.0.13.drv' failed to build
error: 1 dependencies of derivation '/nix/store/9fbsv2km4ng6i3zmf3876w6vg8f1isjz-OpenSSL-Shim.drv' failed to build
error: 1 dependencies of derivation '/nix/store/qjv43zm5r1gain7g3297avswvznvsl5y-ECTesterStandalone-0.3.3.drv' failed to build

Edit: There is progress. I changed the hash and got something to build!

Further edit: I get:

warning: Git tree '/home/jjan/dev/ECTester' is dirty
error: builder for '/nix/store/0f7iywj4wlkpbgdyl0xaj8p3x1xxcnbl-ECTesterStandalone-0.3.3.drv' failed with exit code 1;
       last 25 log lines:
       > Welcome to Gradle 8.7!
       >
       > Here are the highlights of this release:
       >  - Compiling and testing with Java 22
       >  - Cacheable Groovy script compilation
       >  - New methods in lazy collection properties
       >
       > For more details see https://docs.gradle.org/8.7/release-notes.html
       >
       > Starting a Gradle Daemon (subsequent builds will be faster)
       >
       > FAILURE: Build failed with an exception.
       >
       > * What went wrong:
       > A problem occurred configuring project ':applet'.
       > > Invalid JavaCard SDK path : /build/98zvpr6fis716zafyab1969rm3jg53rl-source/ext/sdks/jc320v24.0_kit
       >
       > * Try:
       > > Run with --stacktrace option to get the stack trace.
       > > Run with --info or --debug option to get more log output.
       > > Run with --scan to get full insights.
       > > Get more help at https://help.gradle.org.
       >
       > BUILD FAILED in 33s
       > 
       For full logs, run 'nix-store -l /nix/store/0f7iywj4wlkpbgdyl0xaj8p3x1xxcnbl-ECTesterStandalone-0.3.3.drv'.

[quapka]

@J08nY thanks for trying. You can run ECTesterStandalone directly with:

$ nix run '.?submodules=1' -- list-libs

Or you can build it and then run it with:

$ nix build '.?submodules=1'
$ ./result/bin/ECTesterStandalone list-libs

To specify a particular version of a lib:

$ nix run '.?submodules=1#openssl.v312' -- list-libs | grep OpenSSL
	- OpenSSL
		- Fullname: OpenSSL 3.1.2 1 Aug 2023
		- Fullname: OpenSSL 1.1.1 (compatible; BoringSSL)

[quapka]

You problem with:

       > * What went wrong:
       > A problem occurred configuring project ':applet'.
       > > Invalid JavaCard SDK path : /build/98zvpr6fis716zafyab1969rm3jg53rl-source/ext/sdks/jc320v24.0_kit

Is likely due to .?submodules=1 missing from the incantation. The submodules are still required, due to the SDKs. I think that it is fixable (meaning, there is a way to remove the dependency on the submodules), but it's/wasn't a priority.

[J08nY]

Cool, will try.

Any way to get rid of the extra command options? I looked and saw just a bunch of overlycomplicated ways that did not do what I wanted.

Also is it possible to build just a single library and not all of them?

[J08nY]

Ah ha, found it: https://discourse.nixos.org/t/error-experimental-nix-feature-nix-command-is-disabled/18089/3

[quapka]

I am currently testing the build on Aura and am hitting a problem with building libgpg-error - it fails on a single test

Cool, will try.

Any way to get rid of the extra command options? I looked and saw just a bunch of overlycomplicated ways that did not do what I wanted.

Also is it possible to build just a single library and not all of them?

Yes. Currently I build each shim separately and then copy all outputs, but I could also copy just the on specified - at least in theory :)

[quapka]

Edit: There is progress. I changed the hash and got something to build!

Further edit: I get:

That's unfortunate that you need to change the hash - ideally, this would not be the case. Fortunately, I have the same error when testing on Aura and can debug further.

[quapka]

Few more things I thought of:

• [ ] [Technicality] We shall be using apps instead of packages, see Flakes
• [ ] [Low prio] Add builders for other Gradle targets, such as applet and reader.

[J08nY]

Few more things I thought of:

• [ ] [Technicality] We shall be using apps instead of packages, see Flakes
• [ ] [Low prio] Add builders for other Gradle targets, such as applet and reader.

I would not actually do any of these. The second one especially, I would keep the applet and reader subprojects as they are.