Unclear how to add new dependencies

[kozross]

trafficstars

For this task here. I'm trying to add a dependency on secp256k1-haskell, which (to my best understanding), I've done here. However, when I try to spin up a nix-shell, I get this:

[koz@Sebastian cardano-base]$ nix-shell --show-trace
trace: Using IOHK default nixpkgs
trace: To make project.plan-nix for haskell-project a fixed-output derivation but not materialized, set `plan-sha256` to the output of the 'calculateMaterializedSha' script in 'passthru'.
trace: To materialize project.plan-nix for haskell-project entirely, pass a writable path as the `materialized` argument and run the 'updateMaterialized' script in 'passthru'.
trace: WARNING: No sha256 found for source-repository-package https://github.com/haskoin/secp256k1-haskell 885d0953d5a0fb08be181a80f3643b0135b04bcb download may fail in restricted mode (hydra)
fatal: couldn't find remote ref refs/heads/885d0953d5a0fb08be181a80f3643b0135b04bcb
error: program 'git' failed with exit code 128

       … while fetching the input 'git+https://github.com/haskoin/secp256k1-haskell?ref=885d0953d5a0fb08be181a80f3643b0135b04bcb'

       … while evaluating the attribute 'buildCommand' of the derivation 'hash-path'

       at /nix/store/g9r92jfark3sngrh57nnbbpjx6nbkfjg-nixpkgs-src/pkgs/build-support/trivial-builders.nix:7:7:

            6|     stdenv.mkDerivation ({
            7|       name = lib.strings.sanitizeDerivationName name;
             |       ^
            8|       inherit buildCommand;

       … while evaluating 'hashPath'

       at /nix/store/5zvkchh00rdd0g89cgky4h29p4d4q8c6-haskell.nix-src/lib/call-cabal-project-to-nix.nix:171:14:

          170|
          171|   hashPath = path:
             |              ^
          172|     builtins.readFile (pkgs.runCommand "hash-path" { preferLocalBuild = true; }

       … from call site

       at /nix/store/5zvkchh00rdd0g89cgky4h29p4d4q8c6-haskell.nix-src/lib/call-cabal-project-to-nix.nix:185:56:

          184|               in __trace "WARNING: No sha256 found for source-repository-package ${repoData.url} ${repoData.ref} download may fail in restricted mode (hydra)"
          185|                 (__trace "Consider adding `--sha256: ${hashPath drv}` to the ${cabalProjectFileName} file or passing in a lookupSha256 argument"
             |                                                        ^
          186|                  drv);

       … while evaluating the attribute 'buildCommand' of the derivation 'source-repository-package'

       at /nix/store/g9r92jfark3sngrh57nnbbpjx6nbkfjg-nixpkgs-src/pkgs/build-support/trivial-builders.nix:7:7:

            6|     stdenv.mkDerivation ({
            7|       name = lib.strings.sanitizeDerivationName name;
             |       ^
            8|       inherit buildCommand;

       … while evaluating anonymous lambda

       at /nix/store/5zvkchh00rdd0g89cgky4h29p4d4q8c6-haskell.nix-src/lib/call-cabal-project-to-nix.nix:236:46:

          235|         # Add replacement `source-repository-package` blocks pointing to the minimal git repos
          236|         ( pkgs.lib.strings.concatMapStrings (f: ''
             |                                              ^
          237|               echo "source-repository-package" >> ./cabal.project

       … from call site

       … while evaluating 'concatMapStrings'

       at /nix/store/g9r92jfark3sngrh57nnbbpjx6nbkfjg-nixpkgs-src/lib/strings.nix:53:25:

           52|   */
           53|   concatMapStrings = f: list: concatStrings (map f list);
             |                         ^
           54|

       … from call site

       at /nix/store/5zvkchh00rdd0g89cgky4h29p4d4q8c6-haskell.nix-src/lib/call-cabal-project-to-nix.nix:236:11:

          235|         # Add replacement `source-repository-package` blocks pointing to the minimal git repos
          236|         ( pkgs.lib.strings.concatMapStrings (f: ''
             |           ^
          237|               echo "source-repository-package" >> ./cabal.project

       … while evaluating 'optionalString'

       at /nix/store/g9r92jfark3sngrh57nnbbpjx6nbkfjg-nixpkgs-src/lib/strings.nix:202:5:

          201|     # String to return if condition is true
          202|     string: if cond then string else "";
             |     ^
          203|

       … from call site

       at /nix/store/5zvkchh00rdd0g89cgky4h29p4d4q8c6-haskell.nix-src/lib/call-cabal-project-to-nix.nix:230:9:

          229|       '' +
          230|         pkgs.lib.optionalString (builtins.length sourceReposEval != 0) (''
             |         ^
          231|         chmod +w -R ./cabal.project

       … while evaluating the attribute 'makeFixedProjectFile'

       at /nix/store/5zvkchh00rdd0g89cgky4h29p4d4q8c6-haskell.nix-src/lib/call-cabal-project-to-nix.nix:227:7:

          226|       sourceRepos = sourceReposBuild;
          227|       makeFixedProjectFile = ''
             |       ^
          228|         cp -f ${otherText} ./cabal.project

       … while evaluating the attribute 'buildCommand' of the derivation 'haskell-project-plan-to-nix-pkgs'

       at /nix/store/g9r92jfark3sngrh57nnbbpjx6nbkfjg-nixpkgs-src/pkgs/build-support/trivial-builders.nix:7:7:

            6|     stdenv.mkDerivation ({
            7|       name = lib.strings.sanitizeDerivationName name;
             |       ^
            8|       inherit buildCommand;

       … while evaluating anonymous lambda

       at /nix/store/5zvkchh00rdd0g89cgky4h29p4d4q8c6-haskell.nix-src/lib/import-and-filter-project.nix:5:1:

            4| { pkgs, haskellLib }:
            5| { projectNix, sourceRepos, src }:
             | ^
            6| let

       … from call site

       at /nix/store/5zvkchh00rdd0g89cgky4h29p4d4q8c6-haskell.nix-src/overlays/haskell.nix:488:27:

          487|               callProjectResults = callCabalProjectToNix args;
          488|               plan-pkgs = importAndFilterProject {
             |                           ^
          489|                 inherit (callProjectResults) projectNix sourceRepos src;

       … while evaluating the attribute 'hsPkgs'

       at /nix/store/5zvkchh00rdd0g89cgky4h29p4d4q8c6-haskell.nix-src/overlays/haskell.nix:510:43:

          509|               project = addProjectAndPackageAttrs rec {
          510|                   inherit (pkg-set.config) hsPkgs;
             |                                           ^
          511|                   inherit pkg-set;

       … while evaluating the attribute 'hsPkgs'

       at /nix/store/5zvkchh00rdd0g89cgky4h29p4d4q8c6-haskell.nix-src/overlays/haskell.nix:533:15:

          532|               # Haskell packages
          533|               hsPkgs = final.lib.mapAttrs (packageName: package':
             |               ^
          534|                 if package' == null

       … while evaluating 'cabalProject'

       at /nix/store/5zvkchh00rdd0g89cgky4h29p4d4q8c6-haskell.nix-src/overlays/haskell.nix:752:24:

          751|
          752|         cabalProject = args: let p = cabalProject' args;
             |                        ^
          753|             in p.hsPkgs // p;

       … from call site

       at /home/koz/mlabs/cardano-base/nix/haskell.nix:23:12:

           22|   # https://input-output-hk.github.io/haskell.nix/user-guide/projects/
           23|   pkgSet = haskell-nix.cabalProject {
             |            ^
           24|     inherit src;

       … while evaluating anonymous lambda

       at /home/koz/mlabs/cardano-base/nix/haskell.nix:4:1:

            3| ############################################################################
            4| { lib
             | ^
            5| , stdenv

       … from call site

       at /home/koz/mlabs/cardano-base/nix/pkgs.nix:3:32:

            2| pkgs: _: with pkgs; {
            3|   cardanoBaseHaskellPackages = import ./haskell.nix {
             |                                ^
            4|     inherit config

       … while evaluating anonymous lambda

       at /home/koz/mlabs/cardano-base/shell.nix:2:1:

            1| # This file is used by nix-shell.
            2| { config ? {}
             | ^
            3| , sourcesOverride ? {}

       … from call site

It isn't clear to me how new dependencies are to be added - the URL that is specified by --show-detail definitely exists, and the commit is live. What have I missed?

[michaelpj]

You don't need to add it to sources.json, you do need to add the sha256 line in cabal.project (for haskell.nix). You might also have an issue if the sha you're asking for isn't on the master branch?

(This project needs a CONTRIBUTING.md)

[michaelpj]

You may also need to teach haskell.nix about the pkg-config dependency mapping, like this, but for the secp256k1 library: https://github.com/input-output-hk/haskell.nix/pull/1295

[kozross]

@michaelpj I've updated my branch, but I'm still seeing the same issue. I'm also not sure how to determine what should go in the sha256 section of cabal.project.

[nc6]

Note that the sha256 field should be commented, as in the other source-repository-package stanzas.

[michaelpj]

You should be able to do the usual TOFU dance: Nix will tell you if it doesn't match. Also what Nick said.

[kozross]

@michaelpj @nc6 Commenting the sha256 field worked; this definitely needs documenting for contributors, as I've literally never seen this before.

You may also need to teach haskell.nix about the pkg-config dependency mapping, like this, but for the secp256k1 library: input-output-hk/haskell.nix#1295

How would I do this? There's no equivalent file here.

[michaelpj]

https://input-output-hk.github.io/haskell.nix/tutorials/pkg-map.html

In the short term you can put in a nixpkgs overlay providing libsecp256k1 = secp256k1. Ultimately I think it'll need a PR to haskell.nix so that downstream users have an easy time too.

[michaelpj]

Oh, ignore me, it's already there, so that should be fine.

[kozross]

@michaelpj I now get this error on a build attempt, so clearly something isn't there:

Resolving dependencies...
cabal: Could not resolve dependencies:
[__0] next goal: secp256k1-haskell (user goal)
[__0] rejecting: secp256k1-haskell-0.6.0 (conflict: pkg-config package
libsecp256k1-any, not found in the pkg-config database)
[__0] rejecting: secp256k1-haskell-0.5.0, secp256k1-haskell-0.4.0,
secp256k1-haskell-0.3.1, secp256k1-haskell-0.3.0, secp256k1-haskell-0.2.5,
secp256k1-haskell-0.2.4, secp256k1-haskell-0.2.3, secp256k1-haskell-0.2.2,
secp256k1-haskell-0.2.1, secp256k1-haskell-0.2.0, secp256k1-haskell-0.1.8,
secp256k1-haskell-0.1.7, secp256k1-haskell-0.1.6, secp256k1-haskell-0.1.5,
secp256k1-haskell-0.1.4, secp256k1-haskell-0.1.3, secp256k1-haskell-0.1.2,
secp256k1-haskell-0.1.1, secp256k1-haskell-0.1.0 (constraint from user target
requires ==0.6.0)
[__0] fail (backjumping, conflict set: secp256k1-haskell)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: secp256k1-haskell

[michaelpj]

I think it's easier than you think. I just added the dep to cardano-crypto-class (it's on hackage!) and it seems to work fine: https://github.com/input-output-hk/cardano-base/tree/koz/secp-256k1

[michaelpj]

So you never needed to do the source-repository-package thing, I think.

[kozross]

Good to know, thanks!