Cannot run pandoc via flake.nix on NixOS

[disbolog]

Explain the problem. Trying to run pandoc with nix develop on flake.nix which is available in the root of the repo and get the following error:

user@host ~/pandoc (main)> nix develop
trace: WARNING[haskell-flake]: Multiple haskell overlays are applied in arbitrary order.
error: anonymous function at /nix/store/y2p2al2c5gy06jsxa7jlnn0pyjk86ly4-cabal2nix-pandoc-lua-engine/default.nix:1:1 called without required argument 'hslua-repl'

       at /nix/store/bhi4zh9nh9h3i206sxpp0anr6j1kck5r-source/pkgs/development/haskell-modules/make-package-set.nix:97:40:

           96|       # this wraps the `drv` function to add `scope` and `overrideScope` to the result.
           97|       drvScope = allArgs: ensureAttrs (drv allArgs) // {
             |                                        ^
           98|         inherit scope;
(use '--show-trace' to show detailed location information)

There was a similar (maybe not?) issue from 2019 which is now closed, but I'm lacking expertise to say if it is related/helpful/has a solution in it: https://github.com/jgm/pandoc/issues/5298

Pandoc version?

Master branch of pandoc.

This is on NixOS, with the following configuration:

 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.26, NixOS, 23.05 (Stoat), 23.05.20230430.da45bf6`
 - multi-user?: `no`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.13.3`
 - channels(root): `""`
 - nixpkgs: `not found`

[jgm]

Yes, I know. This has been true for some time and I don't know enough about NiX to fix it. Help welcome!

[jgm]

I've also tried nix flake update and then nix develop; this yields:

error: Package ‘hslua-repl-0.1.1’ in /nix/store/15sa051fs1iq3vymyb8076l9fy3mwj3x-source/pkgs/development/haskell-modules/hackage-packages.nix:152686 is marked as broken, refusing to evaluate.

       a) To temporarily allow broken packages, you can use an environment variable
          for a single invocation of the nix tools.

            $ export NIXPKGS_ALLOW_BROKEN=1

        Note: For `nix shell`, `nix build`, `nix develop` or any other Nix 2.4+
        (Flake) command, `--impure` must be passed in order to read this
        environment variable.

       b) For `nixos-rebuild` you can set
         { nixpkgs.config.allowBroken = true; }
       in configuration.nix to override this.

       c) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
         { allowBroken = true; }
       to ~/.config/nixpkgs/config.nix.
(use '--show-trace' to show detailed location information)

So I guess hslua-repl is broken, but I don't know why or how to fix that.

[jgm]

@peti you have submitted issues before about pandoc and NiX - can you offer any illumination about why hslua-repl is broken and how it can be repaired? It has been broken for a long time.

[tarleb]

Bug reports or fixes are welcome: https://github.com/hslua/hslua

[sternenseemann]

why hslua-repl is broken and how it can be repaired?

hslua-repl only support lua >= 2.3 and hslua-core >= 2.3; while Stackage LTS 20 pins them both to 2.2.*, hence the failure. This can be worked around for specific packages (as pandoc) by manually upgrading lua and hslua-core etc.

From reading through the attempts in https://github.com/NixOS/nixpkgs/issues/221165, I'd expect pandoc to quite a big overlay, since it seems to rely on newer version of quite a few packages that are tracked in Stackage. Things will likely get easier when Stackage LTS 21 gets released and nixpkgs switches to it.

[tg-x]

After the last change to flake.nix I get the following error from nix build:

Configuring pandoc-3.1.3...

Setup: Encountered missing or private dependencies:
doctemplates >=0.11 && <0.12,
gridtables >=0.1 && <0.2,
jira-wiki-markup >=1.5.1 && <1.6,
mime-types >=0.1.1 && <0.2,
pandoc-types >=1.23 && <1.24,
texmath >=0.12.8 && <0.13,
typst >=0.1 && <0.2

[ysangkok]

Stackage LTS 21 gets released and nixpkgs switches to it.

This has happened now, but pandoc was thrown out of Stackage Nightly because it uses connection instead of crypton-connection. But this issue was addressed in

• https://github.com/jgm/pandoc/pull/8896

Not sure whether Nightly affects Nix at all... Might be able to get pandoc-3.1 working with stack2nix once it gets into Nightly?

[maralorn]

There is a working pandoc-3.1 on the haskell-updates branch of nixpkgs right now. The easiest solution would be to wait until that has been merged into master, bump nixpkgs in this flake and potential add overrides accordingly, which should be easy because you can just take copy them out of nixpkgs.

[srid]

@jgm I'm happy to create a working flake.nix (using haskell-flake). I made an attempt here, but it looks like I'm gonna wait for https://github.com/NixOS/nixpkgs/pull/240387 to be merged.

[jgm]

@srid thanks - I have NO idea what I'm doing with nix.

[milahu]

workaround: use a binary build of pandoc, see my pandoc-bin.nix

[alerque]

@srid Any progress? The bit you said you were waiting on is merged now. I know just enough Nix to be dangerous and not enough Haskell to get me out of trouble here, but I have a use case for the Pandoc Flake to be working again and would be happy to help if I can.

[stephen-huan]

@alerque does the following work for your use case? https://github.com/jgm/pandoc/compare/main...stephen-huan:pandoc:fix-flake, corresponding to the diff

diff --git a/flake.lock b/flake.lock
index cb4c63efa..0ca54945b 100644
--- a/flake.lock
+++ b/flake.lock
@@ -5,11 +5,11 @@
         "systems": "systems"
       },
       "locked": {
-        "lastModified": 1681202837,
-        "narHash": "sha256-H+Rh19JDwRtpVPAWp64F+rlEtxUWBAQW28eAi3SRSzg=",
+        "lastModified": 1710146030,
+        "narHash": "sha256-SZ5L6eA7HJ/nmkzGG7/ISclqe6oZdOZTNoesiInkXPQ=",
         "owner": "numtide",
         "repo": "flake-utils",
-        "rev": "cfacdce06f30d2b68473a46042957675eebb3401",
+        "rev": "b1d9ab70662946ef0850d488da1c9019f3a9752a",
         "type": "github"
       },
       "original": {
@@ -20,11 +20,11 @@
     },
     "nixpkgs": {
       "locked": {
-        "lastModified": 1683594133,
-        "narHash": "sha256-iUhLhEAgOCnexSGDsYT2ouydis09uDoNzM7UC685XGE=",
+        "lastModified": 1719468428,
+        "narHash": "sha256-vN5xJAZ4UGREEglh3lfbbkIj+MPEYMuqewMn4atZFaQ=",
         "owner": "NixOS",
         "repo": "nixpkgs",
-        "rev": "8d447c5626cfefb9b129d5b30103344377fe09bc",
+        "rev": "1e3deb3d8a86a870d925760db1a5adecc64d329d",
         "type": "github"
       },
       "original": {
diff --git a/flake.nix b/flake.nix
index 9e7908765..c8f61d34a 100644
--- a/flake.nix
+++ b/flake.nix
@@ -7,18 +7,6 @@
   outputs = inputs:
     let
       overlay = final: prev: {
-        haskell = prev.haskell // {
-          packageOverrides = hfinal: hprev:
-            prev.haskell.packageOverrides hfinal hprev // {
-              crypton-connection = final.fetchFromGitHub {
-                owner = "kazu-yamamoto";
-                repo = "crypton-connection";
-                rev = "5c064b911e7327a4d399fd9dd057663d0d0fb256";
-                sha256 = "00j1nf9glbz0cnzd84vp08j9ybzjbm3b6gcamlqxxcjb31kllz4b";
-              };
-              pandoc = hfinal.callCabal2nix "pandoc" ./. { };
-            };
-        };
         pandoc = final.haskell.lib.compose.justStaticExecutables final.haskellPackages.pandoc;
       };
       perSystem = system:

Currently I get the error

error:
       … while evaluating the attribute 'devShell'

         at /nix/store/gins1fk44x6jq7iambhr8hmal2na2lns-source/flake.nix:30:11:

           29|         {
           30|           devShell = hspkgs.shellFor {
             |           ^
           31|             withHoogle = true;

       … while evaluating a branch condition

         at /nix/store/6gavszahnavcadcw3b85vnsmlxzv503v-source/lib/customisation.nix:92:7:

           91|     in
           92|       if builtins.isAttrs result then
             |       ^
           93|         result // {

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: function 'anonymous lambda' called without required argument 'crypton-x509-system'

       at /nix/store/gqc4k73isabkpy2fdcg9ciihhvxgnpx0-cabal2nix-pandoc/default.nix:1:1:

            1| { mkDerivation, aeson, aeson-pretty, array, attoparsec, base
             | ^
            2| , base64-bytestring, binary, blaze-html, blaze-markup, bytestring

and indeed pkgs.haskellPackages.crypton-x509-system is missing on this version of nixpkgs. However, the latest unstable has this package, so I simply updated and removed the pin on crypton-connection.

To get the environment that pandoc is built in, one can also just run nix develop nixpkgs#haskellPackages.pandoc.

[srid]

I took a stab at it. #9952

@alerque

[alerque]

Thanks @srid. I've been off on other yak shaves for a while, but I did see this ping and am excited to try it out now that I have a chance to work on books.