jaq icon indicating copy to clipboard operation
jaq copied to clipboard

Improve nested module loading

Open mklein994 opened this issue 11 months ago • 1 comments

Are there any plans to improve the module loading process? Currently nested modules don't seem to work how I'd like:

~/.jq-test/env.jq:

module {
  name: "env",
  description: "Define common environment variables for ease-of-use"
};

def foo: env.FOO | @json;

~/.jq-test/prelude.jq:

module {
  name: "prelude",
  description: "Import global functions and modules"
};

import "env" as e;

test-script.sh:

#!/bin/bash

jq_bin="$1"
export FOO=42

echo "jq_bin: ${jq_bin}"
${jq_bin} -r -n '"direct: \(env.FOO | @json)"'
${jq_bin} -r -n -L ~/.jq-test 'include "env"; "include: \(foo)"'
${jq_bin} -r -n -L ~/.jq-test 'import "env" as e; "import: \(e::foo)"'
${jq_bin} -r -n -L ~/.jq-test 'include "prelude"; "prelude: \(e::foo)"'
$ ./test-script.sh gojq
jq_bin: gojq
direct: "42"
include: "42"
import: "42"
prelude: "42"
$ ./test-script.sh jq
jq_bin: jq
direct: "42"
include: "42"
import: "42"
jq: error: e::foo/0 is not defined at <top-level>, line 1:
include "prelude"; "prelude: \(e::foo)"                               
jq: 1 compile error
$ ./test-script.sh jaq
jq_bin: jaq
direct: "42"
include: "42"
import: "42"
Error: undefined module
  ╭─[<inline>]
  │
1 │ include "prelude"; "prelude: \(e::foo)"
  ┆                                ┬       
  ┆                                │       
  ┆                                ╰──────── undefined module
──╯

I understand if you want to go only as far as what jq is capable of, but I consider gojq as the more useful implementation in this case.

Or maybe I'm just holding it wrong. In any case, I'd like to hear your thoughts.


$ jaq --version
jaq 2.0.0
$ gojq --version
gojq 0.12.17 (rev: HEAD/go1.23.2)
$ jq --version
jq-1.7.1

mklein994 avatar Jan 11 '25 22:01 mklein994

What do the jq docs say about include?

Imports a module found at the given path relative to a directory in a search path as if it were included in place. [...] The module's symbols are imported into the caller's namespace as if the module's content had been included directly.

This leads us to another question: What is a symbol in jq? The scoping section has answers:

There are two types of symbols in jq: value bindings (a.k.a., "variables"), and functions.

IMO, that means that include imports the value bindings and functions that are defined in a module, but not those that the module imports. (Unfortunately, the docs do not really specify what the "module's symbols" actually are; i.e. does that include symbols that have been included themselves? But experimentation shows that jq excludes symbols that have been included. Which somehow makes sense, because otherwise, it would become quite tricky to know where a certain symbol has been actually defined.)

It is certainly debatable whether jq's behaviour is a good choice, as I can see your prelude use case. But I think that this issue should be discussed in the jq mainstream, as I try not to diverge from jq if I do not see a very compelling reason to do so (float vs. int). And at least in this particular case, the documentation is relatively clear.

TL;DR: If you would like to see your desired behaviour implemented in jaq, I advise you to propose it to jq upstream first.

01mf02 avatar Jan 14 '25 08:01 01mf02