rules_jsonnet

rules_jsonnet copied to clipboard

First of all, thanks for this great rule set. Makes my life a lot easier :+1:

We are using jsonnet to build grafana dashboards using the external dependency https://github.com/grafana/grafonnet-lib.

Currently, I did not find any documentation on how to do it with this rule set. After some playing around, I got a working example. (see below)

Now, is this the canonical way of doing this, or did I miss a simpler way? In particular, imports = ["external/com_github_grafana_grafonnet-lib"], looks a bit odd to me.

If It is the way to go, I would be happy to contribute some documentation on how to import external dependencies.

My solution: (https://github.com/Oncilla/rules_jsonnet_examples/tree/main/grafonnet)

# BUILD.bazel
jsonnet_to_json_test(
    name = "prometheus_test",
    src = "prometheus.jsonnet",
    golden = "prometheus_compiled.json",
    imports = ["external/com_github_grafana_grafonnet-lib"],
    deps = ["@com_github_grafana_grafonnet-lib//:grafonnet"],
)

# WORKSPACE
http_archive(
    name = "com_github_grafana_grafonnet-lib",
    build_file_content = """
load(
    "@io_bazel_rules_jsonnet//jsonnet:jsonnet.bzl",
    "jsonnet_library",
)
jsonnet_library(
    name = "grafonnet",
    srcs = glob(["grafonnet/*"]),
    visibility = ["//visibility:public"],
)
""",
    sha256 = "571ff5e6a44b583b8069476fb720f70a97a39734d206eae41c1c4f12af0a05d1",
    strip_prefix = "grafonnet-lib-0.1.0",
    urls = ["https://github.com/grafana/grafonnet-lib/archive/v0.1.0.zip"],
)

I was a bit too quick to judge. Turns out this worked because of a happy accident. For nested packages, this is not so straight forward.

Consider the following example: https://github.com/Oncilla/rules_jsonnet_examples/tree/main/grafonnet/nested

@rules_jsonnet_examples/grafonnet$ bazel test  //nested:prometheus
INFO: Analyzed target //nested:prometheus (0 packages loaded, 0 targets configured).
INFO: Found 1 target and 0 test targets...
ERROR: /home/roosd/go/src/github.com/oncilla/rules_jsonnet_examples/grafonnet/nested/BUILD.bazel:3:16: Compiling Jsonnet to JSON for prometheus failed: (Exit 1): bash failed: error executing command /bin/bash -c ... (remaining 1 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox bash failed: error executing command /bin/bash -c ... (remaining 1 argument(s) skipped)

Use --sandbox_debug to see verbose messages from the sandbox
RUNTIME ERROR: couldn't open import "grafonnet/grafana.libsonnet": no match locally or in the Jsonnet library paths
        nested/prometheus.jsonnet:3:17-53       thunk <grafana> from <$>
        nested/prometheus.jsonnet:4:19-26
        nested/prometheus.jsonnet:61:1-10
        During evaluation

Target //nested:prometheus failed to build

Verbose output shows that the following command is run.

/bin/bash -c 'set -e; bazel-out/host/bin/external/jsonnet_go/cmd/jsonnet/linux_amd64_stripped/jsonnet -J nested/external/com_github_grafana_grafonnet-lib -J . -J bazel-out/k8-fastbuild/bin -J bazel-out/k8-fastbuild/bin nested/prometheus.jsonnet -o bazel-out/k8-fastbuild/bin/nested/prometheus.json'

Thus, the problem is -J nested/external/com_github_grafana_grafonnet-lib which prepends the package path.

I was able to solve that in the //nested:prometheus_hack target, but the hack is pretty ugly, especially for deeply nested packages.

jsonnet_to_json(
    name = "prometheus_hack",
    src = "prometheus.jsonnet",
    imports = ["../external/com_github_grafana_grafonnet-lib"],
    deps = ["@com_github_grafana_grafonnet-lib//:grafonnet"],
    outs = ["prometheus_hack.json"],
)

So, looking forward to hearing what the proper way is :sweat_smile:

After a long fight I was able to get it working without the hacky ../ imports! rules_jsonnet handles transitive deps for you, so all you need to do is define the imports in the jsonnet_library definition in the grafonnet build file.

In my case I wanted to import grafonnet files as "grafonnet/grafana.libsonnet" from my jsonnet source, so I just added . to the imports for the grafonnet jsonnet_library target (I think just an empty string should work as well). Here is the build_file I'm using in the http_archive which is pulling in grafonnet:

load("@io_bazel_rules_jsonnet//jsonnet:jsonnet.bzl", "jsonnet_library")

jsonnet_library(
  name = "grafonnet",
  srcs = glob(["grafonnet/*.libsonnet"]),
  visibility = ["//visibility:public"],
  imports = ["."],
)

This will add the appropriate -J flag ( -J external/com_github_grafana_grafonnet-lib/.) for grafonnet whenever you use it as a deps in another jsonnet_library, jsonnet_to_json, etc. I was able to see these flags and debug by using the --sandbox_debug flag. With this I'm able to put a target like the following anywhere in my project:

load("@io_bazel_rules_jsonnet//jsonnet:jsonnet.bzl", "jsonnet_to_json")

jsonnet_to_json(
    name = "prometheus",
    src = "prometheus.jsonnet",
    outs = ["prometheus.json"],
    deps = [
      "@com_github_grafana_grafonnet-lib//:grafonnet",
    ],
)

@sonic-fast-boi this works. Thank you so much for this :+1:

I think this can be fixed with the following patch to this repo:

--- jsonnet/jsonnet.bzl
+++ jsonnet/jsonnet.bzl
@@ -50,7 +50,9 @@
 def _add_prefix_to_imports(label, imports):
     imports_prefix = ""
+    workspace_import = []
     if label.workspace_root:
         imports_prefix += label.workspace_root + "/"
+        workspace_import = [label.workspace_root]
     if label.package:
         imports_prefix += label.package + "/"
-    return [imports_prefix + im for im in imports]
+    return workspace_import + [imports_prefix + im for im in imports]

This "does the right thing" without needing to add imports = ["."] and also works if the desired library is not at the top level of the repository (imports = ["."] would then add -J external/some-repo/path/to/the/rule rather than -J external/some-repo). This solves the problem when using e.g. with https://github.com/yugui/jsonnetunit.

This has been fixed in #182. It no longer matters whether something is an external dependency or not. It can always be imported using a path relative to its workspace root, or relative to one of the provided imports. Thanks for reporting this issue!