Automatic formatting of Pkl files

[sin-ack]

Problem Statement

Currently, there doesn't seem to be any way to automatically format a Pkl file. This means that we cannot enforce a standard style when collaborating in a team, nor automatically flag violations in PRs.

Proposed Solution

Something like pkl format, replicating the CLI of Prettier. Examples:

# Take in a file from stdin, and output it as formatted Pkl code to stdout
$ pkl format <<EOF
> foo { bar = 1; baz = 2 }
> EOF
foo {
  bar = 1
  baz = 2
}
# Take a filename, read its contents and output it as formatted Pkl code to stdout
$ echo "foo { bar = 1; baz = 2 }" > foo.pkl
$ pkl format foo.pkl
foo {
  bar = 1
  baz = 2
}
# Take a filename, read its contents and write it as formatted Pkl code to the specified file
$ pkl format foo.pkl --output=formatted.pkl # or -o
$ cat formatted.pkl
foo {
  bar = 1
  baz = 2
}
# Take one or more filenames, and exit with 0 if it is correctly formatted, or 1 otherwise
$ pkl format --check formatted.pkl; echo $?
0
$ pkl format --check foo.pkl; echo $?
Error: foo.pkl is not correctly formatted
1
$ pkl format --check foo.pkl formatted.pkl; echo $?
Error: foo.pkl is not correctly formatted
1
# Take one or more filenames, format them and write to the original files
$ pkl format --write foo.pkl formatted.pkl
Formatted foo.pkl
formatted.pkl was already formatted
$ cat foo.pkl
foo {
  bar = 1
  baz = 2
}

Open Questions

• [ ] Should it be configurable? (I'm personally fond of the "nobody's favorite" approach to code formatting lately.)
• [ ] When to use multiple lines, and when to use a single line with ; separators?
• [ ] Default indentation width? (Suggestion: 2 as used in the documentation)
• [ ] Tabs or spaces? (Suggestion: Spaces as used in the documentation)

Relevant Links

Discussion in February: https://github.com/apple/pkl/discussions/174

[holzensp]

there doesn't seem to be any way to automatically format a Pkl file

This isn't technically true; pkl-intellij has a formatter. Sadly, that's far too deeply integrated with the IDE SDKs to use in CI and VCS hooks and such.

The current front-end is slightly too deeply integrated with the interpreter. We're working on improving that, so that this kind of tool is easier to build/maintain. It's certainly on our radar.

[yannham]

Hello!

I want to mention that we had frustrations when developing a formatter for the Nickel configuration language. Our parsing infrastructure wasn't really geared toward formatting (which requires more information in the output, i.e. a CST instead of an AST), and wiping the parser entirely for something different sounded like a lot of work.

So we developed Topiary, which is a stand-alone multi-language formatter based on tree-sitter. That is, if you already have a tree-sitter grammar lying around for editor highlighting (or you plan to write one), you can reuse it to create a formatter by writing additional specific tree sitter queries (here is the formatting queries for Nickel for example - it's 500 lines but with many comments and lists with one element per line). On caveat is that the grammar must be fine-grained enough: for highlighting, one can get away with a very coarse grammar specification, but formatting might be more demanding.

Topiary can then be used as a separate tool for formatting, or even embedded as e.g. pkl format (this is currently the case for nickel format which just links to Topiary under the hood). Although I must say, even if Rust is rather easy to FFI into because of its absence of runtime, I don't how well that would work with a Java codebase like Pkl (especially because Topiary also binds to tree-sitter, which means at least bringing C into the picture, and maybe C++, I don't remember exactly).

I hope it's not out of place or that it doesn't sound too much like advertising. It's just that one motivation of Topiary is to create formatters for new languages without too much hassle (by delegating parsing and pretty-printing to tree-sitter and just focusing on transforming the source tree, under the assumption that you'll have to write a tree-sitter grammar anyway for editor support), so it might be a possible direction to explore for Pkl :slightly_smiling_face:

[bioball]

Yann: thanks for the insight! This looks pretty interesting; and, we already have a tree-sitter grammar, so it should be simple enough for us to play around with Topiary and see how it works.

You're right in calling out that Java bindings for tree-sitter is a question mark. There's a couple libraries out there, but they either involve binding through JNI (e.g. java-tree-sitter), which introduces its own pain points, or other interesting libraries that aren't actively maintained (e.g. jsitter).