jank-lang
Jank as a system scripting language?
Python, janet, guile, and other system scripting languages have an environment variable that contains paths to libraries. They also have the default path configured at compilation time.
Janet has jpm. Python has standards for project management tools that can be used by operating systems or in individual projects. Gentoo Linux uses python for its own high-level system tools.
Perl and raku are cousins, and they were explicitly designed as system scripting languages. Raku has separate directories for linux distribution libraries, user libraries, and system libraries. Raku is really unique because it supports installation of multiple versions of the same library and installation of the same version of the library by different authors.
Janet is not my favorite language, but it is simple and very useful for system scripting and high-level system applications. It also has fibers and channels for Go-style concurrency. It's similar to clojure core.async. Janet's concurrency constructs were used by me in j3blocks. For now, I just use janet because it is simple and useful enough for my system scripting needs. It seems clojure is largely used for line-of-business applications instead of system applications.
An entire linux operating system(GNU guix system) was written in guile because guile is a system scripting language. Janet can do the same, but it hasn't been used that way.
As far as I know, racket is also a system scripting language.
Babashka is a project programming language that can have dependencies in a project directory, but it can't have system libraries or user libraries. It can only produce uberscripts and uberjars which can be run by babashka. I would classify babashka as a portable project scripting language.
If basilisp(clojure on python VM) gains PEP517 build backend with a project management tool, it can become a system scripting language, but I haven't yet seen clojure dialect as a system scripting language.
Jank is not becoming a system scripting language, but it could be. What will jank become? For now, it seems jank doesn't have specific deployment methods. ClojureScript lives in a project directory. Clojure can live in a project directory and also produce uberjars. Unless you are aiming for portable deployment and web deployment, uberjars and project scripting won't be very helpful? I personally hope jank was able to load different version of the same library from a user library directory or a system library directory or a project directory. Haskell cabal installs libraries into a user cache directory and uses cached libraries in projects, and cabal can also install haskell system libraries.
What do you imagine jank would need in order to become a system scripting language? Can you compile this into a concise list of requirements so we can determine if/when jank can get there?
- Standard library path determined at compilation time if the standard library has to be installed as files.
- For example, /usr/{share,lib}/jank/core (/usr can be an entirely different path in GNU guix system or NixOS)
- Third-party library path determined at compilation time
- For example, /usr/{share,lib}/jank/site (/usr can be an entirely different path in GNU guix system or NixOS)
- An environment variable that replaces one third-party library path with multiple third-party library paths. This should only be used by operating system. If users want to play around with dependencies without root permission, they can play around in projects. The name for the environment variable may be
JANK_SITE_PACKAGES_PATH.- GNU guix system and NixOS will inject library paths via the environment variable because they install libraries in multiple different directories. These functional OSes don't use standard paths like /usr/share.
- Jank can fetch dependencies from the standard library path and third-party library path(s). Jank executable doesn't care about project as a concept, so it can execute any script inside a project directory like a system script.
- In a project directory, jank package manager has access to the standard library path and the project library path(e.g.,
path-to-project/deps). It doesn't use third-party library path(s) in a project directory. - A jank project declaration "could" specify git repositories for dependencies so that jank doesn't require a centralized library repository.
- During system installation, jank package manager refers to the standard library path and the third-party library path(s) but installs onto the specified library path.
- If you allow installation of multiple versions of a library and loading one version among many, then for simplicity and compatibility with clojure, version constraints may be present only in project declaration. Then, bare system scripts outside project directories may just pick the latest versions possible from the third-party library path(s) or the version in the first library path that has the library. If you allow a single script to specify version constraints, then it is a slight deviation from clojure.
- I want a version range and also a specific version for version constraints.
- If you disallow installation of multiple versions of a library, then a specified library will be picked from the first library path that has it, or the highest possible version of the library will be picked from all library paths.
- Once a jank project or a jank script is compiled as a binary, build-time jank library dependencies can be deleted. A binary can be built at the project level, or it can be built and installed at the system level.
Advanced considerations
- Jank package manager can perhaps use a local user cache(e.g., ~/.cache/jank) shared by all projects instead of project library path if jank supports installation of multiple versions of a library and loading one among the multiple versions.
- If a user library path(e.g., ~/.local/{share,lib}/jank) is determined at compilation time and jank suports installation of multiple library versions and picking one among them, perhaps system scripts and jank package manager can share user library path and also have access to third party library path(s) for system-level libraries, and jank package manager won't need project library path for installing project libraries without root permission.
Thanks for taking the time to build this exhaustive list!
Standard library path determined at compilation time if the standard library has to be installed as files.
* For example, /usr/{share,lib}/jank/core (/usr can be an entirely different path in GNU guix system or NixOS)
Third-party library path determined at compilation time
- For example, /usr/{share,lib}/jank/site (/usr can be an entirely different path in GNU guix system or NixOS)
jank uses maven for package management, but it also will support per-package build scripts. The build scripts will have a few different goals:
- Find system packages, libraries, includes, etc
- Build native packages from source
- Generate files
- Propagate all of this information (library paths, include paths, defines, etc) back up to jank
The build script is just a top-level build.clj in each maven package. jank's leiningen plugin will run all of these automatically for you, using babashka, and will cache the build results. This is very much modeled after Rust's build scripts and it should give us all the flexibility we need.
* An environment variable that replaces one third-party library path with multiple third-party library paths. This should only be used by operating system. If users want to play around with dependencies without root permission, they can play around in projects. The name for the environment variable may be `JANK_SITE_PACKAGES_PATH`.
We delegate to leiningen for this.
* GNU guix system and NixOS will inject library paths via the environment variable because they install libraries in multiple different directories. These functional OSes don't use standard paths like /usr/share.
jank will rely on build scripts to resolve this as necessary, but it will provide a jank.build namespace with common operations, like searching with pkg-config or building a CMake project from source.
* Jank can fetch dependencies from the standard library path and third-party library path(s). Jank executable doesn't care about project as a concept, so it can execute any script inside a project directory like a system script.
The jank binary doesn't care about projects. The lein-jank plugin handles that.
* In a project directory, jank package manager has access to the standard library path and the project library path(e.g., `path-to-project/deps`). It doesn't use third-party library path(s) in a project directory.
leiningen projects are self-contained and will only refer to libraries defined in your project.clj or in your leiningen profile.
* A jank project declaration "could" specify git repositories for dependencies so that jank doesn't require a centralized library repository.
leiningen supports this.
* During system installation, jank package manager refers to the standard library path and the third-party library path(s) but installs onto the specified library path.
I don't understand what you mean here. leiningen will not install system packages; packages are installed into ~/.m2 and cached native builds are per-project.
* If you allow installation of multiple versions of a library and loading one version among many, then for simplicity and compatibility with clojure, version constraints may be present only in project declaration. Then, bare system scripts outside project directories may just pick the latest versions possible from the third-party library path(s) or the version in the first library path that has the library. If you allow a single script to specify version constraints, then it is a slight deviation from clojure.
leiningen already has a way to handle versioning and we just use that.
* I want a version range and also a specific version for version constraints.
leiningen supports this.
* If you disallow installation of multiple versions of a library, then a specified library will be picked from the first library path that has it, or the highest possible version of the library will be picked from all library paths.
This is supported.
* Once a jank project or a jank script is compiled as a binary, build-time jank library dependencies can be deleted. A binary can be built at the project level, or it can be built and installed at the system level.
jank AOT binaries will have two choices for a runtime:
- Dynamic runtime, which statically links to jank and dynamically links to Clang/LLVM; still has fully JIT capabilities, REPL support, eval, etc
- Static runtime, which statically links to jank, but not Clang/LLVM; no JIT capabilities, no REPL, no eval, etc (like a Graal native image)
Advanced considerations
* Jank package manager can perhaps use a local user cache(e.g., ~/.cache/jank) shared by all projects instead of project library path if jank supports installation of multiple versions of a library and loading one among the multiple versions.
leiningen handles this by downloading the immutable maven deps to ~/.m2. However, build script outputs for building native deps are cached at a project level since they can inherit the project's optimization settings and other flags.
* If a user library path(e.g., ~/.local/{share,lib}/jank) is determined at compilation time and jank suports installation of multiple library versions and picking one among them, perhaps system scripts and jank package manager can share user library path and also have access to third party library path(s) for system-level libraries, and jank package manager won't need project library path for installing project libraries without root permission.
leiningen will never need root permission, since we will not be installing packages globally.
Hopefully this gives you a better idea of the intentions for jank and the various use cases we can accommodate.
I was hoping that you were going to write a dedicated jank package manager since you claimed you were going to work on jank full-time.
If there was a medici for jank, perhaps jank will get a dedicated jank package manager, but I'm not your medici or rothschild unfortunately.
I guess that makes jank a project scripting language and a compiled (system?) programming language for now.
I was hoping that you were going to write a dedicated jank package manager since you claimed you were going to work on jank full-time.
Indeed, I have working on jank full-time this year, with no stable income, but my priority is getting jank launched. That includes Clojure parity, unprecedented seamless C++ interop, best in class compiler error reporting, a custom build system, AOT compilation to two different runtimes, and all sorts of native portability and distribution work. Building a custom package manager, when leiningen works, is very low priority compared to everything else. I don't see that changing any time soon.
Given the huge scope of jank, and my limited resources, I need to be brutally pragmatic when it comes to prioritization.
I decided to give up on clojure as a system scripting language. I will keep using janet as my system scripting language.
I guess clojure's niches are commercial backends and web applications.