CLI spec for alternate backends

[f-f]

In #426 we added basic support for alternate backends.

As discussed in https://github.com/spacchetti/spago/issues/355 we should write down a "CLI spec" that alternate backends can conform to so that they'd work with Spago out of the box.

[csicar]

First idea to get the conversation started:

Option	Default	Description
no option	-	generates files in the target language and saves them in the directory ??? and build the executable (or library) in the directory ???
--src	src/	sets the directories where the purescript source files are located (used for finding FFI files)
-o	output/	sets the directory, where the target files will be written to
--run	Main.main	builds & runs the supplied function. The command expects the function to have type :: Effect Unit
--no-build	-	skips building an executable and only generate files in the target language.

Notes:

• it might be preferable to have --build be the default and --no-build be optional
• I'm not sure about the necessity of --src and -o, but as far as I know, spago supports alternative paths for src and output (is that correct?). Therefore it would make sense, that the backends also respect that.
• pskt writes generated executables in the output/pskt/program.jar file; AFAIK psgo writes the executable to ./Main
• The default value for --output could also be just output/. (That's what psgo does)
• Hooking up to the bundle-app and bundle-module commands would be nice: For the alternative backends bundle-app would mean compiling to a executable (native exe or executable .jar) and bundle-module would mean compiling to a library (shared lib or non-executable jar)
  • The --to option would allow specifying where the compilation result should be saved.

Update:

• output/ as the default target folder instead of output/[compilerName]
• Make building an executable (or library) the default. Add the flag --no-build instead of --build

[f-f]

cc @nwolverson @andyarvanitis

[andyarvanitis]

Looks good to me so far.

1. I would vote for --build being the default (--no-build being optional).
2. I'll look into --src and -o for both of my backends (I can at least provide a warning if one of them needs to be ignored)
3. "The default value for --output could also be just output/. (That's what psgo does)" – Please!
4. I can look into bundle-app and bundle-module

[csicar]

I would say either having the --build or the --no-build flag, but not both as that would be confusing.

I would vote for --build being the default (--no-build being optional).

So you mean, that without any options provided to the compiler it will create the files in the target language and build a binary?

[andyarvanitis]

Right. I'm open to discussing it, though.

[csicar]

No, I'm fine with that. That's probably what the user expects.

[f-f]

@csicar it looks like your first proposal seems good by everyone, so I guess we could proceed to standardize this? 🎉

I guess the next steps would be:

• commit what we have so far to a file and call it v1 or something
• define what the spec entails and guarantees - i.e. "if you conform to this then spago will run your thing just fine"
• define a versioning schema for the spec to handle things like breaking changes

[nwolverson]

Started separating the purerl backend to a separate backend executable rather than a compiler fork. Working great so far

Some thoughts

• Wonder if a command based CLI (like purs and spago) may make sense, otherwise there will end up being irrelevant combinations of options

• Don't think '--src' should be required, the CoreFN representation already specifies the source path of a given module - which when used with spago may be 'src/...' or '.spago/...' q

[nwolverson]

1. Problem in the above scheme, optparse-applicative (which I'm using for argument parsing, same as purs and spago) doesn't support optional values for parameters like --run [Main.main]. I'd suggest splitting out the main argument, as existing tools purs bundle, spago, pulp do. Ideally $BACKEND run [-u Main.main] or such. Alternatively, just make the function mandatory, and spago can do the defaulting part

2. I'm a bit unclear what the spago interface to the --no-build would be - as that's different from spago's own "no build" - I think the common usage would be to perform a PureScript build, generate output but not build a binary (or whatever)

3. Assume we can add a spago flag similar to the --purs-args for backend-specific args

[csicar]

Wonder if a command based CLI (like purs and spago) may make sense, otherwise there will end up being irrelevant combinations of options

I was trying to avoid that to keep the CLI of backends simple. But I do see the problem, that we could easily end up the mutually exclusive arguments and other weird stuff. I guess $BACKEND run and $BACKEND build would be the to sub-commands?

Problem in the above scheme, optparse-applicative (which I'm using for argument parsing, same as purs and spago) doesn't support optional values for parameters

I'm using optparse-applicative as well and ran into the same problem. To fix this, I removed the value option from the parser and added the default value manually after parsing in my program. In my mind, making the CLI interface more complicated to avoid a problem of optparse-applicative is not worth it.

I think the common usage would be to perform a PureScript build, generate output but not build a binary (or whatever)

So you mean spago build would only generate erlang / go / kotlin files, but no binary? I also prefer this option, as it would make building quicker. In that case, I would still expect spago bundle-app and spago bundle-module to generate binaries.

Assume we can add a spago flag similar to the --purs-args for backend-specific args

Good point. I'll add that. Should there also be a option in spago.dhall for specifying those options?

[nwolverson]

I guess $BACKEND run and $BACKEND build would be the to sub-commands?

That would be my suggestion - at least right now, my mental model is that "$BACKEND" and "$BACKEND --run" are the 2 commands. I certainly intend to add a repl too. Ultimately it's possible to just have a bunch of -- options for these things, it's just not the interface I would choose to present to anybody using the backend directly.

Regarding --run - I think that the "main function" argument (i.e. Main.main) is going to end up being supplied to whatever the "build an executable" command line is, at least I assume any backend that does this is going to need to specify the entry point to build just as to run - so maybe simpler to make this separate.

So you mean spago build would only generate erlang / go / kotlin files, but no binary?

I thought this was part of the discussion already - the --no-build in the table above ("skips building an executable"). More concretely, for me there are 2 scenarios it would be great to support (and however this falls out in options vs the meaning of 'bundle' is fine):

1. Simple "just works": I can run spago and create something runnable in 1 command (or indeed just run it with run)
2. I am integrating into a larger application, I want to run the compiler and codegen and plug this into existing build. In the erlang case, I want to generate '.erl' and include this in the build of an OTP application

Good point. I'll add that. Should there also be a option in spago.dhall for specifying those options?

I think there's already an issue (#455) around supporting various command line options in spago.dhall, I assume the same scheme would hold.

[f-f]

Good discussion, thanks @nwolverson @csicar :+1:

I'll just note that I'm on board with the Spago changes required to make this work (i.e. adding some --compiler-args flag and more config keys to the spago.dhall)

[andyarvanitis]

FYI, I'm good with implementing whatever changes are needed in psgo/pscpp to support the spago commands

[thautwarm]

Hi, I'm now maintaining this purescript python backend(aka pspy), and I have concerns here.

Firstly, I'm okay with making changes to pspy CLI to support spago commands, and what I care most here is how to deal with FFI files.

Libraries published to pursuit always contain only .js files, i.e., only JavaScript FFI natively. I saw some FFI works made by @andyarvanitis at this repo: https://github.com/purescript-native/go-ffi

I seems that he tried to provide Go FFI for a "batteries" collection of pursuit libraries. This is respectful, but I'd say there shall be better approaches.

We firstly notice a fact that, it's hard to ask authors of pursuit libraries to add support for other backends. And new backends might get come up suddenly(like purescript-python), and the lack of FFI interfaces would hurt their passionates.

Hence, I'd propose here a mechanism to support FFI for alternative backends, in a thoroughly extensible, and customizable approach.

An example is what I'm now taking advantage for pspy. Now I implemented Python FFI for some essential pursuit libraries:

Pursuit Library	A Python FFI Provider
https://github.com/purescript/purescript-console	https://github.com/purescript-python/purescript-console.py
https://github.com/purescript/purescript-prelude	https://github.com/purescript-python/purescript-prelude.py
https://github.com/purescript/purescript-console	https://github.com/purescript-python/purescript-console.py

I introduce a notion FFI provider repository here. In a Python FFI provider repo, its structure is:

- $PROJECT_DIR/
   - python-ffi/
       - A.py    # correspond to src/A.js in the target pursuit project
       - B/
         - C.py  # correspond to src/B/C.js in the target pursuit project 

Hence, this repo can be used to provide FFI for a given pursuit library.

Another notion is mirror, which maps a versioned pursuite package to a FFI provider repository(or even specific branch/tag).

For pspy, this is a mapping function of a working mirror. Currently, it implemented the mapping presented in above table.

Then spago build will trigger pspy, and finally correctly get Python FFI files for purescript-{prelude, console, effect} from the 3 FFI provider repo.

That's all about my proposal mechanism of FFI provider for alternative backends, and I think it'll be quite useful if spago could support this for alternative backends.

To achieve, a sub-command spago mirror shall be required, for managing FFI provider mirrors for alternative backends.

Using spago mirror is simple:

spago mirror <backend name> <mirror name> [add <url> | remove | update]

[paulyoung]

I just discovered this issue and I'm wondering, are there are plans to finalize this? It looks like --run was implemented.

For consistency and clarity, could we go with --output instead of -o?

I also have reservations about the name of the build command since currently spago build means "produce JavaScript from PureScript" but here we're proposing that it means "produce X from PureScript, and also build an executable" where the latter seems more like what spago bundle-app does.

This also got me thinking about IDE integration will be affected since I think we need to run spago build at least once before that works and with the current proposal we would be building an executable for no reason when doing that.

I know we could pass --no-build to prevent that unnecessary work, but only when using an alternative backend. I think it's worth considering making this all consistent with the existing behavior for the JavaScript backend.

[nwolverson]

I've passed having an opinion on the CLI - I think I stated my thoughts at the time - but if we can make a decision I can implement it for purerl relatively soon, can also assist in making spago changes.

I don't have anything to say on the FFI mirror - my approach for purerl is to bake this entirely into the alternate package set, with forks for projects requiring differing FFI (which sometimes also require .purs changes or different dependencies in any case)