[Feature] Implement command line parser builtin

[hdwalters]

Is your feature request related to a problem? Please describe. When writing Bash scripts taking command line options, I use getopt to preprocess the command line, and a while loop and case statement to handle individual options:

#!/usr/bin/env bash

# Parse command line or print help text.
script=""
files=()
title=""
items=()
depth=0
ultra=0
verbose=0
getopt=$(getopt --options=t:i:d:uv --longoptions=title:,items:,depth:,ultra,verbose,help -- "$@") || exit
eval set -- $getopt
while true; do
    case "$1" in
        -t|--title)
            title="$2"
            shift
            shift
            ;;
        -i|--items)
            items+=("$2")
            shift
            shift
            ;;
        -d|--depth)
            depth="$2"
            shift
            shift
            ;;
        -u|--ultra)
            ultra=1
            shift
            ;;
        -v|--verbose)
            verbose=1
            shift
            ;;
        --help)
            cat <<EOF
Test Amber command line parser
Syntax: $(basename $0) [options]
  SCRIPT: Text ......... Script path
  FILES: [Text] ........ File paths
  -t|--title: Text ..... Title text
  -i|--items: [Text] ... Item text
  -d|--depth: Num ...... File depth
  -u|--ultra: Bool ..... Ultra flag
  -v|--verbose: Bool ... Verbose flag
  --help ............... Show help text
EOF
            exit 1
            ;;
        --)
            shift
            break
            ;;
        *)
            exit 1
            ;;
    esac
done

# Copy positional parameters.
script="$1"
shift
files=("$@")

# Print positional parameters and options.
echo "Script: \"${script}\""
for file in "${files[@]}"; do
    echo "File: \"${file}\""
done
echo "Title: \"${title}\""
for item in "${items[@]}"; do
    echo "Item: \"${item}\""
done
echo "Depth: ${depth}"
echo "Ultra: ${ultra}"
echo "Verbose: ${verbose}"

This can handle:

1. Long options with params --label=zzz or --label "zzz"
2. Short options with params -lzzz
3. Long options for flags --flag
4. Short options for flags -f
5. Short options for multiple flags -xyz expanded to -x -y -z
6. Positional arguments left over at the end of the while loop

$ ./cli.sh run.sh one.txt two.txt three.txt --title 'My Fruit' -iapple -ibanana -icherry -d4 -uv
Script: "run.sh"
File: "one.txt"
File: "two.txt"
File: "three.txt"
Title: "My Fruit"
Item: "apple"
Item: "banana"
Item: "cherry"
Depth: 4
Ultra: 1
Verbose: 1

Describe the solution you'd like However, this is such a painful process, I have to copy and adapt a previous script every time. I would like to avoid this pain in Amber, by writing a set of builtin functions. This is my proposal:

#!/usr/bin/env amber

main(args) {
    // Parse command line or print help text.
    let parser = parser("Test Amber command line parser")
    let script = param(parser, "script", "", "Script path")
    let files = param(parser, "files", [Text], "File paths")
    let title = param(parser, "-t|--title", "", "Title text")
    let items = param(parser, "-i|--items", [Text], "Item text")
    let depth = param(parser, "-d|--depth", 0, "File depth")
    let ultra = param(parser, "-u|--ultra", false, "Ultra flag")
    let verbose = param(parser, "-v|--verbose", false, "Verbose flag")
    getopt(parser, args)

    // Print positional parameters and options.
    echo "Script: \"{script}\""
    for index, file in files {
        echo "File #{index}: \"{file}\""
    }
    echo "Title: \"{title}\""
    for index, item in items {
        echo "Item #{index}: \"{item}\""
    }
    echo "Depth: {depth}"
    echo "Ultra: {ultra}"
    echo "Verbose: {verbose}"
}

The parser builtin creates a parser struct to hold the array of (reference counted) argument structs for Bash generation. The param builtin parses the option match string, a default value (which also defines whether it takes a parameter at all, and the type of that parameter) and a help string, and creates the argument structs for the parser. The getopt builtin causes the command line to be parsed. Subsequently, the argument structs are responsible for returning the parsed values for use in the main script.

Describe alternatives you've considered The only alternative would be to attempt to write this as a standard library function, but (i) it requires interaction between the various components, which is just not possible in pure Amber, and (ii) it would be unable to support typed arguments, which are necessary to (for example) iterate over array arguments.

Additional context N/A

[Ph0enixKM]

Great idea! In the future we could extend these library functions with structs:

const args = {
    script: {
        argument: "script",
        default: null,
        title: "Script path",
    },
    verbose: {
        argument: ["--verbose", "-v"],
        default: false,
        title: "Verbose flag"
    }
}

main (args) {
    // Parse the command line or print help text.
    let parser = parser(args)

    // Print the positional parameters and options.
    echo "Script: \"{parser.script}\""
    for index, file in parser.script {
        echo "File #{index}: \"{file}\""
    }
    echo "Title: \"{parser.title}\""
    for index, item in parser.items {
        echo "Item #{index}: \"{item}\""
    }
    echo "Depth: {parser.depth}"
    echo "Ultra: {parser.ultra}"
    echo "Verbose: {parser.verbose}"
}

[hdwalters]

My proposed usage code may look familiar; I modelled it after the Python argparse module.

[Ph0enixKM]

Oh I see @hdwalters then we can stick to it

[adamdecaf]

Would this be better implemented as a library that gets solidified before stdlib adoption? I'm interested in flag support.

[hdwalters]

Hi Adam, thanks for the feedback!

Would this be better implemented as a library that gets solidified before stdlib adoption?

I'm not sure what you mean by this; are you describing (i) the standard library (written in pure Amber) or (ii) the wider ecosystem, including builtins?

If you're suggesting we should write the parser in pure Amber, that was never the plan. And having spent this week writing Rust code, I really don't think it would be possible to do it that way, and get anywhere close to the functionality I want. This is for several reasons:

1. The parser, param and getopt builtins need to interact with each other at a level lower than the Amber compiler, mostly because the parser object requires access to the internals of the param objects, so it can build help text and set Bash variables.
2. The param builtin needs to expose the type of the default value, so we can generate syntax errors if you try to (say) add Text and Num parameters. I've just tested this, and I got a suitable error.

I'm interested in flag support.

What are you trying to acheive? Perhaps additional functionality can be included in the builtins. Though I should point out that this is already supported by the Bash getopt command, and wrapped by the builtin code I've written: if you specify true or false as the param default value, the parser automatically expects a flag like -v rather than an option like -d4:

let verbose = param(parser, "-v|--verbose", false, "Verbose flag")

[adamdecaf]

I'm looking at trying out amber in a situation where we'd use the script(s) on mac and linux machines (mac host pushing script/commands to bash on linux). Amber caught my eye because it simplifies bash programming. The script would have a couple flags/arguments.

Coming from Go the parsing functions (parser, param, etc) would be developed separate from Amber and imported into amber code. Then it could be accepted by Amber (as a builtin or stdlib -- I'm not sure) once the API was stable and accepted.

[hdwalters]

Ok, thanks for elaborating. I'm unfamiliar with the Go development cycle, but from your description, this won't really work in this case. We can add functionality on top of the core compiler in one of two ways:

1. Via builtins (written in Rust, and compiled directly into the Amber binary).
2. Via standard library functions (written in Amber, and eligible for the sort of testing cycle you're describing).

For reasons given above, we cannot implement the parser in pure Amber, and therefore have no choice but to compile it into the binary as a set of builtins.

[b1ek]

i am very skeptical about this. it doesn't have to be implemented in the compiler. any user can use $getopt ...$ and construct a loop. creating builtins for it, especially param and parser is a completely unnecessary feature for us to maintain.

i would support creating a stdlib function for getopt tho, but not a builtin.

theres also no need to rush this. i would've waited until we have lazily evaluated types and proper structs to implement it

[hdwalters]

i am very skeptical about this. it doesn't have to be implemented in the compiler. any user can use $getopt ...$ and construct a loop. creating builtins for it, especially param and parser is a completely unnecessary feature for us to maintain.

i would support creating a stdlib function for getopt tho, but not a builtin.

We need to generate (i) the getopt command with --options=v --longoptions=verbose for flag options, or --options=c: --longoptions=count: for options taking a value, (ii) the help text, and (iii) the value parsing code. Each of these requires iterating over the parameter definitions. I considered writing a standard library function for this, but I don't think that's possible in pure Amber as it stands.

theres also no need to rush this. i would've waited until we have lazily evaluated types and proper structs to implement it

I'm not sure I want to wait for that. I would like a solution now, so I can start writing functionally useful Amber scripts. This approach allows for minimal Amber code, with syntax inspired by the Python argparse library, without having to define structures.