no support for herestrings (<<<)

[spikegrobstein]

The herestring (<<<) operator in bash is used to pass literal strings/variables into the STDIN of another command. For example. the following two lines are functionally equivalent:

echo "$foo" | grep "x"
grep "x" <<< "$foo"

Trying to parse this with bash-parser results in a parse error:

const ast = parse('grep "x" <<< "$foo"')

Error: Parse error on line 1: Unexpected 'LESS'
    at parse (…/js_bash_ast/node_modules/bash-parser/src/index.js:52:9)
    at fs.readFile (…/js_bash_ast/index.js:14:14)
    at …/js_bash_ast/node_modules/graceful-fs/graceful-fs.js:78:16
    at FSReqWrap.readFileAfterClose [as oncomplete] (fs.js:511:3)

documentation on the herestrings: http://tldp.org/LDP/abs/html/x17837.html

[piranna]

Thanks for the example and the equivalence, this will help to do a test case :-)

[parro-it]

Thank you, this is already known, there should be an issue or doc that mentions the lack of herestring. It should be the last POSIX statement missing...

Would you mind doing a PR to implement it? I could provide some suggestion on the sources you have to change to implement it.

[spikegrobstein]

if you can point me in the right direction for getting it fixed, I'll give it a shot. thanks!

I poked around but didn't see this issue documented anywhere. it took me a while to figure out what LESS it was talking about when it failed to parse my script.

[parro-it]

if you can point me in the right direction for getting it fixed, I'll give it a shot. thanks!

Awesome!

I poked around but didn't see this issue documented anywhere. it took me a while to figure out what LESS it was talking about when it failed to parse my script.

Ah sorry, I was speaking about POSIX heredocs (that are also missing); I misunderstood you... So , depending on what you want to accomplish, be aware that there are other bash part missing, you can find a list here: https://github.com/vorpaljs/bash-parser/issues/37.

The operator you're talking about is the last one in the list...

The error you are seeing is because the tokenizer is not aware of the <<< operator, and it is interpreting them (I guess) as tree < (LESS) operators in sequence. and that is not a valid syntax...

[parro-it]

First, you'll have to change how the bash mode is inherited from POSIX mode. You can start look here:
https://github.com/vorpaljs/bash-parser/blob/master/src/modes/bash/index.js#L124

That module export an object that define the bash mode. the object contains the property 'inherits', that specify the mode is based on POSIX:

	inherits: 'posix',

and an init function, that the system will call passing the base POSIX mode and an utils object as argument. The init function then create a copy of the POSIX mode and patch any property or function as needed

[parro-it]

The POSIX object you receive in init is composed by these properties:

{
			enums,                                              // various enums used by the mode
			tokenizer,                                          // the tokenizer: String -> Iterable<Object>
			lexerPhases                                     // The lexer "enhance" the tokens in a way that they can be understood by the grammar. Is implemented as a list of iterators, composed together. Each of these iterators are keep in the phaseCatalog 
			
                         phaseCatalog, 
			grammarSource,                 // the grammar source code 
			grammar,                             // the grammar built using jison
			astBuilder                            // build the AST nodes for each of the grammar production
		};

You probably could implement the <<< operator by adding a phase to the lexer that check for threee consecutive 'LESS' tokens, and, if the pattern is found, it emit a 'CALL-IT-SOMEHOW' tokens.

Then, you'll have to extend the grammar to take into account the new token. You probably will have to extend the io_file here: https://github.com/vorpaljs/bash-parser/blob/master/src/modes/posix/grammar.js#L387

remember that the grammar has to be recompiled using https://github.com/vorpaljs/mode-grammar-builder

[parro-it]

I suggest you to open a PR early, we can continue the discussion there if you have any doubt...

[spikegrobstein]

hey, sorry for the late reply. Things have gotten hectic and I don't think I'm going to have time to put together a pull-request. I don't wanna leave you guys hanging.