Remove comments in the parser

[typhoonzero]

We should output parse result without comment lines to make later processing easier. Currently, below SQL script:

-- some comment
set key=value;
-- some comment
SELECT * FROM table;

this script will be split to two statements: ---- some comment\nset key=value; and -- some comment\nSELECT * FROM table;. We directly send these two statements to database engines now. Yet, we determine set statements by checking the prefix set , so it will fail in this case.

The better way is to remove all comments in the parser and output valid statements only.

[weiguoz]

There are 2 kinds of comments:

-- this is a comment
select 1;

/* this is
another comment */
select 2;

ref lang_comment

[weiguoz]

SQLFlow is able to ignore comments on testing whether a SQL statement is a hint. ref. Still, we should remove the comments in the parser.

[wangkuiyi]

Should we move types in yacc file into an ast package?