antlr4 icon indicating copy to clipboard operation
antlr4 copied to clipboard

Parsing [ pl/sql grammer] too slow with C++ target

Open searocsun opened this issue 5 years ago • 7 comments

Background: I want to parser pl/sql with c ++target .

Grammar: https://github.com/antlr/grammars-v4/tree/master/plsql

results: ParseTree is too slow, about 3 ~4 seconds

int main(int , const char **) { std::string str; std::string str_rep = "TO_NUMBER(*)"; std::string str_where = "WHERE";

std::string tmp_str;
std::string tmp_str_col;

std::string::size_type position;

ANTLRInputStream input; 

PlSqlLexer lexer(&input);

CommonTokenStream tokens(&lexer);

PlSqlParser parser(&tokens);

std::cout << "Please enter PlSql sql string: " << std::endl;

while (std::getline(std::cin, str,'^')&&str != "q"){


	transform(str.begin(), str.end(), str.begin(), ::toupper);

	input.load(str);

	lexer.Lexer::setInputStream(&input);

	tokens.setTokenSource(&lexer);

	tokens.fill();

	/*
	for (auto token : tokens.getTokens()){
	std::cout << token->toString() << std::endl;
	}
	*/

	parser.setInputStream(&tokens);

	tree::ParseTree* tree = parser.sql_script();

} }

Can someone tell me how to fix this problem?

Thanks

searocsun avatar Aug 07 '19 10:08 searocsun

Step 0: download intellij + antlr4 plugin and take look at what is slow in your grammar.

You should be able to see something like this: Screenshot from 2019-08-23 15-39-13

You can sort by time and click on row to highlight the corresponding rule.

(Note that there are also other tools which does the same.)

Nic30 avatar Aug 23 '19 13:08 Nic30

@Nic30 Thank you for your advice

searocsun avatar Aug 26 '19 02:08 searocsun

Actually, no, I did not gave you a good advice. You do have an embedded C++ in your code. (Which I did not see at the first time.)

  • You can rewrite this C++ to java or
  • you have to implement your antlr4::ANTLRErrorListener which does have methods reportAmbiguity, reportAttemptingFullContext etc. which are reporting when some performance problem appear. And then go through records manually.

Nic30 avatar Aug 26 '19 07:08 Nic30

@Nic30 thank you so much.

searocsun avatar Aug 27 '19 07:08 searocsun

I tried the PL/SQL grammer with a CSharp target - it is slow here, too. And it reported some ambiguities...

Could you improve the grammar?

bernd5 avatar Jan 18 '21 14:01 bernd5

Can you tell me how to solve this problem? Thanks

nIxedoahz avatar Apr 20 '22 04:04 nIxedoahz

Please take this conversation to here. This is a problem with the grammar, not Antlr. This issue should be closed here.

kaby76 avatar Apr 21 '22 01:04 kaby76