JSqlParser icon indicating copy to clipboard operation
JSqlParser copied to clipboard

CCJSqlParserUtil fails on different variable names

Open DiegoKrupitza opened this issue 3 years ago • 6 comments

Describe the bug When I want to parse the following query

select p from Customer c join c.productOrder p where p.delayed = true

using (Select) CCJSqlParserUtil.parse(query); I get the following error:

net.sf.jsqlparser.JSQLParserException: Encountered unexpected token: "." "."
    at line 1, column 55.

Was expecting one of:

    "&"
    "::"
    ";"
    "<<"
    ">>"
    "COLLATE"
    "CONNECT"
    "EMIT"
    "GROUP"
    "HAVING"
    "START"
    "["
    "^"
    "|"
    <EOF>

	at net.sf.jsqlparser.parser.CCJSqlParserUtil.parseStatement(CCJSqlParserUtil.java:190)
	at net.sf.jsqlparser.parser.CCJSqlParserUtil.parse(CCJSqlParserUtil.java:63)
	at net.sf.jsqlparser.parser.CCJSqlParserUtil.parse(CCJSqlParserUtil.java:38)
	at org.springframework.data.jpa.repository.query.JSqlParserQueryUtils.parseSelectStatement(JSqlParserQueryUtils.java:111)
	... 71 more

Although when I try to parse the following query

select p from Customer c join c.productOrder p where p.delaye = true

everything works as expected.

It is quite stragne that only the difference between delayed and delaye makes the parsing to fail. Maybe I am getting something wrong but I think an identifier should not make such a huge difference.

To Reproduce See above

Expected behavior Both queries should be parse without failing.

System

  • Database you are using: No DB just calling CCJSqlParserUtil.parse(query)
  • Java Version: Azul Zulu 1.8.0_322
  • JSqlParser version: 4.3.0

DiegoKrupitza avatar Jan 22 '22 15:01 DiegoKrupitza

Greetings,

DELAYED is a reserved keyword\token and needs to be quoted properly. I have sent a PR #1382 dealing with keywords in a more comprehensive way, but it is pending approval.

manticore-projects avatar Jan 24 '22 08:01 manticore-projects

Thank you for the PR! I am just curious why it got detected as a keyword, since it also starts with a table alias. 🤔 Therefore it should be an identifier and not a keyword. Maybe there is within the grammar of the parser something strange?

DiegoKrupitza avatar Jan 24 '22 08:01 DiegoKrupitza

The challenge is like that:

  1. you will need to define Tokens in order to trigger certain productions
  2. then you will need to define exceptions, whenever those Tokens are not meant to trigger a production -- manually and one by one

Step 1 is straight forward and everyone is fast to define a new token when a certain feature is needed and added. But Step 2 is quite cumbersome and has to be done manually -- while nobody is really aware, what people will want to use as object names in the DB and SQL.

The next problem is performance: Step 2 often results in additional LOOKAHEAD which are not too welcome because they exponentially increase the Parser Time.

Please keep in mind: JSQLParser is a "generated parser", very flexible and agnostic to the RDBMS. There is a good reason for the specific RDBMS to rely on "handwritten parsers".

manticore-projects avatar Jan 24 '22 08:01 manticore-projects

The challenge is like that:

  1. you will need to define Tokens in order to trigger certain productions
  2. then you will need to define exceptions, whenever those Tokens are not meant to trigger a production -- manually and one by one

Step 1 is straight forward and everyone is fast to define a new token when a certain feature is needed and added. But Step 2 is quite cumbersome and has to be done manually -- while nobody is really aware, what people will want to use as object names in the DB and SQL.

The next problem is performance: Step 2 often results in additional LOOKAHEAD which are not too welcome because they exponentially increase the Parser Time.

Please keep in mind: JSQLParser is a "generated parser", very flexible and agnostic to the RDBMS. There is a good reason for the specific RDBMS to rely on "handwritten parsers".

Ok thank you for clarifying!

DiegoKrupitza avatar Jan 24 '22 08:01 DiegoKrupitza

Btw, the PR #1382 solves your challenge, I have just tested it. (Solves it by semi-automatic generation of Step 2 described above, allowing all tokens where possible.)

manticore-projects avatar Jan 24 '22 08:01 manticore-projects

Btw, the PR #1382 solves your challenge, I have just tested it. (Solves it by semi-automatic generation of Step 2 described above, allowing all tokens where possible.)

That is perfect! 👍 Keep the great work up!

DiegoKrupitza avatar Jan 24 '22 08:01 DiegoKrupitza