cmake_format icon indicating copy to clipboard operation
cmake_format copied to clipboard

clang-format "CRITICAL __main__.py:635: Unexpected end of token stream while parsing statement" on valid CMakeLists.txt

Open vlebourl opened this issue 3 years ago • 4 comments

The CMakeLists.txt file is valid and in use in prod for a long time, but can't be parsed. The file is long (1643 lines) and unfortunately can't be shared publicly... Any idea how I could help?

vlebourl avatar Jan 12 '21 09:01 vlebourl

Can you share the rest of the error message? I believe that the message will tell you which statement is the "start of the problem". If you can share just that statement that might show us what the issue is.

Note that cmake itself is actually rather lenient and will accept listfiles that don't comply with the documented language syntax. cmake-format is implemented to match the documented syntax. One area where cmake-format is more pedantic is with regard to strings. Some things which should be quoted (according to the syntax definition) are actually allowed by cmake, but not cmake-format.

cheshirekow avatar Jan 14 '21 21:01 cheshirekow

The full error message is 11 thousand lines long... Here's the beginning of it:

WARNING __main__.py:530: While processing CMakeLists.txt
CRITICAL __main__.py:635: Unexpected end of token stream while parsing statement:
 └─ StatementNode: 696:2
    ├─ FunctionNameNode: 696:2
    │   └─ Token(type=WORD, content='ADD_CUSTOM_COMMAND', line=696, col=2)
    ├─ TreeNode: 696:20
    │   └─ Token(type=LEFT_PAREN, content='(', line=696, col=20)
    └─ StandardArgTree: 696:21

and this is the line in question:

		ADD_CUSTOM_COMMAND(TARGET ${TargetName} POST_BUILD COMMAND /usr/bin/codesign --deep --force --verify --verbose --sign \"iPhone Developer: foo bar \(xxxxx\)\" $<TARGET_FILE:${TargetName}>)

Funnily enough, I found while obfuscating the line that the problem are the escaped parenthesis \( ... \). Without them, the formatting succeeds.

vlebourl avatar Jan 15 '21 10:01 vlebourl

I'll take a look at this example a bit later, but my first thought is that you probably intend for \"iPhone Developer: foo bar \(xxxxx\)\" to be a single token, so I would recommend quoting it. You can try the following:

ADD_CUSTOM_COMMAND(TARGET ${TargetName} POST_BUILD COMMAND /usr/bin/codesign --deep --force --verify --verbose --sign "\"iPhone Developer: foo bar \(xxxxx\)\"" $<TARGET_FILE:${TargetName}>)

This may still be a deficiency in the cmake-format lexer, but I suspect this might allow you to move forward.

cheshirekow avatar Jan 15 '21 16:01 cheshirekow

Hi @vlebourl , can you confirm that quoting the string fixes the problem? It seems to work for me:

add_custom_command(
  TARGET ${TargetName}
  POST_BUILD
  COMMAND
    /usr/bin/codesign --deep --force --verify --verbose --sign
    "\"iPhone Developer: foo bar \(xxxxx\)\"" $<TARGET_FILE:${TargetName}>)

cheshirekow avatar Apr 05 '21 05:04 cheshirekow