antlr-kotlin icon indicating copy to clipboard operation
antlr-kotlin copied to clipboard

$ symbol is not escaped properly in generated code

Open MartinHaeusler opened this issue 3 years ago • 1 comments
trafficstars

I'm trying to build a parser for MongoDB query expressions. The corresponding grammar has a lot of terminals which start with $, e.g.:

  • $eq
  • $and
  • $not
  • etc.

The problem is that the kotlin code generator does not escape those $ symbols properly. For example, in the generated lexer file, the LITERAL_NAMES contains:

"'\"$and\"'"

... which of course translates to a template expression in Kotlin that inserts the value of the variable and into the string. This is clearly not what we want. The proper escaping would be:

"\"${'$'}and\""

MartinHaeusler avatar Aug 11 '22 10:08 MartinHaeusler

Hi Martin, thank you for reporting this

ftomassetti avatar Aug 26 '22 06:08 ftomassetti

The KotlinTarget escape sequences should be:

val map = HashMap(defaultCharValueEscape)
addEscapedChar(map, '$')

map[11.toChar()] = "\\u000b" // Vertical tab
map[12.toChar()] = "\\u000c" // Form feed
map[14.toChar()] = "\\u000e" // Shift out
map[15.toChar()] = "\\u000f" // Shift in

charValueEscape = map

lppedd avatar Dec 04 '23 13:12 lppedd

I did a complete overhaul of the project.

  • Migrated to Kotlin 1.9.21
  • Translated all Java files to Kotlin
  • Modernized the build scripts (e.g., version catalog, task caching, proper plugin ID)
  • Migrated, commit by commit, from 4.7.1 to the 4.13.1 runtime
  • Cleaned up and reduced the numbers of expect/actual definitions

What I need to do:

  • Re-enable the native targets as soon as replacements for BitSet, IdentityHashMap and WeakHashMap are found
  • Reformat the runtime sources

@ftomassetti if you're up to it, I'll open a PR soon. Just need to strip some config related to my production environment.

lppedd avatar Dec 04 '23 17:12 lppedd

@lppedd I can't speak for the repo owner but to me that sounds fantastic!

MartinHaeusler avatar Dec 04 '23 18:12 MartinHaeusler

@MartinHaeusler I have immediate need of JVM and JS support, so I didn't lose time on native.

Also, I need to polish caching (@ftomassetti question, why does KotlinTarget cache templates using a thread local?).

And re-integrate tests. Currently testing is done in a way that makes devex a PITA, I'll just include the local plugin into an additional test-specific module.

lppedd avatar Dec 04 '23 19:12 lppedd