OrCombinator behaviors weirdly

[lolicon]

trafficstars

object Remainder : Grammar<String>() {

    val a by token("""a""")
    val one by token("""1""")

    val term by (a or (a * one)).map {
        it.toString()
    }

    override val rootParser = term

    @JvmStatic
    fun main(args: Array<String>) {
        println(parseToEnd("a1").toString())
    }
}

above code raise ParseException: Could not parse input: UnparsedRemainder(startsWith=one for "1" at 1 (1:2)) while it could parse a1 with rule a * one

[h0tk3y]

Thanks for the report.

I agree, this is rather counter-intuitive, but this behavior does not contradict the design. Note that OrCombinator always prefers the first alternative that got parsed successfully. In this case, a parses the first token, and the combinator returns that result, leaving the second token unprocessed, Also note that parseToEnd does not affect the logic of the combinators in any way and only throws the exception when the parser leaves an unprocessed remainder.

I'll think about what can be improved in this behavior.

[lolicon]

thx for your patient explanation, currently I workaround this by appending a -eof token to the rootParser and append a extra TokenMatch(eof,...) to the result of Tokenizer,

  private val eof = token("""\Z""")
  fun parseToEof(input: String) = (rootParser * -eof).parseToEnd(tokenizer.tokenize(input) + TokenMatch(eof....))

any better suggestion?