Bad error message for `1.+1`

[oscardssmith]

JuliaSyntax gives While flisp gives

ERROR: syntax: invalid syntax "1.+"; add space(s) to clarify
Stacktrace:
 [1] top-level scope
   @ none:1

[c42f]

In the flisp parser this error arises from the lexer which is why it didn't get ported across in this case. I guess we've gotta figure out how to get error messages out of the tokenizer. Maybe out of band like ParseStream does it would be fine. Or invent a bunch of codes which can later be translated.

In this case the lexer is producing a bland K"error" which isn't much to go on downstream:

julia> collect(JuliaSyntax.Tokenize.tokenize("1.+1"))
4-element Vector{JuliaSyntax.Tokenize.Token}:
 0-1        error          
 2-2        +              
 3-3        Integer        
 4-3        EndMarker      

julia> dump(collect(JuliaSyntax.Tokenize.tokenize("1.+1"))[1])
JuliaSyntax.Tokenize.Token
  kind: JuliaSyntax.Kind K"error"
  startbyte: Int64 0
  endbyte: Int64 1
  dotop: Bool false
  suffix: Bool false