Make parser more robust / error forgiving

[dummdidumm]

Is your feature request related to a problem? Please describe. The compiler is currently used as part of several language servers which attempt to provide a good intellisense for IDEs like VSCode. As a result, the content handed to the compiler is often incomplete or in an inconsistent state because the user is in the middle of typing something.

So if I hand in something like this:

<script>
   import Comp from './comp.svelte'
</script>
<Co

I get an error.

Describe the solution you'd like It should still throw an error but pass along something like a best guess of the parsed content. Mabye something like a partial output of parse up to the error. If some part of the input (script, style, template) is consistent, it would be great to hand back the complete parser result of that part.

Describe alternatives you've considered Don't use the compiler and somehow extract relevant info on our own - I would like to avoid that.

How important is this feature to you? Important to harmonize LSPs in the long run and be able to integrate with the svelte compiler more deeply. But not that important that you should start working on it immediately. Also, if it's not just something like "oh we have that info already, we just need to append it" but more like "we have to rewrite a whole lot for that", it's ok to discuss first what other possibilities there are.

[tanhauhau]

Hi @dummdidumm , wanna understand more on this issue, I'm pretty new to this topic, so I am curious is there examples of how other language is doing this?

Also

• how do you get the partial output of the parser? Within the Error object?
• should you pass a custom flag to indicate for a partial output? It seemed unnecessary to return that for a normal user

[dummdidumm]

Basically, all intermediate states where the user is in the middle of typing something, you get an error. This is bad for svelte-language-server because it uses the svelte compiler to transform the code into Jsx/Tsx:

1. Let svelte compiler parse the code
2. Walk resulting AST tree to transform the code into Jsx/Tsx code

The transformation is done because that way we can leverage typescripts autocompletions/diagnostics/go to definition etc.

Some examples where parsing fails:

<SomeComponent someProp={someVariable.

--> The svelte compiler will throw an error "unexpected token". What I would like to have is to get output up until that point with the last AST element being "Component SomeComponent with someProp", or even better somehow try to parse the stuff afterwards, too, with a best guess where someProp might be closed.

{#if 

--> as a user, I want to see what variable are available for #if, but svelte compiler will throw an error. What I would like to have is to get output up until that point with the last AST element being "Moustache Tag if", or even better somehow try to parse the stuff afterwards, too, with a best guess where #if might be closed.

An alternative would be to don't use the svelte compiler at all and try to parse the code ourselves, but I feel that is unnecessary work if it could be done in the compiler itself.

How other languages are doing this - frankly I don't know, but they are definitely doing it. If you use typescript for example, it will complain about invalid syntax but will not completely stop giving valid output in other places. Or HTML-parsers, they are very robust with regards to missing closing tags etc.

Answering your questions:

• I would expect the partial output either as part of the Error object, or the normal object is returned with the error set as part of the warnings property. Or maybe both, kind of: Normal object for parse with new prop errors, part of error object for compile (because you cannot create js code from an invalid AST, that is also not needed). We can discuss the specifics if you like, and what works best for you given what code already exists in the compiler.
• Yes, I would expect a custom flag, because as you say this is no common use case / could change the behaviour.

[benmccann]

I just saw on another thread that @simlrh suggested that acorn-loose allows for this: https://github.com/sveltejs/svelte/issues/4518#issuecomment-597519107

[dummdidumm]

I dipped a little into the svelte compiler code and I don't know if acorn is part of the problem. The "I open a tag and don't have closed it yet because I'm in the middle of typing" error is thrown in the svelte parser doing tag analysis. Maybe parsing could be relaxed in such a way (if a relaxed-flag or sth like this is set) that if a { or < is encountered before the > that the tag is autoclosed.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[dummdidumm]

Still needed

[kwangure]

Lezer (Codemirror 6's new parser) is notably great at error-recovery and incremental parsing. Replit recently released their first stab at a Codemirror 6 plugin for Svelte which has a Lezer grammar for Svelte. It might be possible to shoehorn something out of that to run on broken code in the short term? I have no idea.

The caveat is that Lezer produces a non-abstract syntax tree in an effort to be light since it's optimized for syntax-highlighting. I don't know enough about the Svelte compiler and have not seen Lezer being used for anything but syntax-highlighting. At the very least the Lezer writeup could serve as inspiration for whoever takes this up and edutainment everyone else.

Fun unrelated fact: the creator of acorn also authored Lezer.