antlr4
antlr4 copied to clipboard
antlr
[Feature Request] Target for Rust Programming Language
I propose we have a target for Rust Programming Language since:
- Rust has finally delivered stable release last year (currently version 1.16).
- Rust has attracted more and more engineers recent years.
Rust has become a mature programming language with the constant effort from Mozilla. The I haven't found any good parser generators in the Rust world yet. Hopefully, ANTLR can be one.
This will only happen if someone from the community creates such a target and sends a pull request to the main ANTLR repo.
I would be willing to contribute, is somebody working on this subject already, so I dont do double work?
This would be great. A minimal implementation that omits some features would be fine (the Go implementation doesn't have a default visitor implementation for example). I just want to walk the tree myself.
I'd like to give this a shot unless someone else is already working on it.
@metadave I have done quite a lot of work on it recently. You can see it in https://github.com/rrevenantt/antlr4rust A decent bit of things are already working. If anybody wants to help i would gladly appreciate it.
BTW How complete target implementation should be to be merged in?
@rrevenantt I recommend to use a clone of the antlr repo instead of an own repo, with an own structure. This way you can later create a pull request to get your code merged in and you can directly use the scripts, folders and test infrastructure of the main repo. Keep in mind that your target has to run the entire test suite that's executed for each of the targets.
It would also be good to announce this work on the ANTLR4 mailing list.
@rrevenantt excellent! if that's the case, then perhaps I'll pick a different (new) target to work on, as I've wanted to learn all the internals of Antlr4 for quite some time.
@mike-lischke Thanks. I was thinking about using git-subtree, so i have created it as a separate repo. Anyway I can easily merge them later. Nevertheless I am already using test infrastructure of the main repo, and quite some tests from test suit are already passing.
So the only requirement for PR is to pass complete runtime test-suit (and I suppose integrate api tests to CI) and then I can continue to improve it as part of main repo?
Is it better to ask subsequent questions on the mailing list as well?
If that works out for you then fine! I had some trouble when I worked on the C++ target, which also used an own repo, that's why I mentioned it. If a number of the tests already succeed then you are already pretty far. That's very impressive!
For further discussions I'd indeed recommend the mailing list.
I like GitHub issues, I think it's also an appropriate place for feature/bug discussions.
So I have implemented almost everything, less than 10 test are failing. Also there are some refactorings I would like to do beforehand. I think it will take couple more days.
Finally, I have made it pass all tests. Those last parts were heavily based on inheritance and RTTI, which does not integrate with Rust paradigm after some point. So I had to rewrite parser generation for labeled alternatives to create enums. But it can generate almost fully idiomatic Rust syntax tree now. Was also going to remove most of the redundant allocations and cloning, but it appeared much harder than I expected. It is a real problem only in some rare corner cases, though, so I think Rust target can already be used.
Will still try to clean up and make a PR tomorrow. But I doubt that it will be merged quickly, also I will not be able to work on it much for about a week, so if anyone wants to use it ASAP, ping me and I will release it separately in my repo until it is merged.
So while fixing that last issues, and preparing PR, I was thinking how everything is going to work if it gets merged in current state, and decided not to do it right now.
- Firstly, the more I look at my current API design, the more I find ways to improve it ( if you know nothing about Rust, it is very different language in a lot of aspects, which forces you to design API differently). and they can break backward compatibility. Also feels like it needs much more tests than there currently is. And I have looked into other targets and it looks like they were merged in more finished state.
- Also it feels to me that this repository is not well suited for active development. And there is still quite a lot I can do. So I think that for everyone it would be more convenient, if I will continue to develop Rust target in my own repo for some more time.
Nevertheless, I am not 100 % sure and if you prove me wrong, I can create PR next day. Meanwhile I have published first version on crates.io, so everyone is welcome to try it out and fill an issue on rrevenantt/antlr4rust if something is wrong.
Hi,
it's great that you've reached this state!
The code for this target might be merged anytime. However, the pace of releases is low: there was only 1 last year, and there was one just a month ago.
We might afford a release for this, I'd say it's definitely worth it. But you probably want to increase confidence in your API before letting anyone build on the current one only to find that it breaks 3 months later (I haven't looked at the API so only suggesting that from your own sayings).
The recommended way of using ANTLR4 is through listeners or visitors, so make sure you get these right! Without inheritance, it's certainly a challenge!
Early adopters can always get started using your branch, but then they know what to expect in terms of API stability.
Thank you for all your work. I've tried rust target in my project. It worked with my previous grammar file. What is the status of the rust target now? When will it merge in？
antlr4rust hasn't had a commit in almost a year and still seems to only work for rust nightly, are there any forks or something that anyone is working on? I'd really like a rust target, and I would do it myself but I don't think I'm experienced enough in Rust to figure that out. Any updates would be great
Sorry for my absence, was really busy irl. I have just started working on it again, like a two days ago. Nevertheless there is https://github.com/rrevenantt/antlr4rust/pull/29 PR which is where I left off and it already works on stable rust. I am going to publish a prerelease version with it in a couple of days. There are still some more things i wanted to do before a proper next version release though. And after that i will start working on a PR to merge it into this repository
Sorry for my absence, was really busy irl.
Maybe it's even better because I significantly simplified runtime test infrastructure. Just compare test classes for C++ in the old infrastructure and in the new infrastructure. Now it's much simpler to integrate new targets.
@KvanTTT Do we have a document describing how to integrate a new target into the test infrastructure? I vaguely remember having read instructions in the past (after all I integrated the C++ target), but that info is probably outdated now and I don't remember where I read it.
Yes, it's here: https://github.com/antlr/antlr4/blob/dev/doc/creating-a-language-target.md Also, I'm in process of updating some outdated docs.
Also, I'd recommend looking at other targets and implementing a new one in the similar way. For instance, here is the setup for C++: https://github.com/antlr/antlr4/blob/dev/runtime-testsuite/test/org/antlr/v4/test/runtime/cpp/CppRunner.java
A significant potential benefit of a Rust target is... WebAssembly. My own experiments trying to move some bits of the js runtime to webasm have not been encouraging (the cost of crossing the js/webasm fence is higher than the benefits).
Chances are that with a significant part of the runtime natively optimized for webasm, this balance will revert and we could use a rust originating webasm runtime to power the core of the js/ts one.
Guys...sorry but not sure I'd be interested in yet another platform to support / manage.
