rules_closure
rules_closure copied to clipboard
bazelbuild
Support compiler modules
The current API does not make the use of compiler level modules easy (all the flags need to be passed via defs). It would be nice to have first class module support which set at least rename_prefix_namespace, module_wrapper and module. Additionally, automatically counting the number of files in each module and hiding that complexity from the user would be ideal.
Yes this is definitely something we need. But the service I work on internally doesn't really have enough code to justify modules. It isn't exactly G+. Which why I haven't written the support yet.
One thing that's problematic, is that even if you pass the flags to defs, you wouldn't be able to tell bazel about the output files they're going to create, so it still wouldn't work.
It would take a lot of research on my part to learn the module system, and study the internal build solutions, to be able to figure out what would be the most intuitive design for such a thing.
Right now, I'm leaning in the direction that creating a unified suppression system, growing the user base, and bundling externs would be better priorities.
What do you think? Is modules something you really really need?
One thing that's problematic, is that even if you pass the flags to defs, you wouldn't be able to tell bazel about the output files they're going to create, so it still wouldn't work.
Good point.
It would take a lot of research on my part to learn the module system, and study the internal build solutions, to be able to figure out what would be the most intuitive design for such a thing.
This is a fair point. I have not given it much thought myself. Off the top of my head it seems like we might be able to utilize information about which closure_js_library() a given source files is contained in to assign it into the correct module. Obviously this proposal is half-baked and would require much more thought and discussion.
Right now, I'm leaning in the direction that creating a unified suppression system, growing the user base, and bundling externs would be better priorities.
From my view I would rank these in the following order:
- unified suppression system
- bundling externs
- growing the user base
However, my personal feeling is that we should have a nearly finalized API prior to significant expansion of the user base. I'm concerned that adding good module support will require API changes. Therefore, we would ideally figure the correct module API while we still have the flexibility to make backwards incompatible changes. I would rank module support just above "growing the user base".
Is modules something you really really need?
I currently use closure in 5 projects. In 4 of the 5 projects we don't use modules at all (relatively small code bases). In the last project we do use modules as the total size of compiled & gzipped output is large enough that parallel loading of modules has an appreciable impact on user experience. For this last project, it would be a hard sale to the team to remove modules all together while switching to rules_closure.
So I guess my answer is no, I don't really need modules. I'll still use and love rules_closure for most of the projects that I work on.
One general concern that I have with not supporting modules is that it may add a significant barrier to wide adoption inside Google (granted, that might be an uphill battle anyways). I'm guessing many of the largest projects in Google would expect native module support.
However, I really trust your intuition here @jart. You have much better insight into how widely modules are adopted inside Google. If from your view lacking module support will not significantly inhibit adoption then I think we should not add the support.
Internally, we have like all this google web server server infrastructure for serving modules, with a parallel set of build rules, named after legumes, designed specifically to cater to the needs of projects that need module serving infrastructure. It's pretty complex. I'm considering joining the web frameworks team that maintains it. But so far, I've been targeting the small end spectrum.
I would both agree and disagree about needing a finalized api to grow the user base. We might want to get more stability before we start posting on announcements on high traffic blogs. But right now I think is a great time to start growing the project by word of mouth, with other seasoned Closure Tools users, who want to have an impact on the design.
I agree with your priority ordering.
I know this is pretty old issue, but I was just curious if there is any update on this. If not, I'll go ahead and try to write my own rules that will work for us.
This problem is such a deep rabbit hole that involves more than just builds, but also service architecture. It's something we might not have time to work on at this time. We have some of the HTTP serving infrastructure in place here, to lay a foundation. It's possible another team at Google might come along and recreate the service we use internally, which is quite sophisticated, due to the depth of this problem.
@JaredNeil, did you ever end up writing those rules? if so, any interest in sharing code, or at least notes about your approach?
@jart i dunno if you're still involved with this project - firstly, this is such an awesome framework, we absolutely love it! we are a direct result of the grow-your-userbase goal. rules_closure is what brought us to Bazel and we haven't looked back.
i'm just wondering if anyone has any appetite for figuring this one out. i'm happy to pick it up, but, i'm new to the library so perhaps someone has some context to share about how it might best be implemented.
We did end up writing our own rules. They are very customized to the way we use chunking (compiling the same chunks multiple times with multiple applications), but the general idea is having a rule that creates a file with the list of files that should be in that chunk, and then a separate rule that combines all of those chunks into a single closure compiler invocation.
I have not used chunks in the past, although I was interested to be able to do so in the future. I don't think I would be of any help in the implementation unfortunately.
