Implement a module that provides wrappers for common code generators

[dcbaker]

A common problem for Meson is the need to call common code generators in the form of custom_targets (usually do to generating headers). While these generators can be unique, some of them are quite common. In fact, some of them are so ubiquitous that that there are multiple, slightly incompatible, implementations. Others are common and complex enough that a wrapper is very helpful.

This PR implements the module, and provides wrappers for two very common code generators: flex/lex and bison/byacc/yacc. These are used in plenty of projects and especially for bison/yacc they have enough complexity to be vastly simplified through a module interface.

One other future piece that likely belongs here is a wrapper for protobuf.

Fixes: #11895

[eli-schwartz]

Force pushed to get Arch CI to build after libgcrypt changes. Can't do it while triggering an image rebuild since it doesn't modify ci/ :P

[dcbaker]

Latest updates include checking for RE/flex and skipping that, as well as a file watching tool called reflex (similar to what we do with GNU link on windows). I've also removed the RE/flex from the arch image, and tried installing the correct reflex on gentoo (which is the only distro I see in our supported list that packages it)

[dcbaker]

@thesamesam I wrote some code to allow more configuration of the lex and yacc versions to use, which look like:

codegen.find_lex(implementations : ['flex', 'reflex'], flex_version : '>= 2.6', reflex_version : ['>= 1.4', '!= 1.5.1'])

but maybe it actually makes more sense to just have:

codegen.set_lex(find_program('flex', 'win_flex'))

What do you think?

[thesamesam]

Will take a look.

[dcbaker]

@thesamesam Thanks for the review. I updated a few things after porting a couple of projects to use this.

I've still got one interesting issue for Meson that I'm not sure about from an API point of view, Mesa needs to add specific flags for bison only, with a specific version. so something equivalent to:

prog = find_program('bison', 'byacc')
if prog.name() == 'bison' and prog.version().version_compare('> 2.3')
   prog = [prog, '-Wno-deprecated']
endif

I'm not sure what the best way to handle this from a module API would be. I don't want a mutable module API, so it seems like you'd basically just need a lex_binary() method that just returns the ExternalProgram? and then handle that like:

prog = codegen.lex_binary()
flex_args = []
if prog.name() == 'bison' and prog.version().version_compare('> 2.3')
   flex_args += ['-Wno-deprecated']
endif

And then add flex_args everywhere?

[bonzini]

What about having methods like find_lex() and find_yacc() that return a Program, but the returned objects also support a generate() method? This way you automatically support all program methods and can even use it for a custom target if desired.

[dcbaker]

What about having methods like find_lex() and find_yacc() that return a Program, but the returned objects also support a generate() method? This way you automatically support all program methods and can even use it for a custom target if desired.

This sounds like the python.find_installation() method, which has always been super awkward to use. It seems more like that solution should just be "overload find_program() to return special objects for bison, yacc, flex, lex, etc"

[bonzini]

This sounds like the python.find_installation() method, which has always been super awkward to use.

Yes that's what I was thinking about, I've never found it awkward.

It seems more like that solution should just be "overload find_program() to return special objects for bison, yacc, flex, lex, etc"

I think that would be even harder to document and discover...

By the way this fix three-digit #674. :)

[dcbaker]

~~Docs are in need of update~~

Docs have been updated.

[bonzini]

I like this, perfect is the enemy of good. If anyone comes up with a need for full_path() or anything like that we can consider making this an External program subclass, otherwise no.

[dcbaker]

@thesamesam, any more comments?

@eli-schwartz I remember asking you a long time ago about the CI image builder jobs, and I think you said that their failure here is okay?

[eli-schwartz]

Yes, the tests outside the CI image builder succeed so no regressions are going to happen.

• 3 of the failures are for images this PR doesn't modify
• the Gentoo failure means your "reflex" change is getting nowhere and achieving nothing; you aren't testing it in the end, which may be a problem.

(Gentoo tests fail because of CMake 4.)

[eli-schwartz]

@thesamesam

https://bugs.gentoo.org/958032 sigh :D never a dull moment.

[dcbaker]

I threw this at a somewhat exotic problem of generating the same flex/bison definitions in a cross compilation setup for both machines, and ran into a few corner cases. So there's a small update to fix that.