Make x3 docs more visible

[Flamefire]

When I open the documentation spirit 2 is shown: https://www.boost.org/doc/libs/1_86_0/libs/spirit/doc/html/index.html

Only a remark-like comment at the bottom reads:

For the newest (C++14) version, please follow to Spirit.X3

I found that a bit hard to find when looking for the "normal" documentation.

[ribbon-otter]

I would like to second this. I only learned the spirit 2 was not the most recent version because I was looking for exact source code so I could read the headers.

[saki7]

We actually need to make the root index point to X3, and link V2 from there (suggested by https://github.com/boostorg/spirit_x4/issues/15)).

[apolukhin]

It would be nice to have the Spirit.X3 as a separate library, without all the legacy of classic Spirit.

Some modern projects try hard to reduce dependencies and watch the module dependencies at https://pdimov.github.io/boostdep-report/develop/module-levels.html. Moving the X3 into a separate repo would drop most of the pre C++17 boost::* dependencies, making the X3 much more attractive to users

[saki7]

@apolukhin Thanks for your attention. We once had conversation with Peter Dimov and he also said we should split subcomponents. However, at the beginning of that conversation, I insisted on removing Spirit.Classic (after certain period of deprecation period) and thus the debate heated up too much. Eventually we had a consensus that we're going to split (not remove) Spirit.Classic. Peter told us that it's going to be easier if we let Peter do the actual work, but the migration didn't take place since then. Perhaps Peter is busy.

My current opinion is that we should split all Spirit subcomponents into distinct repositories:

• Classic
• V2 (Qi, Karma, Lex)
• X3

@apolukhin I want to make this happen. What can we do right now? Should we wait for Peter?

[apolukhin]

Let's ask @pdimov :)

Peter, is it OK for @saki7 to make separate repos in boostorg/ to separate old and modern Spirit implementations?

[saki7]

FYI, Spirit.X3 is now totally self-contained thanks to boostorg/spirit#817. Classic was already self-contained thanks to Joel's historical efforts.

[pdimov]

I already forgot everything we discussed, so could you please bring me up to date? What is the specific split we're considering, that is, how should the split repositories be named, and what would their directory structure be?

[saki7]

The only explicit consensus we have is that we split Classic into the separate repository. But that consensus was made 4 months ago and the situation has changed. I recently managed to make X3 self-contained too, so I'd suggest splitting all 3 subcomponents into 3 distinct repositories:

• boostorg/spirit-classic
• boostorg/spirit-v2
• boostorg/spirit-x3

The only subcomponent that has trivial content is Classic, which consists of classic/*, include/boost/spirit/home/classic/* and include/boost/spirit/include/classic*.hpp.

V2 and X3 are both self-contained in C++ source code level, but they still share the same parent directory, e.g. test/ and doc/. So if you demand the comprehensive list of files, that requires a lot of communication and I believe it's nonsense to do that.

If you can give me the organization-wide privilege then I can do all above work myself. If you're afraid of giving too much permission then I'd suggest creating 2 empty repositories (boostorg/spirit-classic & boostorg/spirit-v2) then give me full permission only for those.

[saki7]

If you choose to do the migration yourself then that's fine, I won't take it personally. But I think this should be done by a single person due to the communication cost.

[pdimov]

Our repositories correspond to directories under libs/, and as a rule we don't use dashes in those. Assuming underscores, the repos above would correspond to

libs/spirit_classic libs/spirit_v2 libs/spirit_x3

Is that what we want? What about the currently existing libs/spirit, would that be deleted?

[saki7]

libs/spirit_classic libs/spirit_v2 libs/spirit_x3

Is that what we want?

Yes, this looks good to me.

My suggestions:

• Keep the current repository boostorg/spirit as-is for a while
• Clone everything to boostorg/spirit_classic
  • git rm -rf all files except those related to Classic
• Clone everything to boostorg/spirit_v2
  • git rm -rf all files except those related to Qi/Karma/Lex
• Make sure the ordinary Boost Organization infra is working for the new 2 repositories (e.g. static site generation)
• Rename boostorg/spirit to boostorg/spirit_x3
  • git rm -rf all files except those related to X3

This way we keep all history with compatible hash, among all 3 repositories. This process minimizes the downtime as well.

[pdimov]

To clarify once again, in the Boost 1.89 release tarball we have libs/spirit and after the proposed changes, in the Boost 1.90 release tarball we won't have libs/spirit, but libs/spirit_classic, libs/spirit_v2, and libs/spirit_x3, right?

That's probably going to be relatively disruptive. Two things that come to mind: (1) the web site doesn't cope well with removed libraries, because it can show previous releases, e.g. https://www.boost.org/doc/libs/1_64_0/libs/spirit/doc/html/index.html, so it has to retain everything; (2) every CMakeLists file linking to Boost::spirit will now be broken.

I'm not sure what better alternative to propose, though. Maybe we could keep libs/spirit as some sort of a stub library?

[apolukhin]

+1 for keeping the spirit/ as the most modern Spirit.X3. Such move should give the users the signal to use it by default. The legacy libraries should move out into spirit_classic and spirit_old_v2

But I'm only an observer here, words of @saki7 have more weight

[pdimov]

My style would be to keep spirit the classic one and add spirit2 and spirit3, but that's obviously not the preference here. :-)

[apolukhin]

+1. That also gives a good signal to use spirit3 and also keeps most of the backward compatibility

[saki7]

+1 for keeping the spirit/ as the most modern Spirit.X3. Such move should give the users the signal to use it by default.

I actually think this is a good idea. So my preference is:

• boostorg/spirit -- keep, X3
• boostorg/spirit_classic -- new repo, Classic
• boostorg/spirit_v2 -- new repo, Qi & Karma & Lex (& /repository/)

(1) the web site doesn't cope well with removed libraries, because it can show previous releases, e.g. https://www.boost.org/doc/libs/1_64_0/libs/spirit/doc/html/index.html, so it has to retain everything

I'd suggest to just clone the identical docs and host the same content on /libs/spirit/doc/, /libs/spirit_classic/doc/, and /libs/spirit_v2/doc/ (for now).

Spirit's website has been obsolete for insane amount of time so it doesn't harm end-users even if we just end up showing same content. After the migration is complete I'll do the remaining work and update the docs accordingly. Just don't make the URLs dead.

(2) every CMakeLists file linking to Boost::spirit will now be broken.

Spirit's CMakeLists.txt is already broken. I can't manage it to work even if I use the one generated via b2 install. I also can't use it with git submodule add and add_subdirectory.

Even if it appears to correctly dispatch include/ with some intense cmake -D.... patching, the current CMakeLists.txt does not contain any fundamental compile options and shall be considered technically broken.

Believe me, I tried in many applications. It's on my todo list but you don't need to care about it during the migration process.

---

If I'm missing something please let me know.

[pdimov]

Spirit's CMakeLists.txt is already broken.

I don't know what you consider broken, but e.g. Serialization uses Boost::spirit in its CMakeLists.txt file, and builds with CMake, so it's obviously not broken for everyone and every use.

https://github.com/boostorg/serialization/blob/8a8c62864f05732131bf6785d54466d8ac6cd477/CMakeLists.txt#L85

[saki7]

@pdimov Modern CMake applications expect libraries to set target_compile_features for naturally resolving minimal required C++ version. Spirit's CMakeLists.txt does not even have this level of support.

Quoting my comment:

Even if it appears to correctly dispatch include/ with some intense cmake -D.... patching, the current CMakeLists.txt does not contain any fundamental compile options and shall be considered technically broken.

A well-established build script that does not follow the best practices shall be considered broken.

Let's stop talking about this. It's almost off-topic. We should focus on just doing the migration or we'll end up wasting another 4 months. (I'm not accusing you for this; I myself was lazy and didn't contact you recently. That is my bad.)

[pdimov]

A well-established build script that does not follow the best practices shall be considered broken.

What best practices does it not follow?

Let's stop talking about this. It's almost off-topic.

Well, no, let's hash this out. It's relevant for the split.

[saki7]

What best practices does it not follow?

target_compile_features

[pdimov]

The autogenerated CML gets the C++ level from meta/libraries.json, and since Spirit classic supports C++03, your meta/libraries.json specifies C++03:

https://github.com/boostorg/spirit/blob/70f968ef6cda4295d65421068457eef3b4308882/meta/libraries.json#L18

If you put something else there, boostdep --cmake will add target_compile_features with the appropriate value.

[saki7]

A "modular CMake system" intuitively suggests that the library is usable with git submodule add and add_subdirectory.

For Spirit's case, it's: git submodule add https://github.com/boostorg/boost.git && git submodule update --init --recursive and add_subdirectory(boost/libs/spirit). I expect it to work out-of-the-box because Spirit is header-only in the first place.

Even if a CMake project requires additional configuration to work, the "best practices" is to define a custom target that runs such commands automatically.

Is Boost's "modular CMake system" (lol) configured to run such commands? No.

I consider it broken. I don't care how others think. This is my personal feeling, as my full life experience of being a developer. I personally consider it broken, and I plan to fix it soon, as a maintainer. I don't care how you argue because you're saying "it's just working in my environment!" and that logically does not mean "it's well-configured based on the best practices". Period.

Can you stop complaining about my decisions in the first place?

If you can't agree with my opinions then we simply can't collaborate.

Are we migrating the repositories or are we continuing to do this internet battle? Can we collaborate? Thanks!

[saki7]

@apolukhin This is how I gave up the constructive discussion with Peter 4 months ago. What do you think?

[pdimov]

Can you stop complaining about my decisions in the first place?

I only care about your decisions when I'm forced to. I would like nothing more than not having to care about them at all.

When you're maintaining a Boost library you are part of a community, and your decisions affect others within it. For example, you can't put whatever you like in your CMakeLists.txt file, as this may break the Boost CMake build, or installation (as two other maintainers recently discovered.) When you up your requirement from C++17 to C++23, C++17 libraries that depend on you are now broken. When we remove the Boost::spirit CMake target, people whose code depends on it will break. And so on.

This is not something that is specific to you; we're all constrained by our existing users and the existing projects that depend on our libraries.

This aside, let's get back to our renaming. Current consensus seems to be libs/spirit becomes latest and greatest (x3 for now), and we make two other repos spirit_classic and spirit_v2?

If I make two empty repos with these names, will that be enough for you to begin work? Or would you prefer I duplicate boostorg/spirit using this procedure: https://docs.github.com/en/repositories/creating-and-managing-repositories/duplicating-a-repository?r603

[apolukhin]

I'd rather put the CMake discussion into a mailing list and concentrate here on a working solution for separation and directory names bikeshedding.

I have my own concerns with current CMake status, but the concerns are mostly related to testing. I have neither an idea how to make it better nor a solid thought on what is causing pain. Give me a few days to express the problem in mailing list...

There are some issues with CMake, @pdimov puts an insane amount of effort to make it better. We can not solve everyting in this issue, so let's concentrate on less global things

boostorg/spirit -- keep, X3
boostorg/spirit_classic -- new repo, Classic
boostorg/spirit_v2 -- new repo, Qi & Karma & Lex (& /repository/)

spirit_v2 name gives an impression that it is more modern than spirit. spirit, spirit_v2 and spirit_x3 seem better

[saki7]

This aside, let's get back to our renaming. Current consensus seems to be libs/spirit becomes latest and greatest (x3 for now),

My original intent was that we could imply that X3 is the default version, by choosing the simplest "spirit" identifier for X3. However, regarding Antony's comment, I realized that the "_v2" suffix contradicts with this implication.

How about this?

• boostorg/spirit_x3 -- renamed from boostorg/spirit
• boostorg/spirit_classic -- new repo
• boostorg/spirit_v2 -- new repo

I think the only technical concern regarding the absence of the plain "spirit" is that existing systems, including user-facing ones, will going to point to the now-empty "spirit" repository.

However,

• We can additionally create a symlink: /boost/libs/spirit -> /boost/libs/spirit_x3 (Note: this is file path on boostorg/boost)
• GitHub will automatically redirect Git requests for boostorg/spirit to boostorg/spirit_x3, as long as we use the GitHub's web UI to rename the repo
• Someone with privileges can configure our HTTP server to redirect /libs/spirit/doc/ to /libs/spirit_x3/doc/ (Note: this is URL)
  • I think this resolves Peter's concern (1)

... so the absence of "spirit" would not affect the end users. Please correct me if I'm wrong. If not, I think we can choose the above scheme.

---

If I make two empty repos with these names, will that be enough for you to begin work? Or would you prefer I duplicate boostorg/spirit using this procedure: https://docs.github.com/en/repositories/creating-and-managing-repositories/duplicating-a-repository?r603

I'd like to start working on the 2 empty repositories.

I think that special command you mentioned (git push --mirror) generates the special "mirrored from" repository, e.g. https://github.com/gcc-mirror/gcc/, so it's not necessary for our case.

[pdimov]

I suspect that we'll have fewer problems if we retain libs/spirit (and boostorg/spirit) to be classic. It's not ideal from discoverability perspective, but if we put a large and prominent notice at the top of the classic documentation pointing to V2 and X3, that would probably be enough to direct users towards the later libraries.

This might also be a good opportunity to consider retaining x3 as is/was (C++17) and make x4 which is the C++23 one - we're paying the splitting costs anyway so now's the time to add one more repo.

[saki7]

@djowel do you feel okay if the name "spirit" will going to point to the now obsolete library Classic? To be honest I feel like below two schemes are almost equally viable and I'd like to hear your opinion.

saki7's idea:

• boostorg/spirit_x3 -- renamed from boostorg/spirit. GitHub and web server handles redirects
• boostorg/spirit_classic -- new repo
• boostorg/spirit_v2 -- new repo

pdimov's idea (please correct me if I'm wrong):

• boostorg/spirit -- keep repo; all files except for Classic ones are deleted. Big text on docs guide users to the other repos
• boostorg/spirit_x3 -- new repo
• boostorg/spirit_v2 -- new repo

[saki7]

To be fair, I'm going to accept Joel's suggestion.

[djowel]

I'm with Peter here, I think. It offers the least disruption, especially for dependent libraries like serialization, which uses 'classic'. We can just have a big, bold message of this fact and link to the latest X3 library.