Split reagents into multiple libraries

[kayceesrk]

Currently, the only way to use reagents is to provide it with a scheduler. However, building a scheduler requires access to low-level capabilities such as compare-and-swaps and lock-free queues, which are part of reagents. Hence, there is a bootstrapping problem.

It would be useful to split the reagents library into

• k-cas library
• non-reagent lock-free data structures library
• core reagents library
• concurrent data/sync structures built on top of core reagents.

[kayceesrk]

Done. See kcas library and lock-free library.

[rgrinberg]

May I ask why you elected for splitting reagents into separate git repos? Seems like you could easily have them all share a git history/issue tracker/mutual compatibility and maintain the modularity. A good build system like jbuilder makes that very easy to achieve as well.

[kayceesrk]

I think part of the motivation is in the original comment. Reagents is functorized over a scheduler which typically depends on lock-free library. So I'd like to pry the dependencies apart. Moreover, a stand alone lock-free library is useful for someone who cares about high performance parallel programs where the power of reagents composition is unnecessary. Even with a multi-repo setting, jbuilder is great at doing builds by bringing in each project in a sub-directory. Multicore doesn't yet support jbuilder, but this should be done soon.

Is there anything else I'm missing that I should consider before splitting the libraries?

[rgrinberg]

So I'd like to pry the dependencies apart. Moreover, a stand alone lock-free library is useful for someone who cares about high performance parallel programs where the power of reagents composition is unnecessary

That's absolutely true. But splitting the package into multiple opam packages that live in the same repository seems to be enough to accomplish your goal. While splitting it into multiple git libraries has some consequences that I think are generally negative.

Even with a multi-repo setting, jbuilder is great at doing builds by bringing in each project in a sub-directory

That may be true, but I still think that multiple repos introduce other issues of decentralization that usually aren't worth the benefits. Like maintaining multiple bug trackers which cause confusion and are annoying to search. Mangled git history when it comes to refactoring that affects multiple projects. You must now maintain inter project version constraints and test needlessly complicated multi version configurations. etc.

Anyway, I kind of realize it's a matter of taste, but I do feel that the decision in favor of splitting has been taken too often. Looking at projects like mirage, that lead to some pretty experiences for end users.