Move taskchampion to its own project and depend on it in Taskwarrior

[djmitche]

The idea here is to make Taskwarrior rely on Taskchampion as a dependency, rather than integrating the two into a single codebase.

This comes from GothenburgBitFactory/taskwarrior#2887 and a few other issues with trying to intermingle Rust and C++ projects. The meaning of "project" is still vague -- is that another repo? Submodules, like with libshared?

We want to optimize for:

• Ease of contribution to Taskchampion, using "normal" Rust stuff like cargo
• Ease of use of Taskchampion by applications other than Taskwarrior
• Conceptual clarity about the API boundary between TC and TW.

What I mean by that last point is that any change to Taskchampion's public API should be reviewed on its own merits, and not just added "because this Taskwarrior functionality needs it." For example, a Task::get_ff4_string() that returned the encoding of a task in FF4 format (the format used in the text-based .task files) wouldn't make sense, since nothing except Taskwarrior uses that format.

[djmitche]

I think we can release Taskwarrior 3.0 before doing this, but I think it's important that this occur.

[aktaboot]

mentioning #3187 for relevance's sake

[aktaboot]

I strongly support the split of taskwarrior and taskchampion, but maybe a clearer list of what has to be done and what has to change would make things clearer.

• Where will taskchampion-lib, integration tests, xtask and sync-server end up ?
• Does some more splitting into repositories make sense ?

I am not yet very familiar with the code base, and probably not the right person to be suggesting this but I feel that further clarification on the split would be helpful. Are there any issues that are blocking, or some necessary clarifications needed to advance on this issue ?

[djmitche]

As you've seen, there are five crates within the Taskchampion workspace.

• taskchampion-lib is closely tied to taskchampion so I think it must be in the same repo.
• xtask is just development utilities (code generation) so should be in the same repo.
• sync-server will, I think, become a reference implementation rather than anything especially "production ready". I suspect most users will be using the new cloud service sync, instead. Since it's an implementation of a specification, it would be OK to move it out of the repo. It also has a bunch of dependencies that are not shared with the other crates, meaning more to download and compile to run cargo test on the whole workspace.
• integration-tests tests integration of taskchampion with sync-server. It probably makes sense to continue to test this, and we should also find some way to test cloud sync, as well as integration with Taskwarrior. But, maybe those should become simple shell scripts instead of a crate (cargo install taskchampion-sync-server; task add foo; task sync; ...)?

My plan was to initially just move all of this back to a single repository -- https://github.com/taskchampion/taskchampion is where it all came from, in fact. That's guided by the idea that generally managing multiple repos gets complicated, and most engineering organizations work from a monorepo these days. LLVM, Chromium, Firefox, Linux -- all monorepos.

Having N repositories means that the space of things to test has dimension N; for example, if we have a taskchampion repo and sync-server repo, then in principle we need to test every version of taskchampion against every version of sync-server, and practically we need to decide which versions to test against one another: just latest of each? semver compatibility? some kind of oldest-supported-version? If we had 3 repositories then it's even harder.

There are a few other questions to think about:

• How does taskwarrior embed Taskchampion? Git submodule? Library dependency? If the latter, is that library a separate shared object, and is that separate shared object a separate deb or rpm package?
• How does the documentation get split up? There's already an mdBook at taskchampion/docs. Probably that should be amended to include information about how to embed Taskchampion, and the Taskwarrior docs updated to point to it.

The thing that is blocking the work is: I think we should get Taskwarrior 3.0 out the door first. Where the code is stored should be largely invisible to the task interface, so I don't think it's a problem to change it in 3.1 or even 3.0.1. I really want to do this, so I'm trying to restrain myself :)

If you're interested in working on this, and especially if you have or can find some useful guidance on the more vague bits above, that would be fantastic! Examples of other Rust libraries designed to be embedded in other languages would be really useful -- most of the examples of FFI I can find are either embedding other things in Rust (e.g., openssl-sys), or embedding an existing Rust library in a very specific context (e.g., using Rust image decoders in Chromium).

[djmitche]

Oh, I didn't mean to imply we would definitely move back to the taskchampion org -- GothenburgBitFactory/taskchampion would be fine, too, if the factory will have it.

[aktaboot]

I think the cleanest way would be to have taskchampion as a submodule, a separate shared object seems unnecessarily complex. Or more precisely: I don't see the benefit of going that route ! Each version of taskwarrior would interop with at least one version of taskchampion.

For the documentation part, it looks a bit trickier, I will look further into it. One idea could be to have independent documentation, and a shared part for the interoperability/integration part etc.

[aktaboot]

Also checked some Rust embedding into Cpp:

Corrosion + CMake

• https://github.com/YosysHQ/nextpnr/blob/master/CMakeLists.txt
• https://codeberg.org/pitbuster/dolphin-rom-thumbnailer/src/branch/main/src/CMakeLists.txt

Rustc + Meson

• https://gitlab.freedesktop.org/mesa/mesa/-/blob/main/src/nouveau/compiler/meson.build

[djmitche]

Good point regarding the submodule. That would also allow embedders to update at their own rate, rather than having to maintain API compatibility on a shared object.

The links to nextpnr and dolphin-rom-thumbnailer are really helpful! I suspect GothenburgBitFactory/taskwarrior#3272/#3273 (Corrosion) and GothenburgBitFactory/taskwarrior#3426 (cxx) will be prerequisites to splitting the repo, as they will result in a much simpler means of embedding Taskchampion.

Meson sounds cool. but I think we should stick to CMake, just to limit the breadth of the change here. I think another embedder could just as easily use Meson as CMake, even using the CXX configuration provided by Taskchampion.

Thanks for your comments on GothenburgBitFactory/taskwarrior#3273. Hopefully we can get that landed soon. The release of 3.0 is sort of stalled waiting for a Taskwarrior maintainer to take some steps, so we can continue to work forward on this stuff independently. If we finish it before 3.0, that's OK!

[djmitche]

GothenburgBitFactory/taskchampion-sync-server#2 mentions splitting sync-server into its own repo, which would allow it to develop at its own pace separately from the Taskwarrior/Taskchampion repos. Let's do that sooner rather than later?

[djmitche]

2a92b2a4b93714dffe838b732c85da4b353c27d4 bulk-imported sync-server from taskchampion/taskchampion, and very little has changed since, except some version bumps. So, I'm not going to try to preserve history, but just create a new repository.

[djmitche]

That new repo is https://github.com/GothenburgBitFactory/taskchampion-sync-server by the way.

[djmitche]

WIP at https://github.com/GothenburgBitFactory/taskchampion. Still to do:

• [x] move relevant issues from this repo
• [x] finish setting up github actions
• [x] set up branch protection similar to this repositor
• [x] verify that other GitHub minutae match (contirbuting guides, security, etc.)
• [x] ping relevant people who have been waiting for / blocked on this
• [x] release a version of taskchampion
• [ ] update Taskwarrior to depend on Taskchampion as a cargo dependency

[djmitche]

Update: I deleted that repo (which was a fork of taskchampion/taskchampion) and instead transferred taskchampion/taskchampion to GothenburgBitFactory/taskchampion. It looks like I've lost permission there, but have pinged someone to see if I can get permission back, at which time I will un-archive it, push the latest changes, and continue with the work above.