Unify how crates within the workspace depend on each other

[thomaseizinger]

1. Should all workspace dependencies express a version number? We currently sometimes include and sometimes we don't.
2. We mix two styles of depending on workspace crates: a. Depending on the libp2p meta crate and activating certain features b. Depending on the crate directly c. In some cases, we even do both.

Tasks of this issue:

1. Figure out, how this should ideally be done. Take into account release process work, how easy it is detect breaking changes etc
2. Document it in this issue.
3. Implement the solution consistently across the workspace.

cc @mxinden I know you have a release script. Would be good to know which assumptions that one makes f.e.

Folks who have experience in maintaining big workspaces are also welcome to comment.

[mxinden]

Should all workspace dependencies express a version number? We currently sometimes include and sometimes we don't.

Oh, are we inconsistent here? I thought we have version numbers on all dependencies and omit on all dev-dependencies.

2. We mix two styles of depending on workspace crates: a. Depending on the libp2p meta crate and activating certain features b. Depending on the crate directly c. In some cases, we even do both.

For direct dependencies I think crates should depend on the sub-crates. For dev-dependencies I don't have an opinion, though I leaning towards using the root dependency libp2p as a user would most likely use it through libp2p as well.

Folks who have experience in maintaining big workspaces are also welcome to comment.

Oh, yes please. Curious how other large projects solve this.

cc @mxinden I know you have a release script. Would be good to know which assumptions that one makes f.e.

A VERY messy release script, though it does its job. With the worry that no one will ever trust my coding skills again, here it is:

Script to bump versions across workspace

Bump the version of one crate via its Cargo.toml and then run bump.py libp2p-$CRATE_NAME.

#!/usr/bin/env python3

from pathlib import Path
from tomlkit import loads
from tomlkit import dumps
from itertools import tee
import semver
import sys

rootdir = Path('./')

indent = ""

def update_dependents(dependency_name, level):

    file_list_1, file_list_2 = tee(filter(lambda f: not "target" in str(f), [f for f in rootdir.glob('**/Cargo.toml') if f.is_file()]))

    dependency_version = ""
    dependency_changelog_path = ""
    for f in file_list_1:
        cargo_toml = loads(f.read_text())
        if cargo_toml["package"]["name"] == dependency_name:
            dependency_version = cargo_toml["package"]["version"]
            dependency_changelog_path = Path(f.parents[0]).joinpath("CHANGELOG.md")
            break

    print(level + "Updating all dependents of " + dependency_name + " to " + dependency_name + " " + dependency_version + ".")

    level = level + " "

    for cargo_toml_file in file_list_2:
        cargo_toml = loads(cargo_toml_file.read_text())
        dependent_name = cargo_toml["package"]["name"]
        print(level, "Checking: " + dependent_name)
        if dependent_name != dependency_name:
            if "dependencies" in cargo_toml:

                if dependency_name in cargo_toml["dependencies"] and cargo_toml["dependencies"][dependency_name]["version"] != dependency_version:
                    cargo_toml["dependencies"][dependency_name]["version"] = dependency_version
                    cargo_toml_file.write_text(dumps(cargo_toml))

                    changelog_file = Path(cargo_toml_file.parents[0]).joinpath("CHANGELOG.md")
                    if changelog_file.exists():
                        text = list(open(changelog_file))
                        dependent_version = semver.VersionInfo.parse(cargo_toml["package"]["version"])

                        # Update dependent version based on changelog header.
                        if "[unreleased]" in text[0]:
                            if dependent_version.major == 0 and dependent_version.patch == 0:
                                # Nothing to be done, already bumped.
                                pass
                            elif dependent_version.major == 0 and dependent_version.patch != 0:
                                dependent_version = dependent_version.bump_minor().replace(patch=0)
                                text.pop(0)
                                text.insert(0, "# " + str(dependent_version) + " [unreleased]\n")
                                text.insert(1, "\n")
                            elif dependent_version.minor != 0 or dependent_version.patch != 0:
                                dependent_version = dependent_version.bump_major().replace(patch=0, minor=0)
                                text.pop(0)
                                text.insert(0, "# " + str(dependent_version) + " [unreleased]\n")
                                text.insert(1, "\n")
                            else:
                                # Nothing to be done, already bumped.
                                pass
                        elif "# " in text[0]:
                            # Bump version of dependent
                            if dependent_version.major == 0:
                                dependent_version = dependent_version.bump_minor().replace(patch=0)
                            else:
                                dependent_version = dependent_version.bump_major().replace(patch=0, minor=0)

                            text.insert(0, "# " + str(dependent_version) + " [unreleased]\n")
                            text.insert(1, "\n")
                        else:
                            pass

                        if dependent_name == "libp2p":
                            text.insert(2, "- Update to [`" + dependency_name + "` `v" + dependency_version + "`](" + str(dependency_changelog_path) + "#" + dependency_version.replace(".", "") + ").\n")
                        else:
                            text.insert(2, "- Update to `" + dependency_name + "` `v" + dependency_version + "`.\n")

                        text.insert(3, "\n")

                        changelog_file.write_text("".join(text))

                        cargo_toml["package"]["version"] = str(dependent_version)
                        cargo_toml_file.write_text(dumps(cargo_toml))

                        update_dependents(dependent_name, level + " ")



def main():
    update_dependents(sys.argv[1], "")


if __name__ == '__main__':
    main()

[thomaseizinger]

Should all workspace dependencies express a version number? We currently sometimes include and sometimes we don't.

Oh, are we inconsistent here? I thought we have version numbers on all dependencies and omit on all dev-dependencies.

That might be true, I need to check again.

2. We mix two styles of depending on workspace crates: a. Depending on the libp2p meta crate and activating certain features b. Depending on the crate directly c. In some cases, we even do both.

For direct dependencies I think crates should depend on the sub-crates. For dev-dependencies I don't have an opinion, though I leaning towards using the root dependency libp2p as a user would most likely use it through libp2p as well.

Agree with that.