Discussion: leveraging machete descriptor at the repository level for long-lived-branches dependencies

[fcamblor]

Overview

I'm fairly new to using Machete (please, don't throw tomatoes at me if I'm wrong in some of my Machete understanding :-))

I understood Machete as a tool to describe dependencies between git branches. These dependencies are stored in the .git/machete file (e.g. "by design" this config won't be shared with my git teammates)

For a long time now, I've been looking for a tool that would allow me to describe my "long-lived branches" dependencies and enforce some rules on them.

Let's take an example that might be more meaningful : we currently have multiple "parallel release development streams" depending on the upcoming features we want to ship in our product :

• Branch 4.2.x corresponds to a "hotfix" branch containing the latest production release (let's say this production release is 4.2.1 and we plan to ship a 4.2.2 with some hotfixes by tomorrow ... that gives you an idea of the timelines)
• Branch 4.3.x corresponds to our next "minor" release containing some small improvements that will be included in our next minor release next month.
• Branch 5.0.x corresponds to our "next major release" for our product, where we'll ship new major shiny features. I expect 5.0.0 to ship in about 3-4 months (but we have already started developing these new big features as they take time to implement)
• Features from 4.3.x should of course be integrated into 5.0.x. And probably, hotfixes from 4.2.x should be merged into 4.3.x (and, thus, merged eventually into 5.0.x)

These branches are what I call some "long-lived branches" since :

• they can have life spans of weeks to months
• these branches are shared among the team members as we all need to know in which version a feature/bug needs to be shipped to

If I understood Machete well, I would design these branches in the following way if I had to design them in my .git/machete :

5.0.x
    4.3.x
        4.2.x

The good thing is : with Machete, I can quickly see if there are any differences between these branches (if a hotfix has been made in 4.2.x, I don't want to miss it in the next 5.0.x release) as Machete is able to quickly identify such commit differences.

My problem is that since Machete stores everything into .git folder, it's not possible to share this configuration with everyone in the team.

Even though I find it useful to have a "local-only" Machete configuration, I would find some added value in being able to read a configuration file from the repository as well.

I don't think subcommands like traverse would be appropriate for such configuration (as I don't think rebasing + force-pushing such long-lived branches would be considered as a good practice), but at least being able to see if some differences have been applied to a bunch of branches at once would be very helpful.

I'm not sure if this idea is in the originsl spirit of Machete (I assume that storing the configuration in a .git folder was done on purpose), but I would be interested to hear your opinion on my use case.

Expected behavior/command output

I would expect to be able to read a .machete file from git root, where some long-lived branch dependencies may be described.

Some Machete subcommands may note take this "shared" configuration into consideration as it should be considered inappropriate.

Actual behavior/command output

Only local Machete configuration is allowed.

git and git machete versions

git --version
> git version 2.38.0

git machete --version
> git machete version 3.14.3

Please cp -r .git .git.bak or there's no hope!

This is not a bug :-)

Additional diagnostics info

This is not a bug :-)

[PawelLipski]

Wow that's quite an original idea, thanks for an extended description 🥰 Lemme address the points one by one:

I'm fairly new to using Machete

Lemme guess... you've got to know about git-machete from Ben Congdon's post? ;)

I understood Machete as a tool to describe dependencies between git branches. These dependencies are stored in the .git/machete file (e.g. "by design" this config won't be shared with my git teammates)

+1, exactly... the underlying assumption that the set of branches that each developer will want to manage under git-machete will be so different that it in typical case it won't make sense to share .git/machete across developers.

If I understood Machete well, I would design these branches in the following way if I had to design them in my .git/machete

:+1:

My problem is that since Machete stores everything into .git folder, it's not possible to share this configuration with everyone in the team. Even though I find it useful to have a "local-only" Machete configuration, I would find some added value in being able to read a configuration file from the repository as well.

Huh indeed, there's no built-in mechanism for sharing branch layout across machines... there're the following concerns:

1. If the file (say, .machete) is to be tracked under git, then it obviously might have different contents depending on the branch. This causes problems: from which branch should git-machete take the contents of .machete? Taking from main/master/other root branch (in git-machete sense) is a natural choice... but usually .machete in such a root branch won't have the information about newer/more downstream branches. If the contents were to be taken from the currently checked out branch... then there's gonna be a lot of corner cases in handling discrepancies between the branches already present in .git/machete, and what's in .machete on the current branch :thinking:

2. Along these lines, there's gonna be a tooooon of corner cases in syncing .git/machete to .machete: a. branch present in .machete but not present locally in git b. branch present in .machete but only present locally in git as a remote branch (but not local branch: origin/foo but not foo) c. branch present in .machete and already present in .git/machete, but having a different parent in each of them (quite likely after a slide-out etc.) d. ....

I'm not sure if this idea is in the original spirit of Machete (I assume that storing the configuration in a .git folder was done on purpose), but I would be interested to hear your opinion on my use case.

So! While I'm reluctant to add a first-class support for syncing branch layout across machines, what I might suggest for cases like yours is:

1. Do you have GitHub PRs opened for your release branches? If so, then your team might use git machete github checkout-prs [flags...]; this kind-of uses GitHub PRs as a centralized place to store the relations between branches, while partially addressing the problems from (2.) above
2. Still, keeping .machete in your repo and then having a team-wide procedure for manually syncing .git/machete with .machete might be feasible... I'm just hesitant if providing a one-size-fits-all solution officially in git-machete is a good idea :/

Let me know what you think!

[fcamblor]

Lemme guess... you've got to know about git-machete from Ben Congdon's post? ;)

Didn't heard of the blog post before : I read it and we're more or less on the same page. The only difference between our visions is about time : PR (particularly, small stacked ones) are supposed to be reviewed more or less quickly (otherwise this will become a nightmare)

In my case, long-lived branches are supposed to have a long timeline "by design", and the idea is not to stack too much of these long-living branches altogether. I'd say that if we have a depth of 3-4 is the max (in my example, we're working with 3 parallel long-living branches with different development paces)

1. If the file (say, .machete) is to be tracked under git, then it obviously might have different contents depending on the branch. This causes problems: from which branch should git-machete take the contents of .machete? Taking from main/master/other root branch (in git-machete sense) is a natural choice... but usually .machete in such a root branch won't have the information about newer/more downstream branches. If the contents were to be taken from the currently checked out branch... then there's gonna be a lot of corner cases in handling discrepancies between the branches already present in .git/machete, and what's in .machete on the current branch 🤔

I would go for the second option : take content based on currently checked-out branch. I don't see any other viable option (first option has a lot of drawback to me)

Regarding the discrepancies between local and shared machete files (which should also be applicable to 1st option), I was considering "concatenating" both files into 1 "virtually-merged" one.

I mean, if you have something like :

# .git/machete : 
dev
  1234-my-feature
  2345-my-another-feature

4.3.x
  3456-my-small-feature

# .machete : 
5.0.x
  4.3.x
    4.2.x

then Machete would behave exactly as if developer would have a .git/machete file containing :

dev
  1234-my-feature
  2345-my-another-feature

4.3.x
  3456-my-small-feature

5.0.x
  4.3.x
    4.2.x

At the moment, this would make machete fail because branch 4.3.x cannot appear twice in the machete definition (maybe this could be improved by implementing kind of "declaration merging" of branches across the same config file... but that's another story)

I imagine this makes your corner cases exposed on point 2. more straightforward as it shouldn't raise new questions on your actual behaviour.

WDYT ?

[PawelLipski]

Okay! makes sense indeed... by virtually merged you mean that certain commands (like status, but excluding e.g. traverse) would also include the branches imported from .machete right?

[fcamblor]

Yep :-)

You would have to identify, amongst every machete subcommands, which ones are "shared-config"-friendly and which ones are not.

I suspect this might introduce some complexities on the user side... I'm not sure how this could be identified so that users understand it easily (I need to think about it :-) )