git-octopus
git-octopus copied to clipboard
lesfurets
Migrate the project to a proper language
Needs
- Better testability. For instance, I would like to mesure test coverage.
- Get some abstractions because bash is just a mess.
- Something easier to debug and develop
constrains
The biggest constraint is that people are currently using git-octopus on various OS : macOS, ubuntu, Windows... If we want to use a compiled language, would it be easy to distribute binaries ?
I'm considering python3 with pygit2 for git stuff.
Don't hesitate to give opinions :)
+1
In which langages are implémented such extensions ?
Dimitri BAELI
Le 18 nov. 2016 à 18:11, Arnaud Pflieger [email protected] a écrit :
Needs
Better testability. For instance, I would like to mesure test coverage. Get some abstractions because bash is just a mess. Something easier to debug and develop constrains
The biggest constraint is that people are currently using git-octopus on various OS : macOS, ubuntu, Windows... If we want to use a compiled language, would it be easy to distribute binaries ?
I'm considering python3 with pygit2 for git stuff.
Don't hesitate to give opinions :)
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.
gitflow is in bash and has no tests.
In git/contrib there's bash, python and pearl.
I looked on homebrew packages prefixed by git-
- git-archive-all : unique python file directly copied to /usr/local/bin via make
- git-cal : unique pearl script directly copied to /usr/local/bin via make
- git-credential-manager : java program. Declare a brew dependency on java. The formula installs a jar that is an artifact of github release. Puting artifact here is manual I guess but it's interesting.
- git-crypt : cpp. installed the same way as git-octopus after compilation
- git extra : bash scripts copied to /usr/local/bin via make
- git-fixup : bash script copied to /usr/local/bin via make
- git-fresh : bash script copied to /usr/local/bin via custom script
- git-ftp : bash script copied to /usr/local/bin via make
- git-gerrit : bash script installed the same way as git-octops
- git-imerge : single python script copied via make
- git-lfs : written in go. Built locally and installed by homebrew
- git-plus : multiple python scripts using shared piece of code
- git-series : written in rust. Runs a local 'cargo' build and install the binaries via homebrew
- git-tracker : build and install standalone ruby package with rake
I skiped a lot of bash projects that are all using the same installation steps
Most of them are written in bash and are standalone files. They are simply copied as binaries. So are python or pearl scripts. A few of them are compiled in go, cpp...
Go is interesting (devops tooling, lightweight), would it fit the dev need ? Python is obvious if not too big or complex for installing this on a CI agent (compatibility version for ex).
Which of those have tests ? :-)
Dimitri BAELI
Le 21 nov. 2016 à 15:25, Arnaud Pflieger [email protected] a écrit :
I looked on homebrew packages prefixed by git-
git-archive-all : unique python file directly copied to /usr/local/bin via make git-cal : unique pearl script directly copied to /usr/local/bin via make git-credential-manager : java program. Declare a brew dependency on java. The formula installs a jar that is an artifact of github release. Puting artifact here is manual I guess but it's interesting. git-crypt : cpp. installed the same way as git-octopus after compilation git extra : bash scripts copied to /usr/local/bin via make git-fixup : bash script copied to /usr/local/bin via make git-fresh : bash script copied to /usr/local/bin via custom script git-ftp : bash script copied to /usr/local/bin via make git-gerrit : bash script installed the same way as git-octops git-imerge : single python script copied via make git-lfs : written in go. Built in bin within the project and installed in bin by homebrew git-plus : multiple python scripts using shared piece of code git-series : written in rust. Runs a local 'cargo' build and install the binaries via homebrew git_tracker : build and install standalone ruby package with rake Most of them are written in bash and are standalone files. They are simply copied as binaries. So are python or pearl scripts. A few of them are compiled in go, cpp...
— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
I spent some time looking at Go actually. A very interesting thing is that the compiler allow to build binaries for all systems, see. So the distribution of binaries would be quite easy. It seems to me that the downside of go currently is dependency management but since git-octopus would use only a few (maybe just one), we could deal with it
Both go and Python can be easily tested.
I have doubt on python now, packaging seems not that great
Go for go ! ☺
Dimitri BAELI IT Associate Director
[cid:[email protected]]https://www.lesfurets.com/ 12-14 rue Médéric 75017 Paris +33 1 56 02 36 50 LesFurets.comhttps://www.lesfurets.com/ Facebookhttps://www.facebook.com/hervelesfurets - Twitterhttps://twitter.com/lesfurets
From: Arnaud Pflieger [email protected] Reply-To: lesfurets/git-octopus [email protected] Date: Tuesday, November 22, 2016 at 10:41 AM To: lesfurets/git-octopus [email protected] Subject: Re: [lesfurets/git-octopus] Migrate the project to a proper language (#25)
I spent some time looking at Go actually. A very interesting thing is that the compiler allow to build binaries for all systems, seehttps://golang.org/doc/install/source#go14. So the distribution of binaries would be quite easy. It seems to me that the downside of go currently is dependency managementhttps://github.com/golang/go/wiki/PackageManagementTools but since git-octopus would use only a few (maybe just one), we could deal with it
Both go and Python can be easily tested.
I have doubt on python now, packaging seems not that great
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/lesfurets/git-octopus/issues/25#issuecomment-262192809, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ACyNWd9DRJtSg5FgOn-_-Z4nQ3w0hWKCks5rArjBgaJpZM4K2tjH.
+1 for leaving as bash.
We use git-octopus in our CI environment and bash means our docker image can remain around 100MB
Is it a matter of size ? A compiled binary shouldn't be bigger than current scripts (I guess)
size and dependencies. I wouldn't want to install python just for octopus and binaries need compiling unless you will ship precompiled binaries, in which case what about dynamically linked libraries? We are using Alpine Linux docker image for size and its certainly not compatible with redhat/ubuntu precompiled dynamic binaries.
having said that, the current bash implementation depends on perl because it uses shasum and some perl regex logic.. it would be nice to see that move to sha1sum which is provided by coreutils and the search and replace done by sed instead of perl if you were staying with bash.
Is Go a blocker in your case ?
Dimitri Baeli
From: durzo [email protected] Sent: Thursday, November 24, 2016 2:35:09 AM To: lesfurets/git-octopus Cc: LesFurets.com; Comment Subject: Re: [lesfurets/git-octopus] Migrate the project to a proper language (#25)
size and dependencies. I wouldn't want to install python just for octopus and binaries need compiling unless you will ship precompiled binaries, in which case what about dynamically linked libraries? We are using Alpine Linux docker image for size and its certainly not compatible with redhat/ubuntu precompiled dynamic binaries.
having said that, the current bash implementation depends on perl because it uses shasum and some perl regex logic.. it would be nice to see that move to sha1sum which is provided by coreutils and the search and replace done by sed instead of perl if you were staying with bash.
You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/lesfurets/git-octopus/issues/25#issuecomment-262670012, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ACyNWUJ3PDqKyi1LzluzZ7q4odhgF6FWks5rBOnNgaJpZM4K2tjH.
I agree that dependency on python is problem, especially the version. Go seems to do the job. In terms of dependencies, the only one that could be interesting is libgit2 for accessing git primitive instead of running git commands but that's not on the table for a first step.
The plan would be to have precompiled binaries. Here's what go claims to be capable of
| OS | architecture |
|---|---|
| android | arm |
| darwin | 386 |
| darwin | amd64 |
| darwin | arm |
| darwin | arm64 |
| dragonfly | amd64 |
| freebsd | 386 |
| freebsd | amd64 |
| freebsd | arm |
| linux | 386 |
| linux | amd64 |
| linux | arm |
| linux | arm64 |
| linux | ppc64 |
| linux | ppc64le |
| linux | mips64 |
| linux | mips64le |
| netbsd | 386 |
| netbsd | amd64 |
| netbsd | arm |
| openbsd | 386 |
| openbsd | amd64 |
| openbsd | arm |
| plan9 | 386 |
| plan9 | amd64 |
| solaris | amd64 |
| windows | 386 |
| windows | amd64 |
We would provide all of them. So do git-lfs https://github.com/git-lfs/git-lfs/releases Alpine Linux should be able to run linux binaries I guess.
Talking about perl dependency, here's when I introduced it https://github.com/lesfurets/git-octopus/commit/ad4932dc3a95272edeb7c80bebaf8b1e990d6676 There was an incompatibility of sed between ubuntu and macos if I remember well. MacOS allowed only -e "blblbl" whereas ubuntu -e"blblbl" (with no space). That's ridiculous I know, If you have a better solution ;D
Same for shasum, I don't have sha1sum on macos, so shasum looked better for interop
Go binaries should be fine, its just the versions of dynamically linked libraries that I worry about, but it should be OK - happy to give it a try!
I dont use mac so wasn't aware of those missing binaries, makes sense now. The only thing you can do in that case is have an OS detection function first that sets paths to the required binaries for each platform.
see https://gist.github.com/durzo/892c4f81b255c1f24daba92922224db0 for an example
I hope we can build binaries without any dependencies. Go standard library has regex for sure. For sha1 sums computations, we should be able to embed the code (so do git itself)
Actually, go binaries are not that lightweight. I compiled a Hello World, 1.5 MB. git-lfs is 3 MB.
We would have something around 2 MB maybe.
I'll might be there tomorrow https://www.meetup.com/gdg-paris/events/235668597/
Update: I released a first beta version written in go https://github.com/lesfurets/git-octopus/releases/tag/v2.0-beta.1
I still miss the conflict resolution feature. I'm not actively working on it :/ I have a question for you all: Until know, git-octopus and git-conflict are 2 separate commands based on 2 scripts. This is quite ugly to do in go because it would require 2 different binaries. What do you think about having a single binary with subcommands ?
You mean : git-octopus conflict git-octopus merge pattern
That's good for me
Dimitri BAELI
Le 29 mai 2017 à 19:56, Arnaud Pflieger [email protected] a écrit :
I still miss the conflict resolution feature. I'm not actively working on it :/ I have a question for you all: Until know, git-octopus and git-conflict are 2 separate commands based on 2 scripts. This is quite ugly to do in go because it would require 2 different binaries. What do you think about having a single binary with subcommands ?
— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.
Yes.
Actually I consider to have a seperate binary that will not integrate with git directly (not named git-octopus) and we could provide small shell scripts that offer backward compatibility
Given a binary somebinary that would be in the $PATH
git-octopus
#!/bin/bash
somebinary merge "$@"
and git-conflict
#!/bin/bash
somebinary conflict "$@"
Sounds good.
The ambition is to grow somebinary to a bigger project. This is what I talked about to some of you on my https://github.com/apflieger/tie prototype (which has no documentation and is quite obscure to understand, I admit that)
As a first step, migrating to cobra (https://github.com/spf13/cobra) would probably help in writing a more consistent CLI.
yes, that's what i'm using here https://github.com/apflieger/tie. I use libgit2/git2go as well. That's the plan if I merge this project in git-octopus