vendorize
vendorize copied to clipboard
kisielk
Write out file containing description of source package
Hey Kamil, I've been thinking of adding a feature and I wanted to run it by you. I want to write out a small file (vendorize.txt?) in the root of each vendored package containing a description of source package. In particular, I want to include the git sha and whether the tree was dirty (or equivalent for other VCS).
The reason I want this is so that I know what revision a vendored package comes from when I'm hunting down a bug or when I want to update the library.
The details here are a little tricky. We can try to use golang.org/x/tools/go/vcs and figure out the full vcs info from that, or we can use our own heuristics. I can imagine that there's some failure cases so it might be a best-effort kind of thing (you'd probably want vendorize to work even if the source package isn't in version control, for example.)
Also, if this feature does sound good to you, would you be open to accepting an initial PR that only has git support? It's the only VCS I care about and testing it with others is a bit of a hassle :)
Related to #6.
Yeah I think it's a good idea overall. I would be interested in seeing what comes of the golang-dev discussion (https://groups.google.com/forum/#!topic/golang-dev/nMWoEAG55v8%5B201-225%5D) to come up with a standard spec for a dependency file... that's probably the spec we'd ultimately want to be supporting.
So basically I'm not sure if we should implement something now or wait till some consensus is reached there. It sounds like they could be deciding for the 1.5 release
Cool, it sounds like we're on the same page here. At work I've been setting up a "vendor everything" project configuration and I'm trying to make anticipate what will become the officially recommended vendoring solution as indicated in that thread; this led me to start using vendorize. So I too would like to see either (1) vendorize implement the standard file format and vendoring behavior as recommended by the Go team or else (2) the go tool handle vendorizing (as hinted for 1.6+ by Brad in the thread).
That said, reading between the lines a little it sounds like the official vendoring solution will be a little different from how vendorize works today. Vendorize grabs whatever it happens to find in the user's GOPATH; it sounds like the system envisioned by Brad et al is that a config file will specify the dependencies and the vendoring tool will download them(?), ensuring the exact specified revisions, and rewrite imports. (So somewhat analagous to how go generate works.) In that world, of course, the thing I'm proposing is unnecessary; the vendor config file specifies the versions and you rerun the vendoring tool whenever you change that file.
I'd say that the vendorize.txt thing is useful as long as vendorize retains the "grab whatever's in the GOPATH" behavior, and since it might be 6 months or a year before the official thing is fleshed out, it will be some time before vendorize implements the new behavior (if you want to make such a large change to how vendorize works at all).
I propose we add vendorize.txt, and if you end up changing vendorize to implement the official vendoring solution we can remove it. WDYT?
That seems reasonable to me.
The reason I wrote vendorize to just grab whatever is currently in your GOPATH is because I felt it fit my workflow. Generally if I'm developing something I'll just "go get" whatever packages I need as I need them and develop against that. Then when I want to lock down the dependencies it's nice to just run one command and essentially preserve the state of what I have. I don't really have to worry about the details of the specific revisions, it's just whatever is working at the time. Of course this doesn't account for all possible future usages such as updating but it seems to have been working well enough for my purposes for now.
I think the system proposed on golang-dev is not necessarily the way you will specify what dependencies you will be pulling in to your project. It's just a common format that tools will understand and be able to generate or consume, so I think it would still work with the vendorize model. Basically it would take the place of this vendorize.txt.
Given that the proposal is to have just one file and it looks like the preferred format will likely be JSON, what do you think about just writing a single JSON file to the directory into which the vendorized projects are copied? At least then when a solution is settled upon it won't be as large of a change to migrate.
The reason I wrote vendorize to just grab whatever is currently in your GOPATH is because I felt it fit my workflow. Generally if I'm developing something I'll just "go get" whatever packages I need as I need them and develop against that. Then when I want to lock down the dependencies it's nice to just run one command and essentially preserve the state of what I have.
Yep I agree; the behavior is what I'd want (and it's why I started using vendorize). I'd just like a hint about what original vcs version the source came from, since vendoring removes vcs metadata.
I think the system proposed on golang-dev is not necessarily the way you will specify what dependencies you will be pulling in to your project. It's just a common format that tools will understand and be able to generate or consume, so I think it would still work with the vendorize model. Basically it would take the place of this vendorize.txt.
Yes, quite possibly. I was trying to extrapolate a bit from the proposal, but this sounds equally reasonable.
Given that the proposal is to have just one file and it looks like the preferred format will likely be JSON, what do you think about just writing a single JSON file to the directory into which the vendorized projects are copied?
JSON sounds fine. I like the idea of a single top-level file, but how would it handle the scenario in which you vendor the dependencies of multiple libraries into the same place? (We do this at work.) Example:
vendorize mycompany/proj1 mycompany/internal
vendorize mycompany/proj2 mycompany/internal
You wouldn't want the second vendorize command to blow away the JSON file written by the first one. So would you write out the union of old and new? This gets a bit tricky when you remove libraries; if you rm -rf mycompany/internal/unneeded/dep, then you have to manually remove unneeded/dep from the JSON file. Or would vendorize check for the existence of all the entries before writing out the new unioned JSON?
Yes it would have to update the same file. Removal is tricky because you could, for example, remove the dependenices and not update with vendorize ever again. Then again, you're also free to copy stuff in to the directory and not update it either... I don't think there's anything we can do to enforce that.
I think we would just provide a vendorize remove command to manually remove things from the file. It should be an infrequent operation in any case...