project-layout

project-layout copied to clipboard

The README makes clear that this is not official, but even the claim "it is a set of common historical and emerging project layout patterns in the Go ecosystem" is not accurate.

For example, the vast majority of packages in the Go ecosystem do not put the importable packages in a pkg subdirectory. More generally what is described here is just very complex, and Go repos tend to be much simpler.

It is unfortunate that this is being put forth as "golang-standards" when it really is not. I'm commenting here because I am starting to see people say things like "you are not using the standard Go project layout" and linking to this repo.

@rsc could you please share a few links to some trustworthy (from your point of view) resources to learn more about the best practices for writing "idiomatic " Go code? Thank you in advance.

@iarosb, I always recommend anyone starting with Go to read

https://golang.org/doc/effective_go https://github.com/golang/go/wiki/CodeReviewComments https://github.com/golang/go/wiki https://golang.org/ref/spec

I feel relieved to see Russ Cox opening this issue! I always hated how this repository sells the idea of being something official and how inaccurate it is.

Quite a few times, I had a hard time because of its "official-ish" appearance – it makes developers, especially people starting with Go, blindly follow whatever recommendation is here. Awful!

@henvic the reason I ended up here asking for links is that I've seen many developers referring to this repo as if it has some official status, but it seems to be very questionable in a way it relates to the core ideas behind Go's creation. I'll go through those resources and your help is much appreciated.

What @rsc describes here is a problem I observed many times. I saw many projects trying to "fix" their layout based on what's provided as a reference here. It creates confusion and conflict among contributors. The project is doing a good job having a disclaimer in the README, but it doesn't seem to be effective because many people don't read the text and assume this is an officially endorsed project.

I am 100% aligned with both @rsc and @rakyll on their comments. We experience the same misconception about the status of this repository in the Slack workspace, especially for those who are new to the language and are trying to figure out how to do things "right". I am not sure that I get a sense, at least in our space, that it's getting any worse, but it's frequent enough that I would be comfortable classifying it as a problem.

Considering the feedback being given in this issue, @kcq how would you feel about moving this repository out of the current Organization and onto a different one (or your username) to benefit the larger Go community? I'd actually be happy to provide your repo as a resource for Gophers if it wasn't asserting itself as a standard.

Edit: I had forgotten I filed this awhile back, regarding the pkg/ directory: #10

I usually point people to https://eli.thegreenplace.net/2019/simple-go-project-layout-with-modules/, and have done so in several issues in this very repo.

You've had several long-time gophers and a couple of go maintainers chime in now. It would be really great if you could archive this repo or, at the very least, move it to a less-misleading org. I don't like saying it, but overall community impact is negative at this point. It's hurting more than helping.

The issue is the different requirements between a simple system package (which I tend to think the Go team over optimizes for) and more complex apps and services. Every company I've worked at that had larger apps written in Go has a repo that looks very similar to this. Even the pkgsite repo uses some of the structure that this repo recommends. Obviously I would not use this structure for a small system library that isn't meant to be used in a standalone way but in my experience a lot of what this repo recommends is very valid and preferred for large apps.

I think the primary issue is that the org is titled golang-standards which, combined with its age, lends it a certain credibility (not to mention SEO) that is unwarranted and has long been a source of confusion for new developers.

Differing requirements is certainly a thing to be aware of, but they're impossible to capture in a single directory hierarchy, and that's part of the problem. This repo is complex and, despite periodically stating that features are optional in documentation, has a monolithic structure which comes across as prescriptive. This is at odds with the language, IMO. Go is a relatively simple language to get started with, and this is not a good jumping off point. I've personally helped several developers rekindle their waning interest by showing them a simpler way to lay out their code. I'm sure many other folks in community forums have had similar experiences.

For those that do persist, as mentioned above, they often cause further confusion by directing others to follow these guidelines in the form of PRs, issues, or just well-intentioned advice the recipient doesn't know any better about.

It would be really great if at the very least the org name could change. For the reasons discussed above, this is not a very good standard, much less the standard.

I am in support of the idea this project represents, but these conversations bring up one thing to me. Why doesn't something like this get published by the Go team. It doesn't have to be complex, but I feel like there is a gap in the documentation that this repo partially fills. I stumbled on this when I was in search of this information, and it was extremely helpful. I'm well past this point in my learning of Go, but I have to assume others could benefit from an official version of sorts of this type of information.

I am in support of the idea this project represents, but these conversations bring up one thing to me. Why doesn't something like this get published by the Go team. It doesn't have to be complex, but I feel like there is a gap in the documentation that this repo partially fills. I stumbled on this when I was in search of this information, and it was extremely helpful. I'm well past this point in my learning of Go, but I have to assume others could benefit from an official version of sorts of this type of information.

Because the Go team cannot know all uses of Go and what project layout each type of project would benefit the most. In general, just tossing a few .go files in the same dir will take you very far.

I think I missed something in my original comment. This was at a time when I was moving from small one off projects, to something significantly larger at work while I was still learning an uncertain about what I was doing. I don't know the best solution, but it seems like there could be at least a bit more in that regard. Honestly even if it was a wiki page with some "hey this could be useful" comments regarding laying things out when the time comes.

This blog post has always been a great place to start, and it's worth noting that @eliben, whose article I posted above is in the Golang organization. There's also this post by @rakyll (former Go Team member) and this one by @peterbourgon. These articles may not be quite as specific or official as some people might want, but like our earlier poster mentioned, everyone's needs are different, and the official guidelines have had a correspondingly broader scope.

Should is a tough concept in software engineering, and once you say things should be done a particular way, it's tougher to evolve and to develop new best practices. The more specific and proscriptive you are, the tougher it becomes, from my experience. Far better to have clear, philosophical guidelines describing the guiding principles of the language and how those can shape the reusable components you write. And I do think we have that.

Put another way: "it depends", and giving it a proper treatment involves, ideally, multiple case studies on "well structured" packages that take design, intent and API footprint into consideration, along with how those decisions were arrived at, why, and how they may have evolved through the life of the project. Perhaps theres some room for this in the Go Blog, but even then the answer is still "it depends", and what works for one package might not work well for another, even if the guiding principles are the same.

And, of course, when I google "structure golang programs", this repo is still on top, which is the crux of #117

22k stars shows there is a community need for something like this. I understand the "it depends" camp for complex projects, but there are guidelines that most people can agree with. For example, the vast majority of simple go projects, when they are starting out, have a go.mod at the root path with a package that probably matches the github repository. There's probably a LICENSE file and README file. They probably should check in their go.sum file. There should probably be a package_test.go file.

There are other best practices where their existence is important, even if the details are not. For example, almost all have CI, but we don't have to make a decision on which CI is best.

With 22k stars, the community is signaling a strong need for guidance on how to structure go projects. I believe it would be high leverage for an official voice, more recent than 2012, with access to golang.org, to guide people here.

I am not sure we can assume the user's intent when they star a project. I know for many it signals an interest to understand or explore a project later. It does not mean that they have a need for it. I star some projects I never use, nor ever have a need for.

@rsc Does the Go team have an official project layout pattern?

Go doesn't have an official project layout.

I always use the layout like here: https://github.com/golang/go/tree/master/src and itś simple: a folder with that package and all related.

Go is simple, I don't know why developers want to use patterns of others languages here.

@xhit considering we have go fmt to consistently style source code, I could see some then wanting or even expecting to find a style to use consistently to structure their project. I think there are some patterns we've discovered over the years that can help projects be more easily maintained, but I think that's much different that some sort of one-size-fits-all "standard" layout.

I think an excellent start to anything even remotely like this could just be to point out the resources you've all been dumping as alternatives. There's all this great info spread all over. Should be enough to at least have a wiki page that you can point people to, when they've been mislead. It's also a good option incase this org sticks around longer.

I think discussions about a standardised project layout miss the point. Go encourages you to derive package names from the last path component. When importing multiple packages, you want the names of the packages you import to be unique to avoid naming collisions. A standard layout would mean that it's very likely that many of the packages have the same, standard names and hence collide. This is to be avoided. So in order to make for a good programmer experience, the layout must not be standard but rather custom and different for each project.

I believe that people overthink this way too much. Just let your projects grow organically. Except for a few hard coded directory and file names (like vendor, testdata, and go.mod) you are free to chose whatever directory names you like. You should make full use of this freedom and put your stuff into whatever directory name seems most appropriate for the problem at hand. And if this choice turns out to be a poor one, you can always refactor later.

I think that this stems from new people coming to the language looking for guidance on best practices for project layout out. This is especially true if you come from a very well structured world like Django, Flask or Rails and so on where project structure is defined for you, this is where you put your handlers and your models go here etc etc.

Coming to go where that safety net does not exist can be a little daunting for new comers, for myself coming to go from the Django world I struggled with how to layout projects, "I was always asking myself am I doing it right?" and while the correct answer is that there is no right way, only the way makes sense for what you are building, the golang-standards/project-layout gives a guide to how you might start and grow from there.

I've long stopped laying out my projects specifically like this because I've grown in experience and naturally figure out project layout on the fly and it's naturally evolving as the codebase grows and evolves, things will move around a bit and thats ok.

Perhaps the issue is that this project aims to be THE way to layout a go project when really it's more like a here is a place to start to inspire you. It's certainly not the standard way though.

Not long ago, I also encountered such a problem. I used to be engaged in the development of other languages ​​and have certain development experience. When I want to use golang to develop a project, and for this to establish a good code directory structure, I am confused. Later, after looking at this project and understanding the compilation rules of golang, I developed a "standard layout" of my own project.

As a young gopher, I prefer simplicity,i like the layout list :

• https://github.com/golang/go/tree/master/src
• https://blog.golang.org/package-names
• https://golang.org/doc/effective_go

Yeahh, let's delete this missguided repo

I have some sympathy with @rsc about this (OK, a lot of sympathy). I don't think the layout shown in this repo represents either the best way to structure Go projects, or even the most common way (those are quite different things, as you'll appreciate).

That said, though, I guess the author of this repo can publish whatever he wants, even describing it as a "standard", and no one can say boo about it—not even Russ. That's the joy and the hurt of open source. No one is obliged to pay any attention to the maintainer's ideas about Go project structure, and the flipside of that is that he's not obliged to pay any attention to us. If someone thinks they can do better (and the Go team would seem to be an obvious candidate for this), let them go ahead and do so.

I read about a dozen blog posts a day about Go which are (in my opinion) at best misguided, and at worst positively harmful. It would be a full-time job for me to try to get the authors to change their minds, and posts, and it's none of my business anyway. Instead, I publish my own opinions, and let Gophers decide whose ideas they like best. That seems to me a sensible way of going about things.

@bitfield The org should be renamed. It's that simple.

It's misleading, it harms devs new to Go.

It's ok that Go doesn't have a standard project structure. Any given project's structure is a function of its size, complexity and type.

Go is versatile, so it makes sense that there would be a bunch of different structure choices depending on what exactly your project is.

It also fits that new devs would be looking to better organise their code once the single main.go starts to get a bit out of hand, especially when coming from a file per class language. They search Google, they find this repo. Because of its name not its readme.

It's very easy to miss that comment in the readme. It's by no means a sufficient disclaimer. That comment is a massive copout from the owners because they know they'll get no further traction if they renamed the org. Maybe they should instead focus on demonstrating how different potential project layouts could fit different use cases, and make it actually useful to new Gophers with some signposting of that nature. "Start here with a basic layout", "here's one with a bit more organisation", etc.

@m2q The standard opiniated directory structure for Go is no directory structure. Name every package according to its function. Chose unique names, not names everybody else uses. No fun programming if you import 10 packages named util. And do not add boilerplate path components like src, util, or pkg. Nobody needs that.

Perhaps the only thing resembling a convention is having cmd for main packages as the second-to-last path component. And of course the few hard-coded directory and file names.

It's fine to put out a recommended layout, even one that may be imperfect. @bitfield is right to point out that there are plenty of blog posts with questionable suggestions out there. I'm not sure this can be put into that bucket, as blogs published under the names of individuals or organizations will be generally understood as reflecting only the author's perspective. Who would have the audacity to call their organization "golang-standards" if they have neither the authority to set standards nor consensus from the community that what they are promoting is a best practice? It's not uncommon for new gophers to make the fairly sensible assumption that the answer to this question is "nobody", and rather than pursuing further perspectives, they take this repo to be the final answer that its name implies it to be. (No, a disclaimer in the readme does not adequately mitigate that. How often do you read a readme from top to bottom?) It's a fine project, but an irresponsible name, and it would be tremendously helpful to the community if the name were changed to convey the humility which I'm sure the author intended to bring to this project.

Lol, i have a great idea:

If this layout is not official (yet, maybe), instead of renaming, why not to work on it? 🤔 I mean, go still doesn't have best practices of directory layout (unlike rust, dart, etc. note that they have recommendations, not restrictions), why not create a recommended only standard of layout (only as recommendation, like effective go, gofmt, etc.) that suits most of gophers tasks including language core developers?

People say here right thing (like @flowchartsman): go is so simple that you can write the code as you feel it and it looks good. Honestly, this repository is not needed at all in current status 😂 but despite this, some kind of agreement is still needed where long-time gophers where to find specific logic with unknown project to them (for example, Kubernetes and Prometheus are rather complex sources, but at the same time coode looks slightly similar, I do not want to be tied to this particular projects or this repository at all, so with such recommendations it is easier to understand where someone else's code stores and how it relates to each other.)

The question is not so much in standardizing the layout, but in explaining to newbies how long-time gophers are storing their code by functionality

guys how dislike it: instead of being toxic, explain your opinion. I look at your thumbs down and not fully understanding whats wrong with my comment, why not to explain your position?

It seems that there is some confusion around suggestions about project layouts, and the resulting rigid package naming that would be involved. If you take this repo without the golang-standards name, and remove the pkg idea, a lot of these folder suggestions aren't inherently bad. Most of them are related to supporting pieces for a project that aren't specifically go code, but which may live with it. If something official were to surface, it can certainly be presented in a way that avoids most of this confusion. The key thing to it is to start simple, and know that the layout is free form. If a project does start to grow, it's not bad to have some suggestions further on in the documentation that can offer guidance if wanted.

There are two problems with this GitHub repo:

1. it claims to host Go standards and does not, in the sense that these are in no way official standards
2. the project-layout standard it puts forth is far too complex and not a standard

Regarding "why not tell us the standard Go project layout and we'll update the doc?", that only addresses point 2. If there really were standards, they would be in the main Go project doc tree. The standard for project layout would also be a lot shorter. I appreciate your trying to provide a useful resource, but calling it 'golang-standards' is claiming more than it is.

But for the record, the minimal standard layout for an importable Go repo is really:

• Put a LICENSE file in your root
• Put a go.mod file in your root
• Put Go code in your repo, in the root or organized into a directory tree as you see fit

That's it. That's the "standard".

In particular:

It is not required to put commands in cmd/. It is not required to put packages in pkg/. It is not required to put web stuff in web/. It is not required to put APIs in api/. It is not required to put web stuff in web/. It is not required to put configurations in configs/. It is not required to put systemd scripts in init/. It is not required to put shell scripts in scripts/. It is not required to put Docker files in build/package/. It is not required to put CI configs in build/ci/. It is not required to put deployment configs in deployments/. It is not required to put test support in test/. It is not required to put documentation in docs/. It is not required to put supporting tools in tools/. It is not required to put examples in examples/. It is not required to put third_party code in third_party/. It is not required to put git hooks in githooks/ It is not required to put static assets in assets/. It is not required to put website data in website/.

The importable golang.org/x repos break every one of these "rules".