generator-angular
generator-angular copied to clipboard
yeoman
Proposals to Make Components More Reusable
I apologize in advance for this issue's massive length.
I would like to start a discussion on some ways I think the generators could be changed to make the resulting components more easily reusable. I think when developing, we should have the potential for as much of our code as possible to be "drag and drop reusable". AngularJS already sets us well on our way by promoting separation of concerns in its internal architecture, so why shouldn't our development tools do the same?
The reference structure I am using is my own ngBoilerplate.
Each section builds on the previous.
Update [07 Mar, 0840 PST]: I use the term "component" here a little more loosely than Bower. I just mean it to refer to functionality that is somewhat self-contained. Unless otherwise noted, it refers to both bundled code and internal app features.
Organize by Feature
Instead of bundling code by the layer to which it belongs (controllers, services, filters, directives, etc.), I would like to see code bundled by the feature to which it belongs. There can be many manifestations of this, but here are two examples:
(a) Routes
Instead of something like this for a /home route:
|-- src/
| |-- scripts/
| | |-- controllers/
| | | |-- home.js
| |-- test/
| | |-- spec/
| | | |-- controllers/
| | | | |-- home.js
| |-- views/
| | |-- home.html
I'd prefer to see it like this:
|-- src/
| |-- app/
| | |-- home/
| | | |-- home.js
| | | |-- home.spec.js
| | | |-- home.tpl.html
This is a much simpler directory structure, but makes the home module very portable. It is also self-contained: it can be totally refactored without impacting the rest of the application.
(b) Multi-Component Features:
If we have complex component, it is likely composed of multiple smaller components. For example, an editing component might have a persistence service, a representation service for translating markup to HTML, and a directive or two for display. With the existing structure, each component would be created independently and be mixed throughout the various layers; without a comprehensive, holistic understanding of the application, it is difficult to see how these components are (or should be) inter-related. It demands manually surfing through the code and/or doing searches for component names.
A cleaner directory structure would look something like this:
|-- src/
| |-- components/
| | |-- editor/
| | | |-- editing.js
| | | |-- editor.js
| | | |-- editor.spec.js
| | | |-- editingStorage.js
| | | |-- editingStorage.spec.js
| | | |-- editingRender.js
| | | |-- editingRender.spec.js
Now it's pretty clear we're talking about one component, albeit a complex one that spans multiple layers. But each sub-component of the editor is still standalone, if need be - that's just good programming.
Isolate the Modules
A logical extension of this reorganization is to so-define our modules. In this pattern, each directory roughly corresponds to a module. Instead of this:
angular.module( 'myApp' )
.controller( 'HomeCtrl', function ($scope) {
// ...
});
We would have this:
angular.module( 'home', [] )
.controller('HomeCtrl', function ($scope) {
// ...
});
Similarly for the editing component:
angular.module( 'editor', [
'editing.editor',
'editing.editingStorage',
'editing.editingRender'
])
Where those modules are defined in their respective files. And our main app.js file simply requires the top-level modules:
angular.module( 'myApp', [
'home',
'editing'
]);
Adjacent Tests
Everyone coming from server-side development (including myself) is familiar with the separate test directory, but I've always found it vexing. Placing test files directly adjacent to the code they test makes them easier to locate. More important, however, is reusability; if our tests are in a separate directory, we now have two things we have to copy between projects in order to reuse a component.
This is where it's important to separate two kinds of projects: libraries and apps. When developing libraries, we design them to be self-contained and self-sufficient, so we have no need to take tests with us to reuse in another project - that should have all been part of a build. But when developing apps, where some components may depend more subtly on other components (or at least on assumptions about our app architecture), it makes sense to take the tests with us and ensure they still pass in the new environment. Also see the 'Internal "Components"' section below.
It's okay to keep the tests side-by-side because our build tools are sophisticated enough to be able to tell the difference. Grunt 0.4, for example, now includes negative file selectors, so our build can exclude files with patterns like src/**/*.spec.js or src/**/*Spec.js from compilation and minification.
Adjacent Templates
The same concept also applies to views, partials, and templates. I also prefer they be suffixed with .tpl.html or something similar to indicate they're fragments.
Modularized Routing
Using the above example, yo angular:route home would generate this:
|-- src/
| |-- app/
| | |-- home/
| | | |-- home.js
| | | |-- home.spec.js
| | | |-- home.tpl.html
But instead of defining the route in app.js, we would let the home module set up its own routing:
angular.module( 'home', [] )
.config( function ( $routeProvider ) {
$routeProvider
.when( '/home', {
templateUrl: 'home/home.tpl.html',
controller: 'HomeCtrl'
});
})
.controller( 'HomeCtrl', function ( $scope ) {
// ...
})
Nothing is required in the app.js in terms of routing, unless the user should choose to define a default redirect, such as to /home.
Feature Nesting
The existing directory structure is very flat. For small projects, this is perfectly fine, but for non-trivial projects it can become a file management nightmare. If we organize our code by the feature or component they implement and use adjacent templates and tests, it also makes sense to be able to nest them.
Considering the route example:
|-- src/
| |-- app/
| | |-- products/
| | | |-- products.js
| | | |-- products.spec.js
| | | |-- products.tpl.html
| | | |-- create/
| | | | |-- create.js
| | | | |-- create.tpl.html
| | | |-- ...
In this case, each directory should roughly correspond to a single "submodule". The products directory is a module called products, making create something like products.create. Using this pattern, the products module can require all requisite submodules:
angular.module( 'products', [
'products.list',
'products.view',
'products.create',
'products.search'
]);
Again, because the target is reusability, each app module is responsible for declaring its own dependencies, which will "bubble up" from products.create to products to myApp. Routing would work similarly; each submodule can define its own routing, in theory "namespacing" to its parent module. For example, products.create could define a route of /products/create.
This same "nested" pattern would also apply to complex components, though they would obviously not include routing. E.g.:
|-- src/
| |-- components/
| | |-- editor/
| | | |-- editor.js
| | | |-- plugins/
| | | | |-- syntax.js
| | | | |-- align.js
Internal "Components"
Lastly, I make a distinction between app code, the stuff that is somewhat unique to our problem domain, and components, the stuff that may come from a third party but that is more immediately reusable in unrelated projects.
With this concept in mind, we should be able to mix in the components directory the third-party libraries that come from Bower, the third-party libraries we download manually, and the reusable components that we are coding for this application specifically. e.g.:
|-- src/
| |-- components/
| | |-- angular-placeholders/ <downloaded>
| | |-- angular-ui-bootstrap/ <bower>
| | |-- editor/ <internal>
All combined, I think this would improve code reusability and readability.
This is exactly the way it should be! I'm also a fan of ngBoilerplate. Structuring an angular app by feature rather then by layer makes the app more flexible. Your proposal looks also pretty generic, +1 for that!
Wow - that's a fantastically presented set of suggestions. I'm not experienced enough with Angular to know if all of this is a 'good idea' but the logic sounds solid to me.
I have been using this encapsulated type of component based architecture for years in FLEX I agree on everything apart from integrating the routes into the component buddle, I think you might want to leave this part in your app code... But apart from that +100 @joshdmiller
I agree with @Bretto on not integrating the routes into the component bundles however everything else look pretty good.
@Bretto and @ryanzec - I definitely think only app code should be able to define routes and I didn't meant to imply otherwise. I used "component" somewhat loosely in my write-up to mean both the bundled, often third-party code and a defined feature of our application. I'll edit the post to clarify.
Not familiar at all with how Yeoman/Bower works, but I have a question: Does the deployment process from this allow you to limit what files get sent up to your production server? Thinking that sending your tests to a publicly accessible web server not the best idea.
@MikeMcElroy Yes, that would be easy to exclude regardless of whether they're in a separate folder or not.
Then very cool. Carry on. :-)
On Thu, Mar 7, 2013 at 10:34 AM, Pascal Hartig [email protected]:
@MikeMcElroy https://github.com/MikeMcElroy Yes, that would be easy to exclude regardless of whether they're in a separate folder or not.
— Reply to this email directly or view it on GitHubhttps://github.com/yeoman/generator-angular/issues/109#issuecomment-14577943 .
I'm working on a non-trivial angular app currently, and have been flapping about trying to come up with a structure that sat well with me (knowing that the default did not).
This is extremley well articulated and clear -- thanks @joshdmiller for taking the time and effort for writing this up.
+1 Makes perfect sense to me. It makes it easier to manage large projects.
I appreciate all of the positive comments - it's nice to know there are others who agree. :-)
That said, I already have a pretty healthy ego - surely there's someone out there who wants to call me an idiot.
I think the directory structure you laid out is good regardless of portability or not, organizationally it is good. Instead of calling you an idiot, which @rstuven already did, let me ask you a question as I have been thinking about this more. What do I gain from using multiple smaller modules?
For example, I have a set of components I have been building and if I switch to using small modules instead of one big module, I would go from 1 module to ~24 modules (probably more as I still have functionality to add). Now if I use the same directory structure you have laid out but still only use one module, what do I really gain from converting them multiple small modules? If you dont need some of the components I have, just don't include the js file for that component.
Now I am going to need all these modules for an application that I am building. I can see this application easily having 20+ main modules with this setup and a lot of those modules would have sub modules, probably ranging from most commonly from 2-5. Now we are talking about 50 - 100+ small modules in this application and most of the modules in the application itself are not really going to be portable as they are going to be built specifically for this application.
So are there any downsides to having a bunch of smaller modules (50 - 100+) when portability it not important?
@ryanzec - Nice. I see the benefits of multiple small modules as these:
- Reusability. I focused the bulk of my post on this, so there isn't much else to say, except this: we don't always know what we'll need to reuse.
- Ease of refactoring. If we don't follow the "one module per file" rule, refactoring becomes very tricky very fast.
- Comprehensibility. Small modules with a deep directory structure are often much easier for developers new to the team (or for you six months from now) to see where everything is located and understand the original intent behind some of the code. Monolithic files with lots of code or a single directory with a dozen files can be a trifle overwhelming.
I don't really see any downsides to having multiple modules; the build process will take care of concatenation and minification.
That said, this issue really isn't about forcing anyone to use this directory structure, but that the current generator doesn't even support the use of this directory structure. I'd like this tool to be flexible above all else, but also encouraging of a more "correct" architecture. AngularJS is just so different than other client-side libraries that most new developers don't know what an app should look like. I provide a lot of community support on the mailing list, on SO, and through a few projects, and I often have to wade through insanity - seeing a little less crap out there would lower my blood pressure. :-)
A useful next discussion would be how the CLI tools might support this structure; for routes, it's pretty obvious, but for others the debate may get a little more contentious...and interesting.
A useful next discussion would be how the CLI tools might support this structure
+1. This is currently where I'm stalled on this idea. Any bright ideas on what you'd like the CLI to look like for this?
Well, shoot-darn - now you're taxing my brain. I can think of lots of ways to do it, but there is difficulty in finding something flexible, so we don't force anything on anyone, and that doesn't require a bunch of parameters, defeating the purpose of the scaffolding. I should also mention that I've never been a big fan of generators and so don't use them really often; those that do are more likely to provide a better implementation.
But I'll kick things off ...
First, I've encountered a problem with my above recommendations; technically, it's a problem with Bower.
Bower is not nearly robust enough for use with AngularJS projects. With jQuery, it makes sense to have the plugin's repository be the Bower component's source; all it usually has is a JavaScript file, along perhaps with a demo HTML file and some styles. But when we move into something more complicated, using the source repository totally breaks my build. :-(
Take Bootstrap. I see two use cases: (1) we just include the CSS; (2) we leverage the LESS/SASS files. For the former, the automated build works great; for the latter, we need to throw those files into some vendor directory that is not processed by the build because our main.less file in our app will @import the necessary files.
But installing Bootstrap with Bower doesn't well support either. We just get a massive directory of all of the files - including build scripts and jQuery plugins. I can't automate any building with that. But that's not the issue - the issue is that Bower makes no distinction between kinds of packages. It offers no way to specify which parts of the package we need. It offers no way to build on the fly.
So if we use Bower for our components, we cannot place our own components alongside them. Additionally, by using Bower, we have two choices when it comes to the build:
- just include all components in the build, included into
index.html(this is the current approach, right?) - manually adding logic for every component to our Gruntfile so we can get that one payload at the end a lot of us want.
These options totally suck. In my opinion, a great package manager for AngularJS projects would standardize the packages and offer them built or unbuilt, uglified or not, with the ability to select sub-components, so we can completely automate our build regardless of what components we choose to include.
The Takeaway: But Bower is what we have for now. So I have to change my above proposal. src/components/ must be thought of as "vendor" components which we have to add to our build manually - in many cases (perhaps even by default) simply copying the files to our distribution would suffice. But I submit that as more AngularJS components make their way into Bower, this will be less and less palatable. Se here's the redefinition: "mutli-component features" discussed in my initial issue post are now inside src/app/; src/components/ is used exclusively for vendor content (and should perhaps be renamed to src/vendor).
Okay, now that we've re-defined things to fix that issue with my original recommendations, we can get on with the yo commands.
To avoid really complex generator commands, I think we have to buy into a few basic concepts:
- One module per file
- One folder per route (though nesting is developer's choice)
- Tests, views, and styles sit alongside their component
app
$ yo angular:app
Combining everything from the previous post, this yields:
|-- dist/
|-- Gruntfile.js
|-- package.json
|-- src/
| |-- app/
| | |-- app.js
| | |-- app.spec.js
| |-- assets/
| |-- index.html
| |-- styles/
| |-- vendor/
|-- testacular.conf.js
route
$ yo angular:route <route> [module]
To provide maximum flexibility, the user can provide a module name. In the absence of a module name, perhaps just src/app/<route>/ could be the default. Or maybe a modularized version of the route.
$ yo angular:route "/products"
|-- src/
| |-- app/
| | |-- products/
| | | |-- products.js
| | | |-- products.spec.js
| | | |-- products.tpl.html
And, of course, the products module should be added to the dependencies array of the app module. Another example:
$ yo angular:route "/products/show"
|-- src/
| |-- app/
| | |-- products/
| | | |-- show/
| | | | |-- show.js
| | | | |-- show.spec.js
| | | | |-- show.tpl.html
Obviously, this would add the products.show module as a dependency of products.
I would also think that running the second without the first (which may make more sense in some cases), would yield this:
|-- src/
| |-- app/
| | |-- products/
| | | |-- products.js
| | | |-- products.spec.js
| | | |-- show/
| | | | |-- show.js
| | | | |-- show.spec.js
| | | | |-- show.tpl.html
Note there is no template on the products module. There would also be no routes defined on products; in this case it is simply a bundler for its subroutes (again, overwritable with the module CLI option). There may also be some feature shared across products that would end up compiled here, but at this point, the generator doesn't know and doesn't care - the important feature now is that it is packaged with growth in mind.
controller, directive, filter, service, & view
I see these as fundamentally the same (though the templates vary, obviously).
$ yo angular:controller <name>
$ yo angular:directive <name>
$ yo angular:filter <name>
$ yo angular:service <name>
$ yo angular:view <name>
My thinking here is that <name> could optionally contain the module (defaulting to nothing):
$ yo angular:controller DoesSomethingCtrl
|-- src/
| |-- app/
| | |-- doesSomething/
| | | |-- doesSomething.js
| | | |-- doesSomething.spec.js
Or:
$ yo angular:controller products.preview.PreviewCtrl
|-- src/
| |-- app/
| | |-- products/
| | | |-- preview/
| | | | |-- preview.js
| | | | |-- preview.spec.js
The same logic from the route example would be used to add the new module to the dependencies array of its parent module. It should also add non-existing parent modules; so if products didn't exist, it would be created along with products.preview.
There are a few rough spots in the above and I can envision some potential objections. Also, the generator logic to support the above may well be too onerous. This is more to start the discussion.
So, what are your thoughts?
I like the direction this is going.
On routing, would it be possible to let the app decide the prefix for a component? I'm struggling to think of a really reusable component that would need this though. Maybe a user manager 'page' - always routing at /users/, when you want an /admin/ prefix to handle permissions - dunno.
On the subject of angular components and bower being too simplistic: I noticed that the angular team simply use a separate repo for their compiled files. (https://github.com/angular/angular.js and https://github.com/angular/bower-angular). This makes a lot more sense to me. In the bootstrap case you mention, I would expect to either 'bower install' the CSS files or submodule the source. If you are including something to be part of your build process, it's not really a simple web component. Well, that's just a rule of thumb I use. Components aren't very mature.
I also found that most yeoman generators (or any really) are aimed at scaffolding a web app and there is a missing generator to scaffold a component. I was going to try my hand at writing a generator, but then I'd be unable to take advantage of angular:view etc.
Finally, I think it starts to get a bit Java-esque when creating a controller defaults to a directory containing a file (another reason not to have co-located test files). Might it be better to default something like angular:controller back to the existing behaviour?
@stackfull You make good points; I'll respond inline here to avoid confusion.
On routing, would it be possible to let the app decide the prefix for a component?
I may not be following. Once the generator creates the route, the URL string is hard-coded into the file. How would we change this?
On the subject of angular components and bower being too simplistic: I noticed that the angular team simply use a separate repo for their compiled files.
The separate repo is also what we chose for the AngularUI Bootstrap project. But we're really just working around the tool here; when we have to do a workaround, that usually means there's something wrong with the tool - or at least that it's not "optimal".
If you are including something to be part of your build process, it's not really a simple web component.
A component should be able to be concatenated and uglified as part of our build; if we want a single payload, this is really a hard requirement. With Bower, we don't get that option unless we manually add these files to our Gruntfile - that's a shame. But as I said, we don't have any other options at this point.
I also found that most yeoman generators (or any really) are aimed at scaffolding a web app and there is a missing generator to scaffold a component.
Interesting. How do you see this generator looking?
Finally, I think it starts to get a bit Java-esque when creating a controller defaults to a directory containing a file (another reason not to have co-located test files).
Why is "Java-esque" wrong? I'm certainly not saying we should wire our app together with a half-dozen XML files. :-) What's the connection to the co-located test files?
Might it be better to default something like angular:controller back to the existing behaviour?
The existing behavior is incompatible with my proposals. It makes no sense to have a "controllers" directory (and/or module) that doesn't contain all of the controllers. But I'm not sure when this would be an issue; how often are you going to create a controller that is neither part of a route nor has any related code in its module?
+1
For better reusability I think the default router would not be the right choice. Cause it is working on static routes. Maybe ui-router would be a much better choice, cause you can define states instead of static routes. So when you say you want to create a product you go in the state 'product.create' and link it to a route in your app, where you bind all your "components" together. But when saying we have reusable components I think they should all be placed in "components".
Everything that is placed under the app directory should not be intended to be reusable. They should be there for binding all the reusable components together.
Just to ask the question, this refers only the client (browser) side of an app, yes? The server side (node, python, ruby, ColdFusion, etc), would be in a separate repository, or completely outside the scope of the project, with, say, a Twitter client?
@joshdmiller I like it! One small tweak I would make would be to the module names. Make the module names match the route names: / instead of . as a splitter. To use your example:
$ yo angular:route "products/new"
This would would generate the folder structure you suggested, but create the "products/show" module instead of "products.show". Then the controller generator would be easier to guess:
$ yo angular:controller "products/preview" PreviewCtrl
This would make things more consistent and easier to generate.
@joshdmiller I've overestimated what angular routing can do. (thanks for the pointer @mlegenhausen.)
The "java-esque" comment just meant verbosity. Java project tend to have deeply nested directory structures and 2 or 3 files for each logical entity. That's not me hating on Java, that's just how the languages shape their projects.
As for structuring a project for a component, I'm hitting the odd roadblock in angular atm. Not being able to refer to templates relative to module code is making my grandiose plans look a bit far-fetched. My habit is to have src and demo directories at the top level for the component code and a demo app, but it's getting a little ugly to serve up. So I will probably change to fit with whatever generator-angular does - maybe:
|-- dist/ |-- Gruntfile.js |-- package.json |-- component.json |-- src/ | |-- demo/ | | |-- app.js | | |-- app.spec.js | |-- [component-name]/ | | |-- module.js | |-- assets/ | |-- index.html | |-- styles/ | |-- vendor/ |-- testacular.conf.js
Okay, I stayed away for a few days and now everyone has an opinion! Here we go...
@mlegenhausen I think uiRouter is awesome and it should be a component one can add (and possibly an additional generator to install), but for the broadest possible applicability, generator-angular should probably use the standard router, even though it is less flexible. For app versus component, I'm not sure I agree. We don't always know what we will need to reuse. There are two types of reusability: (a) creating components for distribution; and (b) creating parts of our app we may want to leverage later. Designing with reusability in mind costs us nothing but nevertheless prepares us for any eventuality.
@davemerrill The generator is client-side only, so I suppose the structure would depend on how you did the backend. My perspective is that the back- and front-ends should be developed completely independently, but there are many ways to approach this that would depend on your workflow.
@ajoslin Cool. I don't care too much about the module separator; if we like / instead of ., that's cool with me. In some ways, it may make more sense but also may be less familiar. The only strong opinion I have would be that it needs to be consistent across all generator commands.
@stackfull I'm not a fan of Java either. For relative templates, I think this would be a good feature for AngularJS to support, but at the moment it can be solved through standardization. If you use a directory structure like that for which I am advocating, it will actually "just work". A template is specified relative to its module. So in the same way that one must add the module to the dependencies array, so does the template get specified. Because we've standardized how the structure should look, it will be portable across projects (as well as self-contained). But hopefully AngularJS will support relative paths eventually to allow a little more flexibility.
But I don't see why your demo can't exist outside of the source. You don't need demo code "compiled", so can't it exist outside src/ and just reference the components in dist/? Or if that was unfeasible, can't the build simply push your demo files as if they were vendor components into dist/ and you could run it naturally?
demo can exist outside the source, but unless you code your own server task in grunt, the default "serve everything from this directory" breaks the template paths. It's like you said right back at the start - working with the tools you have.
As I said, if you use a "serve everything from this directory" strategy (which I don't recommend - I prefer compiled code, which can be opened on file://) then you can still add your demo to your build process from a separate directory to get it inside dist/.
If you use slashes as the separators this would "just work" with a couple of pretty minor changes to the code. The relevant controller generating code is currently this:
this.appTemplate('controller', 'scripts/controllers/' + this.name);
this.testTemplate('spec/controller', 'controllers/' + this.name);
but changing it to (untested pseudo code):
this.name = this.name.indexOf('/') < 0 ? 'controllers/' + this.name : this.name;
this.appTemplate('controller', 'scripts/' + this.name);
this.testTemplate('spec/controller', '/' + this.name);
would allow you to do yo angular:controller products/AddProduct, etc. since the supplied name is simply path.joined. Note this would still put the test in a separate directory, but I prefer it that way. I do like to have the views and everything else grouped by feature rather than layer, but keep tests in a separate directory tree that matches my source tree. It would, however, be trivial to add a config option to control whether or not tests are placed alongside the files they test.
You guys are great.
I'm starting fresh with Angular and JavaScript (coming from Java) and struggled a bit with the current layout of yeoman's angular generator. I like the idea to put everything related to a feature together. I found angular-sprout (https://github.com/thedigitalself/angular-sprout) and now I found this issue:)
What do you think about adding the css and maybe even images as well? Furthermore I like the idea of suffixing the files:
my-feature/ --> my-feature module
my-feature-controller.js --> my-feature control controller
my-feature.html --> my-feature partial/view
my-feature-service.js --> my-feature control service
my-feature-directive.js --> my-feature item control directive
my-feature-template.html --> my-feature item directive template
my-feature.css --> my-feature style sheets
Thanks, Leif
