Add cascading alternate search paths to fix preprocessor workflow

[robwierzbowski]

This was originally posted in the Yeoman issue tracker, and has been requested in a couple other issues indirectly; I'm reposting here so we have a centralized issue with references and use cases.

Problem

Usemin looks in one directory for every item in a usemin block. When using a CSS or javascript preprocessor, your processed CSS and JS files will be in a different directory than vanilla CSS and JS. This makes it difficult to concatenate preprocessed CSS and JS with vendor or component files without an extra copy step to put them in the same directory.

<!-- build:css /css/main.css -->
<!-- This is a preprocessed css file, output to .tmp directory -->
<link rel="stylesheet" href="/css/main.css">

<!-- These are vanilla css files that live in the app directory -->
<link rel="stylesheet" href="/css/syntax.css">
<link rel="stylesheet" href="/bower_components/some_component/main.css">
<!-- endbuild -->

We can add an alternate path for the block, for example build:css(.tmp), but this doesn't solve the problem of having files in two entirely different root directories.

Solution

The server task solves this by looking for a file in a cascading list of directories, and using the first one it finds. We can duplicate this pattern for the build step by adding a cascading list of alternate directories in the usemin block.

<!-- build:css(app,.tmp) /css/main.css -->
<!-- This is a preprocessed css file, output to .tmp directory.
      Usemin looks in .tmp first, finds this file, and adds it to 
      the concat array. -->
<link rel="stylesheet" href="/css/main.css">

<!-- These are vanilla css files that live in the app directory.
      Usemin looks in the .tmp directory but doesn't find the the files.
      It then looks in the app directory, finds the files and adds them
      to the concat array. -->
<link rel="stylesheet" href="/css/syntax.css">
<link rel="stylesheet" href="/bower_components/some_component/main.css">
<!-- endbuild -->

Notes

This was first proposed in https://github.com/yeoman/yeoman/issues/959. It solves https://github.com/yeoman/generator-webapp/issues/12#issuecomment-15513089 and https://github.com/yeoman/generator-webapp/issues/31. @aligo did some similar work in https://github.com/yeoman/grunt-usemin/pull/116. Their patch uses an attribute in the link or script tag; I think the original proposed syntax is cleaner, but maybe they have some insights or would be interested in helping implement this syntax.

[robwierzbowski]

A good workaround from @aligo:

I am using on my code now. So when the search paths {app,.tmp} be passed to grunt-contrib-concat, each file path will become like {app,.tmp}/scripts/xxx.js, then grunt-contrib-concat will search the file in app and .tmp both, and merge all them into scripts.js.

This could be a permanent solution. Because it adds both file paths, not a cascade, naming collisions would cause issues, but naming collisions should be avoided anyways.

[eddiemonge]

the workaround is fine, unless you app/.tmp folder locations are configurable. then the config is thrown out the window because it wont find them if they are changed. We need to have an option in the config for cascading alternate search paths

[robwierzbowski]

True, a configuration option is the real fix. @sleeper, is there something like this in 2.0?

[sleeper]

Not yet, but I would say "almost" (i.e. most of the code already exists).

[mrtorrent]

The workaround from @robwierzbowski's earlier comment no longer seems to work in 2.0.0. Is there an alternative approach that should be used?

[eddiemonge]

i was hoping this would be fixed by now :(

[robwierzbowski]

Me too, so I should probably DIY for OSS FTW. I'll try and roll a PR this weekend.

[aligo]

: D thx

[szimek]

I'm not sure if that's a good place, but I'm having an issue with alternate search paths. I've got build:js({.tmp,app}) in index.html and I'm preprocessing one of my js files from app to .tmp. The problem is that concat task adds this file twice - once from .tmp and once from app (this one is not preprocessed and breaks the whole app). Is there any way to fix it without having to move this file to a separate build block?

I solved it for now by changing build blocks to look only in .tmp folder and I'm copying all files to .tmp first, then preprocess files in app to .tmp - that overwrites some of them. This seems to work, but it looks like the alternate search path feature is broken.

[arnars]

@szimek Did you solve it. I'm having the same issue right now. Outputting all my css twice.

[bgerstle]

@szimek @arnars same here. e.g. I'm doing some template processing of my JS, then running it through the usemin flow. using the ({.tmp,app}) workaround I end up with the same file twice: the processed template and the preprocessed template.

[robwierzbowski]

The simple but hackey solution is to give your preprocessor files an alternate file name (e.g., .jade, .hbs)

On Tuesday, April 15, 2014, Brian Gerstle [email protected] wrote:

@szimek https://github.com/szimek @arnars https://github.com/arnarssame here. e.g. I'm doing some template processing of my JS, then running it through the usemin flow. using the ({.tmp,app}) workaround I end up with the same file twice: the processed template and the preprocessed template.

— Reply to this email directly or view it on GitHubhttps://github.com/yeoman/grunt-usemin/issues/133#issuecomment-40470568 .

Rob Wierzbowski @robwierzbowski http://twitter.com/#!/robwierzbowski http://github.com/robwierzbowski http://robwierzbowski.com

[lhwparis]

any news here? need that too

[stevemao]

This would be a really cool feature. It might be be even better to have an optional explicit search paths for scripts/styles? eg. I'm using grunt-modernizr and it generates a customised modernizr for dist.

<!-- build:js(.) scripts/scripts.js -->
<!-- bower:js -->
<script src="bower_components/modernizr/modernizr.js"></script>
...
<!-- endbower -->
...
<!-- endbuild -->

I would like to write something like

<!-- build:js(.) scripts/scripts.js -->
<!-- bower:js -->
<!-- src:(.tmp/modernizr-custom.js) -->
<script src="bower_components/modernizr/modernizr.js"></script>
<!-- endsrc -->
...
<!-- endbower -->
...
<!-- endbuild -->

So that the modernizr.js path would instead be .tmp/modernizr-custom.js

Another thought is if we put ignore comments around script(s)

<!-- build:js(.) scripts/scripts.js -->
<!-- bower:js -->
<!-- ignore -->
<script src="bower_components/modernizr/modernizr.js"></script>
<!-- endignore -->
...
<!-- endbower -->
...
<!-- endbuild -->

So usemin would completely ignore the script(s) and the configs in concat, uglify and css, etc somehow will append to usemin generated config.

[craigmdennis]

I'm also now finding the the alternate search workaround fails. It only seems to search the first directory (even though the config shows it should search both).

<!-- build:js({.tmp, app}) scripts/main.js --> will concat the files from .tmp but not app <!-- build:js({app,.tmp}) scripts/main.js --> will concat the files from app but not tmp

[stephanebachelier]

Adding the feature tag. Others things in the pipe before.

[lhwparis]

+1 i need this too

[stephanebachelier]

We need the HTML parser #244 before going on this. About the ignore comment tag it is linked to another issue where some users have found that HTML comments around a script tag is ignored (#503)