additionalStreams not working

[forty8bits]

Given the below code, which I've simplified to make more readable:

gulp.task('build', function() {
  var templateStream = gulp.src('src/**/!(index).html')
    .pipe(processTemplates()) // lazypipe that just calls gulp-angular-templatecache.

  return gulp.src('src/index.html')
    .pipe(addTemplateToIndex()) // Adds <script> reference to 'app/app.templates.js'.
    .pipe($.useref({
      additionalStreams: [templateStream]
    }))
    .pipe($.if('app/**/*.js', processOwnJs()))
    .pipe($.if('*.js', processAllJs()))
    .pipe($.if('*.html', processAllHtml()))
    .pipe(gulp.dest('dist'));
});

I'm getting a File not found with singular glob error, specifying the full absolute path of the 'app/app.templates.js' file.

I can't seem to figure out what exactly is going wrong; through inserting a few calls to gulp-debug I can see that the app.templates.js file appears to be getting passed along with the expected cwd, base and path properties.

[forty8bits]

Here's an example run output with the full error and calls to gulp-debug in place, one at the end of the 'templateStream' pipeline and another right after the call to gulp-useref:

[22:06:07] Starting 'build-index'...
[22:06:07] gulp-debug: 
cwd:   ~/webapp
base:  ~/webapp/src/
path:  ~/webapp/src/app/app.templates.js

[22:06:07] gulp-debug: 1 item
[22:06:07] gulp-debug: 
cwd:   ~/webapp
base:  ~/webapp/src/
path:  ~/webapp/src/index.html

events.js:154
      throw er; // Unhandled 'error' event
      ^

Error: Error: File not found with singular glob: /home/forty8bits/webapp/src/app/app.templates.js
    at DestroyableTransform.<anonymous> (/home/forty8bits/webapp/node_modules/gulp-useref/lib/streamManager.js:90:36)
    at emitOne (events.js:95:20)
    at DestroyableTransform.emit (events.js:182:7)
    at emitOne (events.js:95:20)
    at Through2.emit (events.js:182:7)
    at OrderedStreams.<anonymous> (/home/forty8bits/webapp/node_modules/gulp-useref/node_modules/glob-stream/index.js:138:20)
    at emitOne (events.js:90:13)
    at OrderedStreams.emit (events.js:182:7)
    at emitOne (events.js:90:13)
    at DestroyableTransform.emit (events.js:182:7)

[forty8bits]

Possibly related; it's also worth noting that the current unit test for this functionality seems to fail intermittently.

  1) useref() should support external streams:

      Uncaught expected '/home/forty8bits/gulp-useref/test/fixtures/scripts/renamedyet.js' to be '/home/forty8bits/gulp-useref/test/fixtures/scripts/renamedthat.js'
      + expected - actual

      -/home/forty8bits/gulp-useref/test/fixtures/scripts/renamedyet.js
      +/home/forty8bits/gulp-useref/test/fixtures/scripts/renamedthat.js

  AssertionError: expected 'test/fixtures/scripts/renamedyet.js' to be '/home/forty8bits/gulp-useref/test/fixtures/scripts/renamedthat.js'
      at Assertion.fail (node_modules/should/lib/assertion.js:92:17)
      at Assertion.Object.defineProperty.value (node_modules/should/lib/assertion.js:164:19)
      at DestroyableTransform._transform (test/test.js:789:61)
      at DestroyableTransform.Transform._read (node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:184:10)
      at DestroyableTransform.Transform._write (node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:172:12)
      at doWrite (node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:237:10)
      at writeOrBuffer (node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:227:5)
      at DestroyableTransform.Writable.write (node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:194:11)
      at write (node_modules/through2/node_modules/readable-stream/lib/_stream_readable.js:623:24)
      at flow (node_modules/through2/node_modules/readable-stream/lib/_stream_readable.js:632:7)
      at DestroyableTransform.pipeOnReadable (node_modules/through2/node_modules/readable-stream/lib/_stream_readable.js:664:5)
      at emitReadable_ (node_modules/through2/node_modules/readable-stream/lib/_stream_readable.js:448:10)
      at emitReadable (node_modules/through2/node_modules/readable-stream/lib/_stream_readable.js:444:5)
      at readableAddChunk (node_modules/through2/node_modules/readable-stream/lib/_stream_readable.js:187:9)
      at DestroyableTransform.Readable.push (node_modules/through2/node_modules/readable-stream/lib/_stream_readable.js:149:10)
      at DestroyableTransform.Transform.push (node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:145:32)
      at DestroyableTransform.<anonymous> (lib/reorderTheStream.js:34:18)
      at Array.forEach (native)
      at DestroyableTransform._flush (lib/reorderTheStream.js:33:23)
      at DestroyableTransform.<anonymous> (node_modules/through2/node_modules/readable-stream/lib/_stream_transform.js:135:12)
      at finishMaybe (node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:371:12)
      at endWritable (node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:378:3)
      at DestroyableTransform.Writable.end (node_modules/through2/node_modules/readable-stream/lib/_stream_writable.js:356:5)
      at DestroyableTransform.onend (node_modules/vinyl-fs/node_modules/readable-stream/lib/_stream_readable.js:495:10)
      at endReadableNT (node_modules/vinyl-fs/node_modules/readable-stream/lib/_stream_readable.js:865:12)

Running the tests a few times in a row should let you see this occur.

[jonkemp]

~~Could be related to #184.~~

[urrri]

Moved from #139 I tried the same with the new version of useref 3.0.8 and it doesn't work. When I add additionalStreams it immediately stops concatenate files, just copy them. But it replaces entries in index.html to target. In addition it tries to add templates.js (I see it via debug and comments in the file)at the beginning of the each section (I have 4) , never mind I add entry to the html or not.

[yuricamara]

Moved from #139

"When I add additionalStreams it immediately stops concatenate files, just copy them" Same problem here.

"version": "3.0.8"

[jonkemp]

Anybody having this issue want to try installing older versions of the plugin and see which is the most recent one that works?

[jonkemp]

Possibly related; it's also worth noting that the current unit test for this functionality seems to fail intermittently.

@forty8bits that issue was fixed. The test was dependent on the order of the files which is not being preserved.

[jonkemp]

There is a new version, 3.1.0. Does the new version have the same issues?

[jonkemp]

If no one with this issue wants to follow up on this and help test, I might as well close it.

[forty8bits]

@jonkemp: The File not found with singular glob error from my original comment is no longer present in version 3.1.0, however it still shows the issues others mentioned above; concatenation doesn't work when additionalStreams is used.

It may also be worth noting that the single file contained in the additional stream is showing up twice in gulp-debug directly after the gulp-useref step.

I'll try to dig into this further, although I may not get a chance until the end of next week.

[deltaidea]

I can also confirm that on 3.1.0 when using additionalStreams the assets aren't concatenated as if I pass noconcat: true. I'm going to look into this a bit later.

[deltaidea]

Everything starts working fine for me once I remove lines 13-16 in lib/addFilesFromExtStreams.js:

    // if we added additional files, reorder the stream
    if (additionalFiles.length > 0) {
        source = reorderTheStream.call(self, globs, src);
    }

The reorderTheStream function is supposed to return a different source stream that is used instead of the original src stream. However, it returns a stream that pipes no files. There's a bug somewhere in lib/reorderTheStream.js.

I'm not sure what's the purpose of reordering files here, but if I remove it, everything works fine for me. I can still reorder resulting files before concat using a transform stream:

return gulp.src('app/index.html')
  .pipe(wiredep())
  .pipe(useref({
    searchPath: 'app',
    noconcat: false,
    additionalStreams: [templateCacheStream()]
  },
    compileSass(),
    compileScripts(),
    reorderScripts()
  ))

• templateCacheStream is a stream with a single templates.js file from gulp-angular-templatecache module.
• compileSass and compileScripts work exactly as you would expect, using the gulp-if plugin to pass relevant files through gulp-sass, gulp-ng-annotate, etc.
• reorderScripts uses gulp-order to make sure templates.js is placed after bower_components and before my other JS files:

function reorderScripts () {
  return lazypipe()
    // gulp-order sorts files alphabetically. Bower files shouldn't be sorted because
    // they are placed by wiredep in order of what depends on what.
    // So we're using gulp-if here to run gulp-order only on files in cwd, and not
    // on files in ../bower_components
    .pipe(function () {
      return gulpif('**/*.js', order([
        'templates.js',
        'app.js',
        '**/*.js'
      ]))
    })
}

Is this a good enough solution? Would you accept a PR?

[deltaidea]

Update: there's also an issue that files added via additionalStreams are passed to concat each time addAssetsToStream is called. This means that if an additional stream adds a .js file, it will be concated to .js, .css and everything else.

This is easily fixed by passing current file type in processAssets to addFilesFromExtStreams.js and only adding files that match it.

If these two comments sound good, I'm gonna make a PR.

[jonkemp]

PR sounds good but I would also like a failing unit test before adding a fix.

[achuinard]

I'm having issues combining the use of this plugin along with gulp-angular-templatecache as well. Has this PR been considered for acceptance?

[jonkemp]

The PR was never made. I wasn't aware of this fork. My comments above are still valid.