Regenerating pod-styles.scss each time slows rebuilds

[vlascik]

I have a bunch of apps and an addon used to share components etc. between them, all use e-c-css. I can see that on each rebuild of the app pod-styles.scss is being regenerated for some reason, even if no css was edited and components weren't changed, forcing the start of sass compiler and adding time to rebuilds. I would really love to get the ember rebuilds under 1 second one day, currently just this rebuild adds over 30% of the total time to it.

Could these rebuilds be avoided if there was no change to css/pod styles?

[webark]

@vlascik first, what version are you on? Second, this will only run when a style file is added, removed, or changed. (At least this was the case last time I checked about a year ago. I don't think anything has changed since then to alter that behavior, but I could be wrong.) Any other "rebuild" issue that you may be having would be something in the preprocessor that you are using.

What made you feel that this was the culprit? And not just the preprocessor that you are using? (not saying that this didn't add to the time, but wanted it for debugging purposes).

[vlascik]

I'm on 0.3.4, but I don't think there was any change on this in 0.3.5 the reason is I can see this in logs:

  fs-tree-diff: [SassCompiler#12527 -> BroccoliMergeTrees: Merge (ember-component-css merge namespacedStyles with style manafest)#12528 -> derivePatches#12537] invalidation reason: 
  fs-tree-diff: before Entry { mode: 33206, relativePath: 'pod-styles.scss', basePath: 'P:\\Project\\tmp\\broccoli_merge_trees-input_base_path-pSok6l1n.tmp\\1', size: 655, mtime: 2017-10-31T04:25:38.865Z, linkDir: false }
  fs-tree-diff:  entryB Entry { mode: 33206, relativePath: 'pod-styles.scss', basePath: 'P:\\Project\\tmp\\broccoli_merge_trees-input_base_path-pSok6l1n.tmp\\1', size: 655, mtime: 2017-10-31T04:26:55.755Z, linkDir: false } +7ms
  broccoli-caching-writer:SassCompiler walking undefined +4ms
  broccoli-caching-writer:SassCompiler  cache eviction due to: 
  broccoli-caching-writer:SassCompiler      - {file, P:/Project/tmp/sass_compiler-input_base_path-8D6aJELn.tmp/0/pod-styles.scss, pod-styles.scss, 33206, 655, 1509424015755} 
  broccoli-caching-writer:SassCompiler      - {file, P:/Project/tmp/sass_compiler-input_base_path-8D6aJELn.tmp/0/pod-styles.scss, pod-styles.scss, 33206, 655, 1509423938865} +11ms

[webark]

hmm. I can dig into that a little more than and get back to you?

[vlascik]

sure, thanks.

[webark]

@vlascik ok. I see the issue. I'll see if I can get the fix in today, if not this week definitely.

[webark]

Thanks for pointing it out!

[vlascik]

Perfect, thank you.

[webark]

@vlascik I think I fixed at least half of this issue. Care to try out v0.4.0 and see if you notice a difference?

[vlascik]

Yep I tried it a while back, .scss was not recreated, I think this was fixed, but .js still was iirc. Not sure how much it helped though.

It seems the main problem is a walkSync of complete app tree, which if you use bower or app.import from an addon with huge node_modules (which in 2.16 adds the whole addon folder to the app tree), takes forever (45000 files and 17 seconds in my case). I don't seem to be able to find out if it's coming from e-c-sass or this addon, it seems to start in e-c-css, but debugging broccoli is not exactly easy :/.

Could you perhaps try to try to add app.import('node_modules/some-addon/....js'), on an addon with say 100mb of random packages in its node_modules and watch the rebuild? You might be able to see the problem with walkSync too.

[webark]

ya. the .js package is still being rebuilt. Was going to look at that side a little later, but that should cause too much overhead. you don’t happen to have a sample repo with all your node modules by chance do you?

[webark]

@vlascik are you still having an issue with the js rebuild side of this?

[vlascik]

I'm currently too busy with back-end work, but I will update with what I find out once I'm back to ember. However, after https://github.com/broccolijs/broccoli-plugin/pull/23 lands (if ever), regenerating pod-names.js will definitely trigger an unnecessary rebuild, so it might be worth fixing it anyway.