karma-webpack icon indicating copy to clipboard operation
karma-webpack copied to clipboard

Slow rebuilds

Open jhollingworth opened this issue 10 years ago • 38 comments

I'm finding re-builds are taking much longer than I'd like. Right now it takes around 6-7s between me saving a file and the tests actually running. This is compared to ~500ms when WebPack is watching files. We saw some speed up with the suggested alternative usage but there's still an annoying delay.

Any thoughts on how to improve this? Or is this fundamental to Karma?

jhollingworth avatar Jun 16 '15 13:06 jhollingworth

I'm seeing the same issue. I have sourcemaps enabled so I know there is a performance hit there. I'm seeing 10-15s build times for about 40 files and of course their required node modules. I'd love to take advantage of a vendor bundle like I do in production but doesn't seem supported.

kevinold avatar Jul 16 '15 01:07 kevinold

I'm currently at 5s. It's really frustrating. How is everybody esle dealing with this?

vojto avatar Aug 19 '15 09:08 vojto

I gave up and moved to node with jsdom. so much nicer

jhollingworth avatar Aug 19 '15 09:08 jhollingworth

+1 The rebuild time is unacceptable for a typical TDD workflow We are experiencing this even in a medium sized project

leoselig avatar Sep 01 '15 09:09 leoselig

It shouldn't be that hard to implement. Just need to somehow hook into Webpack's incremental building (what happens when you run --watch). I'll give it a try when I find some time.

vojto avatar Sep 01 '15 10:09 vojto

Would definitely love to see this!

eknuth avatar Oct 05 '15 23:10 eknuth

Sorry intended to comment with my workaround. I am running weback with --watch as a separate process and not using karma to build :(

eknuth avatar Oct 05 '15 23:10 eknuth

check out https://www.npmjs.com/package/karma-webpack-with-fast-source-maps https://github.com/aaronjensen/karma-webpack

our builds run in a few hundred ms, and it only runs the tests it needs to

aaronjensen avatar Oct 05 '15 23:10 aaronjensen

@aaronjensen that is incredible, thank you so much!

eknuth avatar Oct 09 '15 22:10 eknuth

You bet.

aaronjensen avatar Oct 09 '15 22:10 aaronjensen

Nice! Any chance this becomes a PR soon? :)

leoselig avatar Oct 11 '15 17:10 leoselig

@leoselig maybe, @sokra is on vacation, we can discuss when he gets back. I'd love to have it in, but it removes some features (that don't appear to work any way-- #77)

aaronjensen avatar Oct 11 '15 17:10 aaronjensen

also, part of it is a pull already: #76

aaronjensen avatar Oct 11 '15 17:10 aaronjensen

You, my friend @aaronjensen, is the best thing since the invention of electricity!

I'm really looking forward to the day four fork is merged into webpack-karma mainline.

Great work!

tarjei avatar Oct 16 '15 06:10 tarjei

That's really nice @aaronjensen! Thanks. I ended up using your fork and creating a helper function to encapsulate the entry file boilerplate it depends on. Our entry file looks a bit like this:

const hotLoader = require('./tools/hotTestLoader')
hotLoader(require.context(/* etc /*))

necolas avatar Oct 27 '15 02:10 necolas

@aaronjensen FYI, your package stopped working from karma@>=0.13.11. (Your repo doesn't have issues enabled).

necolas avatar Nov 08 '15 23:11 necolas

@necolas It works for me w/ 0.13.15. What problem are you seeing?

aaronjensen avatar Nov 09 '15 00:11 aaronjensen

When a file changes, it reruns 0 tests with every version of karma >= 0.13.1. I've moved the boilerplate into a function that is called with the webpack require.context.

// hotloader.js
module.exports = function hotLoader(context) {
  const __karmaWebpackManifest__ = [];

  function inManifest(path) {
    return __karmaWebpackManifest__.indexOf(path) >= 0;
  }

  let runnable = context.keys().filter(inManifest);
  // Run all tests if we didn't find any changes
  if (!runnable.length) { runnable = context.keys(); }
  runnable.forEach(context);
};

// entry
hotLoader(require.context(...))

necolas avatar Nov 09 '15 06:11 necolas

@necolas sorry, can't reproduce. Try the most simple case in your entry:

const __karmaWebpackManifest__ = [];

require('./specHelper');
const testsContext = require.context('.', true, /\/(?!(flycheck))[^/]+\.spec$/);

function inManifest(path) {
  return __karmaWebpackManifest__.indexOf(path) >= 0;
}

let runnable = testsContext.keys().filter(inManifest);
if (!runnable.length) {
  runnable = testsContext.keys();
}

runnable.forEach(testsContext);

aaronjensen avatar Nov 09 '15 06:11 aaronjensen

Actually, yes, I believe that's what the problem is. I actually replace __karmaWebpackManifest__ with the actual manifest in webpack, so moving it out of your entry probably won't work.

aaronjensen avatar Nov 09 '15 06:11 aaronjensen

Hmm, it had been working and works with some versions of Karma.

necolas avatar Nov 09 '15 16:11 necolas

Tried inlining everything in my entry file and I have the same problem. I'm using webpack 1.12.2

necolas avatar Nov 09 '15 17:11 necolas

@aaronjensen would it be possible to add a way to signal to karma (via a kill signal or some http url) so that I can get karma to run all the tests without restarting karma?

tarjei avatar Nov 18 '15 19:11 tarjei

@tarjei touching or saving your test entry point will run all the tests.

eknuth avatar Nov 18 '15 19:11 eknuth

@aaronjensen https://www.npmjs.com/package/karma-webpack-with-fast-source-maps is a great plugin which really speed up tests. I think it should be part of karma-webpack. What do you think?

maksimr avatar Mar 19 '16 01:03 maksimr

@maksimr i'm all for it, I just haven't had a lot of time to make it so. I'm also still not entirely clear on some of the changes I made, like with the multiple configurations.

aaronjensen avatar Mar 19 '16 02:03 aaronjensen

@aaronjensen I see. My opinion If you push current solution to the karma-webpack we can understand what people use and community will faster fix karma-webpack then your fork. Maybe they simple does not use multiple configuration. :)

Thanks

maksimr avatar Mar 19 '16 07:03 maksimr

//cc @MikaAK

joshwiens avatar Jul 26 '16 06:07 joshwiens

It's not directly related to karma-webpack but this https://github.com/lucassus/angular-webpack-seed/pull/51/files PR demonstrates how I workaround slow rebuild problem. Basically in the first terminal I run webpack watcher, in the configuration I have two chunks - vendor, app + specs and in the second terminal I'm running karma watcher against the generated files.

Rebuild speed in this case is almost x5 better, perfect for TDD ;)

lucassus avatar Nov 17 '16 12:11 lucassus

Very interesting!

2016-11-17 13:35 GMT+01:00 Lukasz Bandzarewicz [email protected]:

It's not directly related to karma-webpack but this https://github.com/lucassus/angular-webpack-seed/pull/51/files PR demonstrates how I workaround slow rebuild problem. Basically in the first terminal I run webpack watcher, in the configuration I have two chunks - vendor, app + specs and in the second terminal I'm running karma watcher against the generated files.

Rebuild speed in this case is almost x5 better, perfect for TDD ;)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/webpack/karma-webpack/issues/54#issuecomment-261236098, or mute the thread https://github.com/notifications/unsubscribe-auth/AAM5PxNcQb3WdBDpOQzqfMfjCwXJpMrnks5q_EodgaJpZM4FEZuQ .


Tarjei Huse Mobil: 920 63 413

tarjei avatar Nov 17 '16 12:11 tarjei