gtfs-realtime-validator icon indicating copy to clipboard operation
gtfs-realtime-validator copied to clipboard

Published 20 hours ago • verified publisher icon CUTR-at-USF

Speed up gtfs-validator component build

Open barbeau opened this issue 7 years ago • 3 comments

Summary:

mvn package can take a while to execute - most of this time is due to pulling the components needed to build gtfs-validator-webapp (https://github.com/conveyal/gtfs-validator/tree/master/gtfs-validator-webapp) so we can display the results in our webapp.

We should try to speed this up. We are using https://github.com/eirslett/frontend-maven-plugin to build the webapp via npm and gulp + browserfy.

Possible options for speedup:

  1. Examine caching - does frontend-maven-plugin/npm/gulp/browserfy allow caching so it doesn't do a clean build every time?
  2. Examine execution phase for frontend-maven-plugin - in pom.xml, we have the <phase> as compile, but the default according to https://github.com/eirslett/frontend-maven-plugin#usage the default <phase> is generate-resources. Would changing to generate-resources speed up the build?
  3. Include an option to allow people to turn off building the gtfs-validator component if they don't need static GTFS validation.

Steps to reproduce:

Execute mvn package

Expected behavior:

Build fairly quickly

Observed behavior:

Builds take a while to complete - around 3 min 16 sec for mvn clean package, and about 2 min 9 sec for mvn package after that initial build on my dual CPU Xeon @ 2.5 GHz and 16 GB of RAM.

barbeau avatar Apr 25 '17 16:04 barbeau

For speeding up NPM installs, see https://lincolnloop.com/blog/speeding-npm-installs/.

barbeau avatar May 30 '17 20:05 barbeau

For caching with frontend-maven-plugin see discussion at https://github.com/eirslett/frontend-maven-plugin/issues/27, and potential solution at https://github.com/eirslett/frontend-maven-plugin/pull/118.

barbeau avatar May 30 '17 20:05 barbeau

I tried the solution at https://github.com/eirslett/frontend-maven-plugin/pull/118, but it didn't seem to have any effect - I didn't see any new cache directory. I bumped the version of frontend-maven-plugin in https://github.com/CUTR-at-USF/gtfs-realtime-validator/commit/0d0f3811bcedae16320739b2645f23595ce02c94 just so we inherit any improvements there.

Running mvn -T 4 install to increase number of threads doesn't seem to have any speed up effect either.

barbeau avatar May 30 '17 21:05 barbeau