Flaky node_test run

[johanbrandhorst]

🐛 Bug Report

One of our CI test jobs, node_test had a flaky failure.

To Reproduce

Run node_tests in CI enough times.

Expected behavior

The test should have deterministic behaviour.

Actual Behavior

It failed. See logs here.

Abbreviated logs:

[22:37:48] Using gulpfile /src/grpc-gateway/examples/internal/browser/gulpfile.js
[22:37:48] Starting 'gateway'...
[22:37:48] Starting 'server'...
[22:37:53] Finished 'server' after 5.13 s
[22:37:53] Starting 'serve-server'...
[22:37:53] 'serve-server' errored after 6.49 ms
[22:37:53] Error: ENOENT: no such file or directory, lstat '/src/grpc-gateway/examples/internal/browser/bin/example-gw-go-tmp-umask'
    at Object.lstatSync (fs.js:845:3)
    at Gaze._addToWatched (/src/grpc-gateway/examples/internal/browser/node_modules/gaze/lib/gaze.js:277:14)
    at Gaze.add (/src/grpc-gateway/examples/internal/browser/node_modules/gaze/lib/gaze.js:175:8)
    at new Gaze (/src/grpc-gateway/examples/internal/browser/node_modules/gaze/lib/gaze.js:74:10)
    at gaze (/src/grpc-gateway/examples/internal/browser/node_modules/gaze/lib/gaze.js:86:10)
    at Object.module.exports [as watch] (/src/grpc-gateway/examples/internal/browser/node_modules/glob-watcher/index.js:12:17)
    at Gulp.watch (/src/grpc-gateway/examples/internal/browser/node_modules/gulp/index.js:35:16)
    at Gulp.<anonymous> (/src/grpc-gateway/examples/internal/browser/gulpfile.js:30:8)
    at module.exports (/src/grpc-gateway/examples/internal/browser/node_modules/orchestrator/lib/runTask.js:34:7)
    at Gulp.Orchestrator._runTask (/src/grpc-gateway/examples/internal/browser/node_modules/orchestrator/index.js:273:3)
    at Gulp.Orchestrator._runStep (/src/grpc-gateway/examples/internal/browser/node_modules/orchestrator/index.js:214:10)
    at /src/grpc-gateway/examples/internal/browser/node_modules/orchestrator/index.js:279:18
    at finish (/src/grpc-gateway/examples/internal/browser/node_modules/orchestrator/lib/runTask.js:21:8)
    at cb (/src/grpc-gateway/examples/internal/browser/node_modules/orchestrator/lib/runTask.js:29:3)
    at /src/grpc-gateway/examples/internal/browser/node_modules/async/lib/async.js:52:16
    at /src/grpc-gateway/examples/internal/browser/node_modules/async/lib/async.js:269:32
[22:37:53] Finished 'gateway' after 5.19 s

It looks like the test failed to start the gateway, then just sat quietly for 10 minutes without failing. There might be two issues here, one is a random failure in gulp, and another one is that it doesn't fail the job immediately.