webmachine-ruby icon indicating copy to clipboard operation
webmachine-ruby copied to clipboard

Stop relying on timeouts in specs

Open ghost opened this issue 11 years ago • 11 comments

The specs should run in a way that doesn't rely on timeouts, in order to avoid load-dependent heisenspecs, like this one: https://travis-ci.org/seancribbs/webmachine-ruby/jobs/37496051#L1659

ghost avatar Oct 09 '14 13:10 ghost

can it be done with travis that each adapter runs in its own instance?

Asmod4n avatar Oct 09 '14 13:10 Asmod4n

Sure, but I don't see how that solves the problem of the specs asserting that a certain piece of code finish within a certain timeframe. When saying "load-dependent heisenspecs", I mean specs that succeed or fail based on the load stress of the machine they're running on. One straightforward workaround would be to increase the timeouts. A proper solution, on the other hand, would look at the correct thing, of which I'm not sure what it is.

ghost avatar Oct 09 '14 13:10 ghost

One could use a IO.pipe and wait on it till the server is started

Asmod4n avatar Oct 09 '14 13:10 Asmod4n

I had simillar idea once when spawning processes to wait on something like:

  • specific text in stdout
  • socket readiness
  • ability to respond to request

Have a look at https://github.com/drugpl/bbq-spawn/blob/master/lib/bbq/spawn.rb#L54-L84

mostlyobvious avatar Oct 09 '14 13:10 mostlyobvious

rd, wr = IO.pipe
Thread.new {
  puts rd.read
  rd.close
}
Thread.new {
  wr.write('initialized')
  wr.close
}

Or with a queue

queue = Queue.new
Thread.new {
  puts queue.pop
}
Thread.new {
  queue << 'initialized'
}

Asmod4n avatar Oct 09 '14 13:10 Asmod4n

Got new errors from HTTPkit now, https://travis-ci.org/seancribbs/webmachine-ruby/builds/38243104

Asmod4n avatar Oct 17 '14 09:10 Asmod4n

Does https://github.com/seancribbs/webmachine-ruby/blob/master/lib/webmachine/spec/adapter_lint.rb#L36 get run every time it checks something? it looks totally like it..

Asmod4n avatar Oct 17 '14 09:10 Asmod4n

@lgierth have tried something, dunno if it works as expected: https://github.com/seancribbs/webmachine-ruby/blob/spec-timeouts/lib/webmachine/spec/adapter_lint.rb#L36

Asmod4n avatar Oct 17 '14 10:10 Asmod4n

Had to re add Adapter#shutdown again, but i believe it works now... Will run it three more times to be sure.

@seancribbs @lgierth

Asmod4n avatar Oct 17 '14 14:10 Asmod4n

Eventmachine gives me headaches, but it's working now.

Asmod4n avatar Oct 17 '14 14:10 Asmod4n

Not sure if still an issue, in RES we've been using http://ruby-concurrency.github.io/concurrent-ruby/master/Concurrent/Exchanger.html to synchronize threaded tests.

mostlyobvious avatar Dec 18 '18 13:12 mostlyobvious