Parallel command sequences

[oker1]

I've been skimming through the docs, and coming from ScalaCheck i was looking for generating parallel command sequences. Is it supported?

[untoldwind]

No, that is still on the todo-list. Unluckily it's the most complicated part and since parallel processing in go is quite different to multi-threading in java/scala, I still have to find a suitable example (in go) where it would make sense to have this feature.

[oker1]

Thanks for the clarification!

[prateek]

I have a use-case in mind for this - testing the different parts of a timeseries database with concurrent operations (user operations - reads/writes; housekeeping tasks - serialisation to disk/cleanup/etc). We've reproduced numerous bugs by running operations in parallel, some examples - https://github.com/m3db/m3db/pull/773, https://github.com/m3db/m3db/pull/502, https://github.com/m3db/m3db/pull/409, https://github.com/m3db/m3db/pull/372. We currently create such tests once we see issues in production but we'd obviously prefer to catch them earlier. To that effect, we're looking to add parallel stateful property tests.

Here's how I'm thinking the implementation in gopter might end up looking like (based on some ideas from Erlang): generate a linear sequence of operations A -> B -> C -> D -> E -> F -> G, and then transform that into two forks:

      ,-> C -> E -> G
A -> B
      '-> D -> F

And then simulate the operations using go-routines (with a gate to ensure all are up and runnable before execution starts).

Open questions:

1. How many bugs this will catch in regular tests (i.e. will go test -race be enough).

Ideas to try if doesn't work very well: i. Simulate all interleavings of the commands too - i.e. not just the forked tree, but also ensure that the execution switches after each command execution ii. Compare state of the system after the operations run against state of the system after running each possible interleavings of the operation sequentially, at least one of them should match. This'd required State to implement Equal(other State) bool. iii. Add support for yield points in end user code by adding a new subpackage gopter/commands/parallel, something like:

// +build parallel
// gopter/commands/parallel/yield_enabled.go
import "runtime"
func Yield() {
  runtime.Gosched() // or time.Sleep(0)
}

// +build !parallel
// gopter/commands/parallel/yield_enabled.go
import "runtime"
func Yield() {}

[prateek]

@untoldwind I'd love to get your feedback before I start implementing any of this.

[untoldwind]

ScalaCheck operates a bit more simple to my understanding. It "just" generates several sequences of commands/operations and applies them in parallel on the system under test checking the expected state within each thread.

The "problem" with this approach is that usually you just see that something goes haywire without a clear idea what exactly caused the problem. Even though in most cases it's already enough to know that there is a problem, the branching idea might generate some more helpful insights.

I'm not so sure about the Yield() though (maybe just because I'm not very keen on build switches in general). For starters it might be more helpful just to run the command-sequences with twice the number of GOMAXPROCS. Or maybe add a WaitGroup after each command (i.e. at all or some of the '->' in your diagram), though this might have a huge impact on the runtime of the tests.