Sonic pi non-deterministic behavior with multi-thread time state increment

[CharlesFauman]

While attempting to implement an atomic increment with sonic pi's time state system, I came across some non-deterministic behavior using the time state system. Below is the smallest reproduceable example I could come up with.

The following code is nondeterministic:

set :foo, 0

in_thread do
  set :foo, get[:foo] + 1
end

in_thread do
  set :foo, get[:foo] + 1
end

in_thread do
  set :foo, get[:foo] + 1
end

sleep 0.1
puts get[:foo]

it will print 1 most of the time, and about 1/10 of the time print 2. Note that adding more of the in_thread blocks increases the chance of a non-1 result.

System: I am running Sonic Pi v3.3.1 on Windows

[samaaron]

Hi there, thanks for reporting this.

Could you just try to see if the behaviour is similar for you on v4 of Sonic Pi?

https://github.com/sonic-pi-net/sonic-pi/releases

[emlyn]

I just tried running the code on v4.0.3 a few times, and got mostly 1 printed, but occasionally 2, so it seems that the non-deterministic behaviour is still there.

[CharlesFauman]

Downloaded the windows v4.0.3 release on Windows 11, and re-ran. I'm still running into the same issue.

[samaaron]

Useful to know, thanks. I’m planning on fully rewriting the timestate system in Elixir so this will serve as an excellent test case.

[CharlesFauman]

For my understanding, what is the intended behavior?

One possibility is that one atomic block of each thread (in the order of thread creation) is run before any thread can execute its next block. In this case, each thread does a get -> 0, then does a +1 -> 1, and then the value is set to 1. Here, 'atomic block' means something like any stretch of commands without a get/set/sleep.

another possibility is that 'atomic block' can include any number of get/sets, but cannot contain sleep commands. In this case, we would get(0), set it to 1 in the first thread, get(1) set to 2 in the second thread, same for the third thread and end up with 3.

If the first is intended, would it be possible to include a "get and set" operation in future releases, such that this example could return 3? as that is the behavior i'd like to use - eg. just keep track of a counter.

[samaaron]

The idea is that get and set use the virtual time to do lookup. All 4 calls to set are at the same virtual time, but also happen in different threads. Threads also have a creation id which should be used to order them and the most recently created thread should win.

However, the issue here is that the call to get needs to wait until there is no possibility of no other threads being created and call set to therefore modify the result of get. This hasn't been implemented correctly and the couple of attempts I made to do so totally broke everything - so I just hacked it by calling sleep to force a delay - but obviously that's non-deterministic as is demonstrated here.

This is the main bug I want to fix with a complete rewrite - but I also need to consider how it will work with the new Link metronome - there's likely some subtle but hairy edge cases to consider.

[emlyn]

I'm not sure if I'm understanding correctly, but I think you're implying that the expected output of this snippet would be "1", is that correct?

If that's the case, then as @CharlesFauman mentioned there should really be some sort of atomic get and set operation to be able to implement counters. Maybe something along the lines of:

swap :foo do |val|
  val + 1
end

[samaaron]

I'm not sure if I'm understanding correctly, but I think you're implying that the expected output of this snippet would be "1", is that correct?

No, my goal was for the value to be 3 as all the sets would have already taken place in order of their creation time/order. However, it may not prove to be trivial to implement this correctly - time will tell.

[emlyn]

No, my goal was for the value to be 3 as all the sets would have already taken place in order of their creation time/order

Ah OK, that makes sense, thanks!