SpacetimeDB icon indicating copy to clipboard operation
SpacetimeDB copied to clipboard
verified publisher icon clockworklabs

Monotonic timestamps

Open coolreader18 opened this issue 8 months ago • 1 comments

Description of Changes

Fixes for timestamps, so that a reducer that happens after another is observed to have a later timestamp. Specifically:

  • when running a scheduled reducer, clamp it so that ctx.timestamp will be at least the timestamp the reducer was scheduled for.
  • when scheduling a reducer for now + delay, clamp now so it is at least the timestamp of the start of the current reducer run.

The first commit is pulled out from the second, to make it clearer what the actual logic change is.

Expected complexity level and risk

2 - timing stuff is intricate and this could mess with assumptions people have made about the way it works, but I'm pretty confident this is strictly a strong guarantee.

Testing

Not sure what would make sense - I suppose we could have a test where we set back the system clock in the middle of a reducer run, or something?

  • [ ]
  • [ ]

coolreader18 avatar Apr 15 '25 21:04 coolreader18

Alright, added a test. I couldn't get the issue to reproduce naturally, but calling timedatectl set-time <the_past> in the middle of the test run caused it to fail on master, but still succeed on this branch.

coolreader18 avatar Apr 16 '25 20:04 coolreader18

I also can't get the issue to repro naturally on master. I tried pretty hard, too: I changed the test to run 100000 iterations, set the delay to 0, and still didn't see an out-of-order run. Makes me wonder what was going on with the reporter's setup, that they apparently saw multiple anomalies of several seconds.

I'm not sure there's much value in having the test checked in, in that case; I think a test with a 100% rate of false negatives is either useless or worse than useless. The manual test where you set the system clock is enough to convince me to merge this. Remove the new smoketest, then I'll approve. Thanks!

gefjon avatar Apr 17 '25 14:04 gefjon

Yeah, I'm wondering if it's a windows-specific issue or something like that. At the very least it's a their-machine-specific issue.

coolreader18 avatar Apr 17 '25 16:04 coolreader18

Yeah, I'm wondering if it's a windows-specific issue or something like that. At the very least it's a their-machine-specific issue.

They used the Linux names for CLOCK_MONOTONIC and CLOCK_REALTIME, so I assumed they were on Linux, but who knows?

gefjon avatar Apr 17 '25 17:04 gefjon