spec icon indicating copy to clipboard operation
spec copied to clipboard

Remove monotonicity guarantee from spec

Open ad-si opened this issue 4 years ago • 4 comments

That's basically the summary of https://github.com/ulid/spec/issues/11 Reasons mentioned are e.g.:

  • Makes it stateful
  • Worst case scenario leads to just one ULID per millisecond
  • Can only work in a concurrent setting and not in an async one
  • ...

Maybe it can be moved to an addendum as a recommendation of additional features that could be provided by an implementation.

ad-si avatar Jan 12 '20 22:01 ad-si

I agree, although I would argue for a sub-specification, or, dare I say a ULIDv2.
Because I think providing a specification around actual monotonically increasing ULIDs is that important.

Ulid is a great spec. It's simple and effective, and it could be useful for high-performance, multithreaded applications if it offered a recommendation for actual monotonic guarantees.
I am thinking of use cases such as RDBMS extensions, multiplayer games, high-speed trading, etc. As computers gain more and more and more cores, this becomes even more relevant.

We should update the spec and the documentation to remove the existing recommendation for monotonicity, since it doesn't actually work, and add a new approach of updating the most significant bits of the random section.

rrichardson avatar Jun 04 '20 19:06 rrichardson

Counter point: The ULID monotonicity requirement is extremely useful to me for storing data such as logs. Making them non monotonic removes much of the value - it's then just a unique Id with a time stamp and not even sortable despite that.

Re the reasons thus far:

Makes it stateful Yes. A benefit at a cost. I haven't seen this as a blocker in practice though.

Worst case scenario leads to just one ULID per millisecond The worst case scenario can be avoided easily (#39)

Can only work in a concurrent setting and not in an async one There are alternate solutions - perhaps this is the wrong tool for the job in that setting? Having threads that operate on the same clock and requiring order between events within a given millisecond doesn't seem like an every day need.

TomMD avatar Feb 03 '21 07:02 TomMD

With all the current server architectures (amd64, aarch64) providing at minimum a 16-byte atomic compare-exchange, concurrency might be not that bad after all. The problem is, well, I've yet to see anyone make a lock-free ulid implementation other people can refer to...

Artoria2e5 avatar May 10 '21 03:05 Artoria2e5

Monotonicity is the primary feature of ULID, without it it is just a base32 uuid, as it cannot be used for ordering even when all the ULIDs are generated from the same point/host.

On the other hand, when ULIDs are generated from different hosts, since time synchronization at the millisecond level cannot be guaranteed in practice, a single host with an incorrect time can make ULID's ordering irrelevant for a whole dataset. So sub-millisecond ordering is nitpicking, as you can be off by seconds (common), hours (timezone issues) or even days (time synch failures), and this issue needs to happen only once in a dataset's lifetime to negate the advantages of ULID.

EricGrange avatar Dec 20 '22 07:12 EricGrange