monot

monot copied to clipboard

Just making a suggestion here. Take it all with a grain of salt.

I understand how this lib is useful when you're going to create several items all at once, and you want sortable unique timestamps which are roughly time-accurate, but not strictly so.

You'd want to use this lib if your need for generating many timestamps at once was of the sort where you only create a few (maybe up to a couple dozen at most) in a burst, then a period of latency without creations, then doing more creations, etc.

If it was in a nearly-constant creation environment, and the rate were consistently higher than 1 per millisecond, then you're obviously going to introduce a lot of skew to the timestamp'ing.

I am suggesting that there should be a threshold above which this lib would not allow "too much" skew between the generated timestamp and the actual time at that moment. It might be a configurable tolerance (like no more than 500 milliseconds off "real time", or "no more than 20 timestamps in the same "tick", etc), or whatever.

If you always used the lib within those constraints, you'd never see a problem. If, either accidentally or aberrantly, the lib was being used in a way that was creating sort of run-away skew, where the meaning of those timestamps became orders-of-magnitude less accurate, then the lib could hit a threshold and refuse to do more.

It could throw an error, or an event, and you could catch this, and manually "defer" what you're doing for a few moments (a setTimeout for instance), to adjust a little bit to the load. By being configurable, the threshold could be set to whatever your tolerances are for your given use.

---

Without any threshold, however, this lib confuses me because it's using timestamps, which have a semantic meaning, in a way that far too easily (like the test shows), the timestamps become extremely disassociated from their semantic meaning. If a timestamp is skewed by 10000+ milliseconds, it clearly becomes not just a little inaccurate but vastly inaccurate.

If all you care about is a strictly monotonic counter, just emit an incrementing floating-point number based on the timestamp, and don't ever represent them as actual timestamps. If you want monotonic increment AND you want to roughly interpret the counter as a timestamp, there needs to be a way to cap the skew, IMO.

Just some food for thought. :)

BTW, as a very different alternative, this lib could simply emit both a timestamp AND the skew value for that timestamp, and let the calling code deal with whether the difference is within their tolerance or not. Imagine storing records where you sort by the monotonic timestamp, but you ALSO store the skew, so in any place where you interpret the counter as a timestamp, you can adjust it back to "real-time", allowing then for non-uniqueness in places where that's more semantically accurate.

kyle,

this is a good point, thanks for taking the time to make it.

i certainly wouldn't use this library for such a serious use case... for such throughput, it definitely makes sense to introduce something else to provide uniqueness until the skew is acceptable... for example, in a theoretical chat app, i wouldn't rely on this for uniqueness across all messages, but only the messages per "room". in other words, i'd rather use this to create timestamps and create keys outside of this library than change the library to accomodate such throughput.

perhaps the best compromise is to provide a disclaimer in the README.

jed