BunSpreader
BunSpreader copied to clipboard
ThePrimeagen
Messing around with a lockfree Map
Haven't tested this at all, so it could be a lot worse idk. I think a custom hasher would boost performance a lot, but I'm too lazy.
Also there was a bug with the original code. When emptying the queue, it stops at the first item which is not outdated, without checking the rest of the queue. Meaning any outdated items behind it will not be seen, nor removed. These changes make sure to check the entire queue, which is technically slower.
Another way to boost performance would be to individually schedule the removal of each item using time-out workers. This way we don't have to iterate over the items at all, just remove expired ones directly with their id. However this would stray pretty far from the comparison model.
No items in the queue can be earlier than the one at the head.
All items put into the queue are put in with the exact same amount of time in the queue. All expiration dates should be equal or further in time. Check both implementations.
I cannot imagine this is somehow faster. You have to iterate on every request every single item in the map.
I think we have to go back to the drawing board. I have a great idea with two lock-free queues
No items in the queue can be earlier than the one at the head.
Imagine three requests are made, one with 3 seconds, the next 30 seconds, then another with 3 seconds again. In this scenario we would see the first 3 second is expired, but the 30 second is not, leaving the other expired 3 second in the queue.
All items put into the queue are put in with the exact same amount of time in the queue. All expiration dates should be equal or further in time. Check both implementations.
The server code does not enforce this, and the assumption was not clear.
I cannot imagine this is somehow faster. You have to iterate on every request every single item in the map.
Yeah probably not faster then. Let's see if I can spit out my worker idea real fast.
@ThePrimeagen okay modified so we're doing exactly 0 iterations. Now it's a test on how efficiently tokio manages its green threads! I'd be interested in seeing the performance, but with your given constraints it's not likely to beat a simple lock-less queue. Had fun writing it all the same :)
Okay after some basic tests on my machine, this seems to be about twice as fast as the current implementation? I may be doing something wrong, as I'm not sure how you tested.
With these changes I'm getting:
total_time: 5096 success 1000 errors 0 rps 0
And the same parameters on master:
total_time: 10084 success 1000 errors 0 rps 0
Hmm after some more testing, it seems the result vary wildly for both implementations. Sometimes it's just a minor improvement, sometimes it's as shown above.
So when I add things to the queue they all get added with the exact same amount of time. 15 seconds was my requirement. Or else the queue steady state would never exist.
I'm glad to run that Tokio steady state test though.
To me it seems like I should also throw in tokio MPSC