stdlib
stdlib copied to clipboard
gleam-lang
Change the semantics of bitwise operations to be fully platform-dependent
What @joshi-monster proposed here Joshi brings up some good points, I'd like to discuss this more
Is the idea here to sacrifice all other bit manipulation use cases for a faster multiply and divide by 2?
Bitwise operations can be used for platform specific optimisations, but they are more commonly used for manipulating bits, which this proposal seems to make impossible as the results are no longer reliable?
If you are already doing platform specific optimisations then you are already using FFI, so using platform specific operators causes you no problems that I can see.
Hi!!
First of all: sorry, I'm always a little bit overwhelmed by github and the process; the PR was meant to be a discussion 🙂
I would definitely like to see peoples' opinions on this change, and on what integer semantics on JavaScript should be in general. Right now Gleam seems to target the full range (52-bit) of safe integers.
Of note here is also that JavaScript engines include special optimisations if they can figure out that values are 32-bit integers. Patterns like (a + b)|0 are emitted by Emscripten (asm.js), and JIT engines will recognise those and generate really good code for these!
Prior art: The following other compile-to-javascript languages all use 32-bit integer semantics for their respective natural numbers type: Elm, Reason, F# uses int32 by default and has a separate bigint type, PureScript uses 32-bit semantics for all operations except division where it matches Gleams implementation, Scala.js, js_of_ocaml. I have in fact not found one that doesn't do that at least as a default.
Many thanks also to Jak for opening this issue!! ~ 💜
We deliberately moved away from this as it was repeatedly resulting in surprisingly bugs in real programs.
A future optimising compiler could apply this as an operation if we could infer the ints are small enough, but this seems unrelated to the changing of the bitwise functions?
At the very least it would be good to document that this is surprisingly slow on the JavaScript target. "Surprising" in the sense that it is unreasonable to expect the conversion to BigInt and back without documentation.
Engines typically have an optimisation for BigInt operations performed on fixed-sized numbers. We should benchmark if wrapping our big int casts in BigInt.asIntN and fixing them to 64 bits triggers this and speeds things up.
That's cool I didn't know about this method! Unfortunately, asIntN requires its argument to already be a BigInt, so it makes things strictly worse:
Input Function IPS Min P99
input stdlib 1291.0057 0.6915 0.8967
input native 90096.9753 0.0094 0.0619
input asIntN 1040.2527 0.8785 1.1372
this is 10k runs each call to minimize gleamy_bench/function call overhead)
but they are more commonly used for manipulating bits which this proposal seems to make impossible as the results are no longer reliable
Because we cast the operation back down to number anyway isnt it already unreliable to manipulate bits on the JavaScript target? We currently do undocumented lossy truncation of results larger than 64 bits vs a platform-defined truncation of values larger than 32 bits.
No because integer size is documented. Making is smaller does not change that, would be a breaking change in the language, and repeatedly caused problems in real applications when we did use such small ints.
Hi!
Sorry I was a little bit confused here I think - from what I understood from the original ask on Discord was that "nows the time to change that" meant to make that breaking change while the standard library is still < v1. Going for the performance improvement while preserving semantics would also be a huge improvement though, I think.
As you said before, integer size is documented and we would not break the language here. The goal would have been to break these specific functions instead.
Anyways, I'd be happy to make it faster too by only using BigInts or another implementation if required. This adds an extra runtime check, but I think would still be a lot better.
Optimisations sounds good, so long as we continue to support up to the current 53-bit limit.