rfcs
rfcs copied to clipboard
rust-lang
add float semantics RFC
Rust's floating point operations follow IEEE 754-2008 -- with some caveats around operations producing NaNs: IEEE makes almost no guarantees about the sign and payload bits of the NaN; however, actual hardware does not pick those bits completely arbitrarily, and Rust will expose some of those hardware-provided guarantees to programmers.
On the flip side, NaN generation is non-deterministic: running the same operation on the same inputs several times can produce different results.
And there is a caveat: while IEEE specifies that float operations can never output a signaling NaN, Rust float operations can produce signaling NaNs, but only if an input is signaling.
That means the only way to ever see a signaling NaN in a program is to create one with from_bits (or equivalent unsafe operations).
Floating-point operations at compile-time follow the same specification. In particular, since operations are non-deterministic, the same operation can lead to different bit-patterns when executed at compile-time (in a const context) vs at run-time. This is the first case of allowing a non-deterministic operation inside const. Of course, the compile-time interpreter is still deterministic. It is entirely possible to implement a non-deterministic language on a deterministic machine, by simply making some fixed choices. However, we will not specify a particular choice, and we will not guarantee it to remain the same in the future.
(The first paragraph is basically just documenting existing behavior. The second paragraph aims at stabilizing floating-point operations in const fn, where they are currently unstable.)
@rust-lang/lang, probably the most controversial part is the section on const.
Just as a note, the LLVM LangRef does not currently actually specify that floating-point operations obey IEEE 754 (https://github.com/llvm/llvm-project/issues/60942). Should this be mentioned in the RFC?
Just as a note, the LLVM LangRef does not currently actually specify that floating-point operations obey IEEE 754 (llvm/llvm-project#60942). Should this be mentioned in the RFC?
Good point, I added a note.
...Though based on these StackOverflow posts it appears what I want is actually an equivalent to C's feenableexcept(FE_DIVBYZERO|FE_INVALID|FE_OVERFLOW); this would raise an exception on any operation I might want to trace that produced a NaN.
As opposed to actual signaling NaNs, which (when enabled) would produce an exception as soon as they are loaded into a floating point register (this could be as soon as a floating point variable were initialized with it).
It seems like this RFC just documents current behavior while leaving open the possibility of supporting alternative floating-point environments in the future. It doesn't close doors to doing that, other than saying that we should be able to run optimizations on Rust code today that assume the default floating point environment.
I like that there was an effort to document LLVM's behavior ahead of this. The resulting specification seems like a good mix of pragmatism while guaranteeing what we can.
@rfcbot fcp merge
...Though based on these StackOverflow posts it appears what I want is actually an equivalent to C's
feenableexcept(FE_DIVBYZERO|FE_INVALID|FE_OVERFLOW); this would raise an exception on any operation I might want to trace that produced a NaN.As opposed to actual signaling NaNs, which (when enabled) would produce an exception as soon as they are loaded into a floating point register (this could be as soon as a floating point variable were initialized with it).
That's just an issue with the x87. In IEEE terms, the x87 load and store instructions correspond to format conversion (which signals invalid operation on SNaN), not copy (which does not). To my knowledge, no other FPU does this.
Though based on these StackOverflow posts it appears what I want is actually an equivalent to C's feenableexcept(FE_DIVBYZERO|FE_INVALID|FE_OVERFLOW); this would raise an exception on any operation I might want to trace that produced a NaN.
The compiler currently assumes that FP operations don't trap, so setting any kind of exception flag is UB. This needs "strictfp"-like semantics in LLVM terms, which we currently do not expose (but we could consider exposing them in the future).
@epage you have filed some requests for clippy lints related to NAN signs
- https://github.com/rust-lang/rust-clippy/issues/11717
- https://github.com/rust-lang/rust-clippy/issues/11718
- https://github.com/rust-lang/rust-clippy/issues/11720
One of them explicitly lists as motivation "Help libraries that are trying to preserve NAN signage." I would like to understand this better, since the assumption of this RFC is that by and large nobody cares about the sign of a NaN. Why do you care?
One of them explicitly lists as motivation "Help libraries that are trying to preserve NAN signage." I would like to understand this better, since the assumption of this RFC is that by and large nobody cares about the sign of a NaN. Why do you care?
The sign of nan is specified in the TOML spec, so to be able to properly preserve it, we need to track it.
Oh, interesting. I wonder how many toml implementations do that correctly... IEEE does not guarantee anything about the sign of NaNs and most languages (more or less implicitly) follow that.
The rationale for TOML supporting "-nan" appears to be this comment, which suggests that it isn't intended that "-nan" necessarily denote a different bitpattern from plain "nan".
Just to add: I could also see preserving the sign of NaN to be useful for things like Luajit which use NaN tagging when implementing JITs. How useful, I don't know, but in general you can do all sorts of horrible tricks with float types that rely on exact bit patterns.
I haven't done horrible float bit tricks that rely on the sign of NaN, but it can show up in sorting. One may establish a total order on floats where you convert them to ints. I thought I had a Compiler Explorer link for this but I don't, and I can't type the code off memory. Broadly, with the technique I'm thinking of, positive NaN sorts to the beginning and is less than all values, and negative NaN is greater than all values, as I recall.
There's also a related trick (currently supported by Rust and not involving NaN) where one can get from 16-bit samples to floats by manipulating the float such that setting the mantissa gives one a value between 2.0 and 4.0, then subtracting 3.0 (good enough for audio and avoids a division. Don't ask me for a proof; I've used it in practice though).
I doubt I'd ever care about the sign of a NaN, it just happens that I do know a few weird but useful float tricks that are adjacent to that motivation.
Yeah there could certainly be use. But currently LLVM is not able to preserve NaN signs so our hands are kind of tied here.
The proposed semantics are specifically designed to allow NaN boxing with the payload bits.
What do you mean by "preserve"? We do promise that signs are preserved under copies. The underspecification is in terms of what the sign of a freshly generated NaN is.
There's also a related trick (currently supported by Rust and not involving NaN) where one can get from 16-bit samples to floats by manipulating the float such that setting the mantissa gives one a value between 2.0 and 4.0, then subtracting 3.0 (good enough for audio and avoids a division. Don't ask me for a proof; I've used it in practice though).
I have a vague idea of what you're getting at, and it should in fact be exact, though it probably requires some bit-twiddling to deal with the sign?
@Muon It doesn't for my use case, but I apparently don't have a link to the original source and it's off topic and Rust supports it in any case since it's not NaN manipulation. If you're using it to generate random noise, you can just use unsigned 16-bit mantissas. My C++ implementation of this is public but doesn't link back to the original source and I discovered this years ago, so credit to the random blog I found it and the argument for why it works via I guess. Still, it is a really neat way to get 4 audio-quality random floats out of one random u64 as output from something like Xoroshiro.
I realized that we should probably talk about the behavior of float-returning platform intrinsics, in particular the SIMD operations in core::arch. Some of these are currently implemented using portable SIMD operations that LLVM understands and optimizes, which means they do not have the NaN behavior of the underlying assembly instruction. That might be surprising.
To be clear, the platform intrinsics do return NaN when the platform would return NaN always, right? It's just not deterministic.
Regardless of this RFC that should probably be made more clear...somewhere. Non-deterministic NaN doesn't affect me personally but I would have assumed that any intrinsic that matches the platform's documentation which claims to be a specific instruction would come out as that instruction.
Discussion seems to have calmed down; nominating for T-lang.
EDIT: turns out FCP is already ongoing... but basically nobody checked their boxes and I don't think this was nominated before.
@tmandry
It seems like this RFC just documents current behavior while leaving open the possibility of supporting alternative floating-point environments in the future. It doesn't close doors to doing that, other than saying that we should be able to run optimizations on Rust code today that assume the default floating point environment.
It's not just current behavior.
The RFC also says we should allow float operations in const fn, which is currently not stable. This is a somewhat profound decision since it is the first non-deterministic operation we stably allow in const fn. (We already allow those operations in const/static initializers.)
And there is a caveat: while IEEE specifies that float operations can never output a signaling NaN, Rust float operations can produce signaling NaNs, but only if an input is signaling. That means the only way to ever see a signaling NaN in a program is to create one with
from_bits(or equivalent unsafe operations).
This clause in the summary currently seems very lax. Can an operation with an SNaN-input produce any SNaN-output?
LLVM currently documents:
Signaling NaN outputs can only occur if they are provided as an input value. For example, “fmul SNaN, 1.0” may be simplified to SNaN rather than QNaN.
Floating-point math operations are allowed to treat all NaNs as if they were quiet NaNs. For example, “pow(1.0, SNaN)” may be simplified to 1.0.
While their wording is not entirely unambiguous, I would (maybe naively) read it as "output the specific SNaN that was an input", with the implicit assumption that this is only done when the operation is an identity operation for numeric values.
One possibility that would closely match the possible uses of SNaN without inhibiting actual optimizations could be:
If some part of a Rust floating point computation is equivalent to an operation purely on the sign bit of a numeric input, then that same sign operation may be used to perform the computation even when the input may be NaN.
For example, if
x * yis evaluated withy == -1.0, the result may be computed as eitherx * -1.0or-x, which differ only in their handling of NaN: In that case,x * -1.0returns a quiet NaN with unspecified sign and payload, whereas-xflips the sign bit and preserves the quiet bit and payload exactly.
This clause in the summary currently seems very lax. Can an operation with an SNaN-input produce any SNaN-output?
No. The set of NaNs (signaling and otherwise) that can be generated is described in great detail in the "reference" section. It is not the point of the summary to fully repeat that.
One thing which seems to be missing is 32bit to 64bit FP conversions and the sign of NaNs. For most ISAs, it carries over the sign. This is NOT true for RISC-V though. This has caused confusion in some cases with C code due to float being passed as double for variadic functions (varargs).
How is it missing? The rules in the "reference" section apply to such casts as well. This is explicitly stated. The sign of the returned NaN is non-deterministic.
@rfcbot reviewed
This is a complex space, big thanks to @RalfJung for laying it out so clearly.
@rustbot labels -I-lang-nominated
We discussed this in the lang design meeting today:
- https://github.com/rust-lang/lang-team/issues/273
People were feeling good about this, and it's now in FCP, so we can unnominate.
Huge thanks to @RalfJung for putting together this substantial body of work.
The final comment period, with a disposition to merge, as per the review above, is now complete.
As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.
This will be merged soon.
The lang team has accepted this RFC, and we've now merged it.
Thanks to @RalfJung for pushing forward this important work, and thanks to all those who reviewed this and provided useful feedback.
For further updates, follow the tracking issue:
- https://github.com/rust-lang/rust/issues/128288