DirectXTex
DirectXTex copied to clipboard
Add FP16 NaN/infinity handling flags
Adds 2 flags for more control over special floating point values when converting to FP16. May be useful for applications that depend on NaN/infinity for special handling, like using NaN as a transparency sentinel.
TEX_FILTER_FLOAT_SATURATE_TO_INF saturates out-of-range values to infinity, including existing infinities. TEX_FILTER_FLOAT_KEEP_NANS keeps NaNs instead of forcing them to zero.
Default behavior is convert NaN to zero and saturate to max range.
I'd have implemented these as part of the _ConvertScanline
special-case path rather than _LoadScanline
/_StoreScanline
since it's specific to conversion codepaths. I realize something would need to change in Store
with respect to the clamp, but maybe a bool clamp = true
parameter would do it.
Since it's specific to FP32 to FP16 conversions, maybe name the flags: TEX_FILTER_FLOAT16_*
I totally get what you are going for here, I'm just wondering about some other impacts this might have elsewhere. I'll think about it a bit more...
Revisiting this issue part of the problem I think is the functionality offered in CVT16/F16C. These instructions let you control rounding, but don't have any ability to control NANs and/or DENORMS. I of course can implement it in software, but it will slow it down a bit.
As such, I think the default behavior needs to be "use NANs and DENORMs" and provide explicit flags for flushing DENORMs (which would need to be implemented for both F16 and F32 to make sense) and preferring INFs to specials for conversions.
I have some DirectXMath bugs in the handling of NANs. and DENORMS to deal with first, and when I wrap those up I'll revisit this PR.