Improve precision of rand sampling without conversion
The implementation from #73 is slightly less precise than necessary, why not use the same one used by
Float16? (e.g.BFloat16(Float32(rand(r, UInt8)) * Float32(0x1.0p-8))
Originally posted by @oscardssmith in https://github.com/JuliaMath/BFloat16s.jl/issues/73#issuecomment-2338109848
@oscardssmith I think I misunderstood your comment first, this is about the "[1,2)-1" method for sampling uniformly from $$[0,1)$$ being not full entropy for the last bit(s), e.g. it's always 0 in $$[1/2, 1)$$; the last two bits are always 0 between $$[1/4, 1/2)$$ etc. I described this here when I wrote randfloat for RandomNumbers.jl to illustrate with the rand(::BFloat16) that's now in #master
julia> bitstring.(filter(x -> x >= 1/2, rand(BFloat16, 50)))
20-element Vector{String}:
"0011111100010100"
"0011111101101100"
"0011111100111110"
"0011111101100110"
"0011111101001110"
"0011111101100100"
"0011111101110010"
"0011111100100010"
"0011111101110110"
"0011111101110110"
"0011111100011100"
"0011111100110110"
"0011111101000110"
"0011111100000110"
"0011111101100100"
"0011111100101100"
"0011111100000000"
"0011111101101100"
"0011111100011110"
"0011111100000010"
(last bit is always zero)
What's proposed here is better because it does sample from every float in $$[1/2, 1)$$ but it has the same problem for smaller powers of two, e.g. only every float is sampled in $$[1/4, 1/2)$$
julia> rand2(::Type{BFloat16}) = BFloat16(Float32(rand(UInt8)) * Float32(0x1.0p-8))
rand2 (generic function with 2 methods)
julia> bitstring.(filter(x -> 1/2 > x >= 1/4, [rand2(BFloat16) for i in 1:100]))
17-element Vector{String}:
"0011111010111000"
"0011111011010100"
"0011111011000000"
"0011111011000010"
"0011111010001110"
"0011111010011100"
"0011111010000110"
"0011111010000110"
"0011111010110100"
"0011111011100100"
"0011111010101000"
"0011111011111100"
"0011111011100000"
"0011111010011110"
"0011111011100000"
"0011111011100100"
"0011111010010110"
And so statistically it's 1 bit of entropy better than the [1,2)-1-method.