mitsuba2 icon indicating copy to clipboard operation
mitsuba2 copied to clipboard

Published 20 hours ago • verified publisher icon mitsuba-renderer

[❔ other question] Sampler with masked parameter

Open pjessesco opened this issue 3 years ago • 5 comments

Summary

I'm confused how next_1d() works with masked parameter.

System configuration

  • Platform: Ubuntu 20.04
  • Compiler: clang9
  • Python version:
  • Mitsuba 2 version: 2.2.1
  • Compiled variants:
    • packet_rgb
    • ...

Description

I'm making few modifications with independent sampling, and I found a weird output that i'm confused.

I was expecting that if i put mask parameter to next_1d(), for example, [T, T, T, T, F, F, F, F], first 4 lanes outputs new random number, and last 4 lanes does not. However I found that it does not works as expected, so I wrote a short code inside render() to investigate how does it works exactly.

ref<Sampler> sampler = sensor->sampler()->clone();

if constexpr(is_array_v<Float>){
    sampler->seed(1);
    std::cout<<"Test 1 : next_1d(Mask(true))"<<std::endl;
    Mask test1 = Mask(true);
    std::cout<<test1<<std::endl;
    std::cout<<sampler->next_1d(test1)<<std::endl;
    std::cout<<sampler->next_1d(test1)<<std::endl;
    std::cout<<sampler->next_1d(test1)<<std::endl;
    std::cout<<sampler->next_1d(test1)<<std::endl;
    std::cout<<"-----\n";

    sampler->seed(1);
    std::cout<<"Test 2 : next_1d(Mask(true,true,true,true,true,true,true,true))"<<std::endl;
    Mask test2 = Mask(true,true,true,true,true,true,true,true);
    std::cout<<test2<<std::endl;
    std::cout<<sampler->next_1d(test2)<<std::endl;
    std::cout<<sampler->next_1d(test2)<<std::endl;
    std::cout<<sampler->next_1d(test2)<<std::endl;
    std::cout<<sampler->next_1d(test2)<<std::endl;
    std::cout<<"-----\n";

    sampler->seed(1);
    std::cout<<"Test 3 : next_1d(Mask(true,true,true,true,false,false,false,false))"<<std::endl;
    Mask test3 = Mask(true,true,true,true,false,false,false,false);
    std::cout<<test3<<std::endl;
    std::cout<<sampler->next_1d(test3)<<std::endl;
    std::cout<<sampler->next_1d(test3)<<std::endl;
    std::cout<<sampler->next_1d(test3)<<std::endl;
    std::cout<<sampler->next_1d(test3)<<std::endl;
    std::cout<<"======\n";
}

and here is the output.

Test 1 : next_1d(Mask(true))
[1, 1, 1, 1, 1, 1, 1, 1]
[0.939024, 0.640392, 0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769]
[0.691957, 0.647093, 0.0863611, 0.130192, 0.481728, 0.0908253, 0.423571, 0.290383]
[0.969772, 0.389585, 0.409119, 0.188869, 0.441992, 0.258783, 0.609579, 0.807601]
[0.649832, 0.733565, 0.639435, 0.435836, 0.996463, 0.341717, 0.894973, 0.0976492]
-----
Test 2 : next_1d(Mask(true,true,true,true,true,true,true,true))
[1, 1, 1, 1, 1, 1, 1, 1]
[0.939024, 0.640392, 0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769]
[0.939024, 0.640392, 0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769]
[0.939024, 0.640392, 0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769]
[0.939024, 0.640392, 0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769]
-----
Test 3 : next_1d(Mask(true,true,true,true,false,false,false,false))
[1, 1, 1, 1, 0, 0, 0, 0]
[0.939024, 0.640392, 0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769]
[0.939024, 0.640392, 0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769]
[0.939024, 0.640392, 0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769]
[0.939024, 0.640392, 0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769]
======

Test 1 works as intended, but strange thing is difference between test 1 and test 2. They both given true mask, but test 2 outputs same numbers, and test 3 too.

I was confused that is it issue for enoki or mitsuba, and i wrote here because my reproduce code is in mitsuba.

Is there something that I'm missing?

Steps to reproduce

For bug report, please enumerate the steps to follow in order to reproduce the issue <- [remove this]

  1. ...
  2. ...

pjessesco avatar Aug 31 '20 03:08 pjessesco

Hi @pjessesco ,

Here the active argument of next_1d() can be used by the sampler to skip some "expensive" computation as it doesn't need to process the inactive lanes. IIRC the current samplers always generate the random numbers for the whole wavefront as horizontal operations (e.g. ek::all(), ek::none(), ek::any(), ...) are usually more expensive than the computation itelf.

The same thing happens for the independent sampler, which doesn't even use the active argument as it wouldn't be cheaper to only generate a subset of the random numbers.

On the other hand, the caller shouldn't use the returned value for the inactive lanes, as those might could contain anything (e.g. when using a different sampler that could leave those uninitialized).

In your case, you should get exactly the same output with the 3 different masks.

Speierers avatar Aug 31 '20 07:08 Speierers

Actually, here is what I get when trying on my machine:

Test 1 : next_1d(Mask(true))
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769, 0.420491, 0.320307, 0.57313, 0.525919, 0.619794, 0.573439, 0.361748, 0.0709599, 0.577559, 0.460458]
[0.161539, 0.98692, 0.60787, 0.761549, 0.501588, 0.768057, 0.495872, 0.154365, 0.454206, 0.541663, 0.218202, 0.464486, 0.797986, 0.158145, 0.68261, 0.851795]
[0.0718271, 0.0913476, 0.475357, 0.0853932, 0.215652, 0.492392, 0.10917, 0.115694, 0.618508, 0.722628, 0.36452, 0.637849, 0.0988867, 0.791447, 0.243442, 0.331446]
[0.224776, 0.236, 0.448957, 0.156048, 0.172392, 0.642663, 0.418691, 0.575462, 0.476496, 0.239153, 0.584359, 0.792597, 0.600566, 0.215244, 0.465891, 0.710187]
-----
Test 2 : next_1d(Mask(true,true,true,true,true,true,true,true,true,true,true,true,true,true,true,true))
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769, 0.420491, 0.320307, 0.57313, 0.525919, 0.619794, 0.573439, 0.361748, 0.0709599, 0.577559, 0.460458]
[0.161539, 0.98692, 0.60787, 0.761549, 0.501588, 0.768057, 0.495872, 0.154365, 0.454206, 0.541663, 0.218202, 0.464486, 0.797986, 0.158145, 0.68261, 0.851795]
[0.0718271, 0.0913476, 0.475357, 0.0853932, 0.215652, 0.492392, 0.10917, 0.115694, 0.618508, 0.722628, 0.36452, 0.637849, 0.0988867, 0.791447, 0.243442, 0.331446]
[0.224776, 0.236, 0.448957, 0.156048, 0.172392, 0.642663, 0.418691, 0.575462, 0.476496, 0.239153, 0.584359, 0.792597, 0.600566, 0.215244, 0.465891, 0.710187]
-----
Test 3 : next_1d(Mask(true,true,true,true,true,true,true,true,false,false,false,false,false,false,false,false))
[1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0]
[0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769, 0.420491, 0.320307, 0.57313, 0.525919, 0.619794, 0.573439, 0.361748, 0.0709599, 0.577559, 0.460458]
[0.161539, 0.98692, 0.60787, 0.761549, 0.501588, 0.768057, 0.495872, 0.154365, 0.57313, 0.525919, 0.619794, 0.573439, 0.361748, 0.0709599, 0.577559, 0.460458]
[0.0718271, 0.0913476, 0.475357, 0.0853932, 0.215652, 0.492392, 0.10917, 0.115694, 0.57313, 0.525919, 0.619794, 0.573439, 0.361748, 0.0709599, 0.577559, 0.460458]
[0.224776, 0.236, 0.448957, 0.156048, 0.172392, 0.642663, 0.418691, 0.575462, 0.57313, 0.525919, 0.619794, 0.573439, 0.361748, 0.0709599, 0.577559, 0.460458]
======

The state of the PCG32 random number generator only changes for the active lanes, which makes sense. Are you running this on latest master?

Speierers avatar Aug 31 '20 07:08 Speierers

Hi, @Speierers , thanks for reply.

I just cloned and tested again on the 2fb0c9634e696887fae3921a60818a3a503c892e, latest master.

Screenshot from 2020-08-31 16-42-46

And here is the result :

Test 1 : next_1d(Mask(true))
[1, 1, 1, 1, 1, 1, 1, 1]
[0.939024, 0.640392, 0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769]
[0.691957, 0.647093, 0.0863611, 0.130192, 0.481728, 0.0908253, 0.423571, 0.290383]
[0.969772, 0.389585, 0.409119, 0.188869, 0.441992, 0.258783, 0.609579, 0.807601]
[0.649832, 0.733565, 0.639435, 0.435836, 0.996463, 0.341717, 0.894973, 0.0976492]
-----
Test 2 : next_1d(Mask(true,true,true,true,true,true,true,true))
[1, 1, 1, 1, 1, 1, 1, 1]
[0.939024, 0.640392, 0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769]
[0.939024, 0.640392, 0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769]
[0.939024, 0.640392, 0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769]
[0.939024, 0.640392, 0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769]
-----
Test 3 : next_1d(Mask(true,true,true,true,false,false,false,false))
[1, 1, 1, 1, 0, 0, 0, 0]
[0.939024, 0.640392, 0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769]
[0.939024, 0.640392, 0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769]
[0.939024, 0.640392, 0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769]
[0.939024, 0.640392, 0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769]
======

I got a same result as before.

pjessesco avatar Aug 31 '20 07:08 pjessesco

I tried on SIMD 16 machine, and works as expected. Could you reproduce the issue on SIMD 8 machine?

Test 1 : next_1d(Mask(true))
[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
[0.939024, 0.640392, 0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769, 0.420491, 0.320307, 0.57313, 0.525919, 0.619794, 0.573439, 0.361748, 0.0709599]
[0.691957, 0.647093, 0.0863611, 0.130192, 0.481728, 0.0908253, 0.423571, 0.290383, 0.620354, 0.983104, 0.815128, 0.0097878, 0.0061661, 0.468453, 0.104588, 0.628291]
[0.969772, 0.389585, 0.409119, 0.188869, 0.441992, 0.258783, 0.609579, 0.807601, 0.235767, 0.992704, 0.156054, 0.555246, 0.781429, 0.429375, 0.789486, 0.177813]
[0.649832, 0.733565, 0.639435, 0.435836, 0.996463, 0.341717, 0.894973, 0.0976492, 0.911742, 0.0294338, 0.622592, 0.201443, 0.0469642, 0.697679, 0.489288, 0.694442]
-----
Test 3 : next_1d(Mask(true,true,true,true,false,false,false,false))
[1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0]
[0.939024, 0.640392, 0.625784, 0.823438, 0.220119, 0.886624, 0.144777, 0.628769, 0.420491, 0.320307, 0.57313, 0.525919, 0.619794, 0.573439, 0.361748, 0.0709599]
[0.691957, 0.647093, 0.0863611, 0.130192, 0.481728, 0.0908253, 0.423571, 0.290383, 0.420491, 0.320307, 0.57313, 0.525919, 0.619794, 0.573439, 0.361748, 0.0709599]
[0.969772, 0.389585, 0.409119, 0.188869, 0.441992, 0.258783, 0.609579, 0.807601, 0.420491, 0.320307, 0.57313, 0.525919, 0.619794, 0.573439, 0.361748, 0.0709599]
[0.649832, 0.733565, 0.639435, 0.435836, 0.996463, 0.341717, 0.894973, 0.0976492, 0.420491, 0.320307, 0.57313, 0.525919, 0.619794, 0.573439, 0.361748, 0.0709599]
======

pjessesco avatar Aug 31 '20 08:08 pjessesco

We are refactoring the whole codebase, including major changes/redesign in enoki. Most likely this issue will go away once we are done with this.

I won't have the time to debug this on a SIMD 8 machine unfortunately.

Speierers avatar Aug 31 '20 08:08 Speierers