how to check the visibility

[happydpc]

I have a question. After rasterization, the depth buffer is ok, but how to get a visibility check?

[rajkosto]

+1, obviously there's queryVisibility that gives the answer for an AABB but I thought the point was to get number of pixels where depth of rasterize was >= previous depth so you know exactly if an occluder needs to be drawn or not

[corporateshark]

You can use queryVisibility() for that. Just calculate and transform an AABB for your mesh. Here's some pseudocode assuming Box is already transformed:

__m128 bMin = _mm_set1_ps(+std::numeric_limits<float>::infinity());
__m128 bMax = _mm_set1_ps(-std::numeric_limits<float>::infinity());
for (size_t i = 0; i < 8; ++i) {
	const __m128 p = Box.GetCornerPointM128(i);
	bMin = _mm_min_ps(p, bMin);
	bMax = _mm_max_ps(p, bMax);
}
// Set W = 1 - this is expected by frustum culling code
bMin = _mm_blend_ps(bMin, _mm_set1_ps(1.0f), 0b1000);
bMax = _mm_blend_ps(bMax, _mm_set1_ps(1.0f), 0b1000);
bool needsClipping = false;
const bool isVisible = rasterizer->queryVisibility(bMin, bMax, needsClipping);

Obviously, this test is very conservative (and yet very fast) and can yield false positives - objects which are classified as visible with no actual pixels rendered. Though, this never is a problem.