Texturing: wrong colors at island borders

[devernay]

Hi,

When I look closely at the texture map, it looks like the image below, with a thin bright border around each island.

EDIT:

• This only happens if downscale >1. With downscale=1 the UV island borders are sharp and clean
• This also happens with EXR, so not a problem with premult.

So my guess is that downscale just uses the wrong filter. It's probably doing a lowpass filter on the image with the black holes, thus the mixing with black (borders are darker) and the overshoot (brighter line near the edge). Since black areas are supposed to be transparent, I would downscale the image with the alpha channel (using whatever filter you use) and then divide by the downscaled alpha. This corresponds to what is done in the first level of fillholes_pushpull from OpenImageIO.

~~This looks to me like a bad unpremultiplication (eg unpremultiply after conversion to sRGB? unpremult should always be done in linear color space).~~

~~Besides, I have suggestions for the texture maps. Real-time renderers, especially game engines, generate mipmaps from the textures. Here, mipmapping would cause the black to bleed into the textures. For this reason, I would recommend the following changes:~~

• ~~add a "UV island margin" parameter to enforce some space between the islands. This margin typically depends on the number of mipmap levels one intends to use (16 pixels is safe, 8 pixels is fine, but this should be configurable. I don't know if you're using xatlas (from GEOGRAM) to pack the UV atlas, but I remember that it may be a bit tricky to get the right parameter.~~ (check out "padding" parameter- available from "advanced parameters")
• ~~texture maps shouyd be inpainted rather than filled with black. I know this is bad for PNG, but it's good for JPEG or JPEG2K or DWA, and it's good for mipmapping too. I can give you some code for a fast inpainting method based on on Gaussian pyramid (just ask).~~ (check out "fillHoles" parameters)

Meshroom 2021.1/linux (precompiled)

[fabiencastan]

add a "UV island margin" parameter to enforce some space between the islands

The padding parameter is doing that.

texture maps shouyd be inpainted rather than filled with black

There is an option "Fill Holes", but it is not enabled by default as it is a bit slow. We currently use the fill hole method from openimageio, but I agree that it would be good to implement something more efficient and use it by default. Or maybe add another method directly in openimageio.

This looks to me like a bad unpremultiplication

Good question, I have not tested much on input images with alpha for now.

[devernay]

Actually what I'm showing is using input images without alpha. I'm showing the border of a UV island, which is probably rendered with antialiasing, so alpha < 1 in the texture map even though the input images have no alpha.

Then some bad unpremult operation probably happens.

To check this, take pictures of some uniform gray or white object (eg plaster), build the model and texture, then zoom on the texture map. This should work with the dino from https://vision.middlebury.edu/mview/data/

[devernay]

The fillholes_pushpull function in OpenImageIO does pretty much what I was doing (push-pull): https://github.com/OpenImageIO/oiio/blob/05f215cb12cd72121a76e80be673830bbd328bdf/src/libOpenImageIO/imagebufalgo.cpp#L1157 But you said it is too slow. I wonder what makes it slow.

[devernay]

The padding parameter is doing that.

I just discovered the notion of "advanced" parameters, sorry about that.

[devernay]

I edited the issue. This is an issue with how downscale is used (on the image with holes, which is wrong)

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[devernay]

This issue should not be closed, but rather fixed (that's a rather easy fix)