webots icon indicating copy to clipboard operation
webots copied to clipboard

Camera can see through some objects unexpectedly

Open PeterJCLaw opened this issue 1 year ago • 1 comments

Describe the Bug

I've hit a couple of cases where a camera can see recognition objects "through" other objects which ought to be opaque (credit to @Adimote for pinning down these cases).

In this cut-down world we have a number of recognition objects which are playing the part of fiducial markers. They're attached to the outside of other objects which are themselves not intended to be directly recognisable and should be opaque. However in a couple of cases the "markers" can be seen through other objects: image

Of the three objects being picked up, two shouldn't be:

  • the red outline on the left shows the camera picking up a recognition object attached to the brown cube within the bucket, this object is near the side of the bucket but not against it
  • the red outline above right shows the camera picking up a recognition object attached to the outside far side of the bucket

The middle one (the white square on the grey bucket) is fine -- this object is nearer the camera than anything else and is behaving as expected.

Note that the brown cube has recognition objects on all six faces, yet only the one facing the camera is picked up. Even the one which is actually partly visible out the top of the bucket is (correctly) occluded.

This is with the default occlusion setting (i.e: 1, which I believe uses centre-points). While debugging this I tried the other values:

  • Setting the occlusion value to 0 allows all of the recognition objects to be picked up, as would be expected.
  • Setting the occlusion value to 2 also allows all of these to be picked up, which I wasn't expecting. I'm not sure I understand what the expected behaviour for setting 2 is though.

Steps to Reproduce

I've put the world (camera-see-through-things.wbt) and its trivial controller in https://gist.github.com/PeterJCLaw/505d36575cddbd1cb1bc21adf489d3a1. I've cut this down significantly from our original world, though it may be possible to cut it down further -- I haven't pushed for an extreme cut-down. Running it in Webots should result in seeing the same thing as shown in the screenshot above.

Expected behavior

Ideally only the nearest of the recognition objects should be picked up -- the one on the outside of the grey bucket ("S3" in the world file) which is on the side nearest the camera.

System

  • Operating System: Ubuntu 22.04, though observed on other platforms

Additional context

PeterJCLaw avatar Oct 31 '23 22:10 PeterJCLaw

Oh, forgot to add -- there's another case we hit where if two of the brown cubes are stacked then the camera can sometimes see the "marker" from the topwards face of the bottom one in some cases where it isn't actually visible in the picture. I've added that as another world in the same gist (camera-see-through-more-things.wbt), though I'm assuming it's going to be similar to the cases already described above.

image

PeterJCLaw avatar Oct 31 '23 22:10 PeterJCLaw