CesiumGS
Pixel perfect pick/drillpick for imagery layers
Copied from https://github.com/CesiumGS/cesium/issues/8692
https://github.com/CesiumGS/cesium/pull/9651 added ImageryLayerCollection.pickImageryLayers which determines the imagery layers that are intersected by a pick ray. This only checks the rectangle extents and does not check the alpha channel of the imagery. There are many cases where a pixel perfect check is required.
| Transparent imagery | Imagery Cutout | Imagery Layers Split | Day/night split |
|---|---|---|---|
 |
 |
 |
 |
As I was reviewing https://github.com/CesiumGS/cesium/pull/9651 I started to think more about what it would take to implement per-pixel picking. I'm abstracting over a lot of details but the approach would be something like this.
Create a pick id for each imagery layer. I'm not sure if this would go in the globe or in the imagery layer itself, but the globe would eventually need access to the pick ids.
var pickId = context.createPickId({
// This is slightly awkward because imageryLayer isn't actually a primitive
// But it does have a show property which is required for drill picking to work.
primitive: imageryLayer
})
Each globe tile has a set of overlapping imagery textures called "day textures" that are sent to the GPU as a uniform array of samplers. There would need to be a corresponding uniform array of pick colors. The pick color is the 32-bit integer ID packed into an RGBA8.
Next is crafting the pick shader. The point of the pick shader is to render the pick color instead of the normal globe color so that the rendered imagery can be identified. The pick pass works by rendering the scene and then reading back the color and finding the pick object associated with the color.
Special care is needed so that the pick color is not modified in any way in the shader. Brightness, contrast, HDR, atmosphere, fog, and other effects need to be disabled. Alpha blending of imagery layers needs to be disabled. If a texture has any amount of alpha it should overwrite the pick color of the layer below it. The imagery alpha channel is important but the color is completely ignored, except when using the color-to-alpha effect. If the final color is completely transparent then the shader should call discard so that no pick color is written.
This may also require changes in GlobeTranslucencyState.
Is there any plan to implement pixel-perfect picking? In my use case, I’m working with overlapping drone imagery layers, some of which are cutout images processed by a server. I need a way to retrieve a list of items ordered by their 'z-index' when a click event occurs, and also get the coordinates of the click event. Would this feature be feasible?"
as many other people, I am interested in this, especially for wms layers!
how could we help achieving this?
I thought it would be nice to achieve it by somehow retrieving the 2d geometries which form a layer, so we could check if the mouse position is inside one of those 2d polygon, but I guess this is a wms limitation also.
currently if I enable multiple layers of the same wms services, with the current methods (like pickImageryLayers, pickImageryLayersFeatures) I can't retrieve the wms layer selected, with pickImageryLayers I got all the layers added to the globe, while with the pickImageryLayersFeatures, the result of the promises returned have not enough info to understand which is the selected layer.
