Multi select objects by dragging over the viewer

[simplylogicninjas]

We have a idea to create a multi-select feature. By click/drag over the viewer, create a selection area. Select model objects inside this area.

Kind of like this:

https://dragselect.com/

Maybe the viewer has to be locked in a specific orientation to make this work.

Any ideas how to start with this?

[simplylogicninjas]

I can draw a select box now. But now I need to "pick" entities inside this boundary of the box. Is this possible already with Xeokit? There is a "pick" method, but that returns 1 entity based on a specific canvas position.

[simplylogicninjas]

This is my select box, rendered above the canvas. For illustration

[simplylogicninjas]

I made an implementation. Based on the scene "pick" method. Basically I create a grid on my selected area and then on the x and y axis for every grid cell there will be a "pick" method executed. It works perfectly, the only performance tweak can be to dynamically adjust the grid cell size.

[mhendrikxce]

We (well my colleague) also implemented this feature.

How our implementation works is that we map the rectangle to a frustrum that is a scaled version of the camera frustrum. Next, we check inclusion of the entities in the scene with regard to the sub-frustrum. Currently, we only include items that are completely within the frustrum by testing inclusion of their bounding boxes in the frustrum. Probably going to change that to allow partial inclusion.

Our approach however breaks down on orthographic mode. There our approach only works for "pure" orientations (FRONT, BACK, TOP, BOTTOM, etc.). So we constrained the action.

Another issue is with huge scenes in which NEAR and FAR become huge. In that case, our frustrum ends up not stretching far enough. We solved that by scaling our model. This is possible for our scenario as we load an IFC model; and thus can use BIMServer to load the bounds.

Maybe we could share :)? We would be interested in applying your approach for ortographic mode.

[xeolabs]

Hi, yes please share, would be super interesting. BTW this kind of selection is on our roadmap, but we haven't got to it yet. Since we've switched to WebGL2, we've been looking at a technique using transform feedback (WebGL2 capability), but that's still some time away (no idea when). If you could contribute this as a "non-core" feature, ie. a plugin or a utility class, then we could bundle this with a release, as an experimental feature. I'm thinking we'd also want to bundle some demo examples that test its scalability with large object sets.

[simplylogicninjas]

@mhendrikxce You're implementation sounds good, I was looking for it but didn't found out how to implement this. My implementation also has issues with large scenes, but I'm basically picking on the scene. If you can share you're implementation, it will really help and this feature will be a good plugin to extend Xeokit.

[mhendrikxce]

While I would love to claim the credit, my colleague designed and implemented it. He will get back to you this sprint (today or next week).

[OmarHussein1]

Hi all, @mhendrikxce directed me to give some implementation information for the rectangle select. Given you are able to create a rectangle ribbon: I basically generate the positions of the new frustum planes as ratios based on the rectangle ribbon.

//Clipping plane goes from -1 to 1 so length of step is 2.
  const xUnit = 2.0 / width;
  const yUnit = 2.0 / height;
//MinP is the corner point of the rectangle ribbon with smallest coordinates
  let left = minP[0] * xUnit + -1;
//MaxP is the corner point of the rectangle ribbon with largest coordinates
  let right = maxP[0] * xUnit + -1;
  let bottom = -maxP[1] * yUnit + 1;
  let top = -minP[1] * yUnit + 1;

Create a new frustum projection matrix using the new dimensions. Ratio here is the aspect ratio of the screen. This makes the frustum viewing plane a rectangle instead of a square thus fitting to the screen. NEAR_SCALING is 17. I found it using manual testing, otherwise the frustum would start behind the camera and you would be able to pick items outside of the scene. You can set the far plane based on your needs.

const subFrustumMat = math.frustumMat4(
            left,
            right,
            bottom * ratio,
            top * ratio,
            camera.frustum.near * (SelectionHandlingMode.NEAR_SCALING * ratio),
            SelectionHandlingMode.FAR_PLANE,
            math.mat4(),
);

Next step is to use the projection matrix with the cameras view matrix to create a new viewing frustum.

        let mSubFrustum = new XeokitFrustum();
        setFrustum(mSubFrustum, camera.viewMatrix, subFrustumMat);

To check for containment you can use the frustumIntersectsAABB3 method from Xeokits Frustum class,

for (let object of Object.values(this.viewer.xeokit.scene.objects)) {
            if (this.isEntityInsideFrustum(object, _frustum)) {
                entities.push(object);
            }
        }

where isEntityInsideFrustum is just a simple check:

isEntityInsideFrustum(_entity: Entity, _frustum: XeokitFrustum): boolean {
\\Rectangle selection mode can be either a 0= inside, 1= intersect or 2 = outside
        return frustumIntersectsAABB3(_frustum, _entity.aabb) === this._rectangleSelectMode;
    }

This has been working very well for us even with bigger scenes. You can debug whats going on by manually creating a frustum from an AABB and using the draw functions to visualise

        let tempFrustum: CustomFrustum =
            // near
            [[-1, -1, -1, 1], [1, -1, -1, 1], [1, 1, -1, 1], [-1, 1, -1, 1],
                [-1, -1, 1, 1], [1, -1, 1, 1], [1, 1, 1, 1], [-1, 1, 1, 1]];

        let transformMat = this.getTransformationMatrix(projMat, viewMatrix);

        for (let i = 0; i < 8; i++) {
            tempFrustum[i] = math.transformPoint4(transformMat, tempFrustum[i]);
        }

        this.scalehomogeneousCoordinates(tempFrustum);
//Create your own drawing function here, I used lines and connected them based on the vertex order
        this.drawFrustum(tempFrustum, (this._debugFrustumMap.size+ 1) % 3, color);

Hope this helps!!

[xeolabs]

I just added a marquee selection tool to BIMViewer - see issue: https://github.com/xeokit/xeokit-bim-viewer/issues/132

Please find links to commits etc in that ticket.

Thanks for your help, @OmarHussein1 , @mhendrikxce !