3D object disappears when moving out of the bounding box

[kelvkong]

• [x] Review the documentation: https://virocore.viromedia.com/
• [x] Search for existing issues: https://github.com/viromedia/virocore/issues
• [x] Use the latest ViroCore release: https://virocore.viromedia.com/docs/releases
• [x] Turn off Instant Run in Android Studio

Environment

Please provide the following information about your environment:

1. OS: Windows
2. Version: 1.13.0
3. Device(s): Pixel2

Description

The 3D object disappears when the camera cannot see the bounding box. It can move around and does not always stick to the original position and stay within the box.

1. How is the bounding box generated? Is it merely based on the 3D object itself?
2. Is it possible to programatically set the region of the bounding box? If not, any suggestion to increase the size of a bounding box?

Related issue #194

Reproducible Demo

Clone the ARBlackPanther project and add the following code

private Node createBoundingBoxNode(){
        Node boundingBoxNode = new Node();
        boundingBoxNode.setOpacity(0.25f);
        boundingBoxNode.setVisible(false);
        boundingBoxNode.setPosition(new Vector(0,0,0));
        return boundingBoxNode;
    }

Add a class variable for the bounding box node

private Node mBoundingBoxNode;
private void onRenderCreate() {
        ...

        // Create a Node for bounding box
        mBoundingBoxNode = createBoundingBoxNode();
        mBlackPantherNode.addChildNode(mBoundingBoxNode);
       
       ...
    }

Inside the listener, update onAnchorFound()

@Override
public void onAnchorFound(ARAnchor anchor, ARNode arNode) {
	String anchorId = anchor.getAnchorId();
	if (!mImageTarget.getId().equalsIgnoreCase(anchorId)) {
		return;
	}

	Vector anchorPos = anchor.getPosition();
	final Vector pos = new Vector(anchorPos.x, anchorPos.y , anchorPos.z - 5);
	mBlackPantherNode.setPosition(pos);
	mBlackPantherNode.setRotation(anchor.getRotation());
	mBlackPantherModel.setVisible(true);

	//ADDED
	new Handler(Looper.getMainLooper()).postDelayed(new Runnable() {
		@Override
		public void run() {
			BoundingBox boundingBox = mBlackPantherNode.getBoundingBox();
			Log.e("Test", "BoundingBox : " + boundingBox.minX + " , " + boundingBox.maxX);
			Log.e("Test", "BoundingBox : " + boundingBox.minY + " , " + boundingBox.maxY);
			Log.e("Test", "BoundingBox : " + boundingBox.minZ + " , " + boundingBox.maxZ);

			float width = Math.abs(boundingBox.maxX - boundingBox.minX);
			float height = Math.abs(boundingBox.maxY - boundingBox.minY);
			float length = Math.abs(boundingBox.maxZ - boundingBox.minZ);

			Log.e("Test", "Dimension : " + width + " , " + height + " , " + length);

			Box greenBox = new Box(width, height, length);
			Material greenMaterial = new Material();
			greenMaterial.setDiffuseColor(Color.GREEN);
			greenBox.setMaterials(Arrays.asList(greenMaterial));

			mBoundingBoxNode.setGeometry(greenBox);
			mBoundingBoxNode.setVisible(true);
		}
	}, 500);



	mImageTargetFound = true;
	startPantherExperience();
}

And update the black panther model to scale [1,1,1]

ViroActivityAR.zip (ViroActivityAR.java)

[dam00n]

Is this an ARCore issue? Could the issue be that ARCore does not have enough feature points to track the environment and thus when you move up and down like you are, the object shifts?

[dthian]

Interesting, could you share 1 - how you are attaching the black panther model to the scene? 2 - show us any code that toggle's the black panther's visibility (are you hiding it else where)?

How is the bounding box generated? Is it merely based on the 3D object itself?

Currently in Viro, the bounding box for visibility calculations is computed by considering all nodes from the root to the child. Thus, the black panther's head should still be visible as you move up. You should also be able to determine the size of the bounding box at the panther's location, and that the camera's view is still looking at the box or not.

[kelvkong]

This demo is cloned from the ARBlackPanther project. Only the main class "ViroActivityAR.java" has been changed.

ViroActivityAR.java: Full code

For your questions:

1. how you are attaching the black panther model to the scene?

It is as same as the ARBlackPanther code sample.

private void onRenderCreate() {
	// Create the base ARScene
	mScene = new ARScene();

	// Create an ARImageTarget out of the Black Panther poster
	Bitmap blackPantherPoster = getBitmapFromAssets("logo.jpg");
	mImageTarget = new ARImageTarget(blackPantherPoster, ARImageTarget.Orientation.Up, 0.188f);
	mScene.addARImageTarget(mImageTarget);

	// Create a Node containing the Black Panther model
	mBlackPantherNode = initBlackPantherNode();
	mBlackPantherNode.addChildNode(initLightingNode());
	mScene.getRootNode().addChildNode(mBlackPantherNode);

	// Create a Node for bounding box
	mBoundingBoxNode = createBoundingBoxNode();
	mBlackPantherNode.addChildNode(mBoundingBoxNode);

	//
	mViroView.setScene(mScene);
	trackImageNodeTargets();
}

2 - show us any code that toggle's the black panther's visibility (are you hiding it else where)?

Inside the onAnchorFound() of the ARScene's listener.

/*
 Sets up our ARScene.Listener such that when we detect the Black Panther poster, we activate
 the Black Panther model, making it jump out of the poster.
 */
private void trackImageNodeTargets() {

	mScene.setListener(new ARScene.Listener() {
		@Override
		public void onTrackingInitialized() {
			// No-op
		}

		@Override
		public void onAmbientLightUpdate(float lightIntensity, Vector color){
			// No-op
		}

		@Override
		public void onTrackingUpdated(ARScene.TrackingState state, ARScene.TrackingStateReason reason) {
			// No-op
		}

		@Override
		public void onAnchorFound(ARAnchor anchor, ARNode arNode) {
			String anchorId = anchor.getAnchorId();
			if (!mImageTarget.getId().equalsIgnoreCase(anchorId)) {
				return;
			}

			Vector anchorPos = anchor.getPosition();
			final Vector pos = new Vector(anchorPos.x, anchorPos.y , anchorPos.z - 5);
			mBlackPantherNode.setPosition(pos);
			mBlackPantherNode.setRotation(anchor.getRotation());
			mBlackPantherModel.setVisible(true);

			//ADDED
			new Handler(Looper.getMainLooper()).postDelayed(new Runnable() {
				@Override
				public void run() {
					BoundingBox boundingBox = mBlackPantherNode.getBoundingBox();
					Log.e("Test", "BoundingBox : " + boundingBox.minX + " , " + boundingBox.maxX);
					Log.e("Test", "BoundingBox : " + boundingBox.minY + " , " + boundingBox.maxY);
					Log.e("Test", "BoundingBox : " + boundingBox.minZ + " , " + boundingBox.maxZ);

					float width = Math.abs(boundingBox.maxX - boundingBox.minX);
					float height = Math.abs(boundingBox.maxY - boundingBox.minY);
					float length = Math.abs(boundingBox.maxZ - boundingBox.minZ);

					Log.e("Test", "Dimension : " + width + " , " + height + " , " + length);

					Box greenBox = new Box(width, height, length);
					Material greenMaterial = new Material();
					greenMaterial.setDiffuseColor(Color.GREEN);
					greenBox.setMaterials(Arrays.asList(greenMaterial));

					mBoundingBoxNode.setGeometry(greenBox);
					mBoundingBoxNode.setVisible(true);
				}
			}, 500);

			mImageTargetFound = true;
			startPantherExperience();
		}

		@Override
		public void onAnchorUpdated(ARAnchor anchor, ARNode arNode) {
			//No-op
		}

		@Override
		public void onAnchorRemoved(ARAnchor anchor, ARNode arNode) {
			String anchorId = anchor.getAnchorId();
			if (!mImageTarget.getId().equalsIgnoreCase(anchorId)) {
				return;
			}

			mBlackPantherNode.setVisible(false);
			mBoundingBoxNode.setVisible(false);
		}
	});
}

[dthian]

Thanks @kelvkong, interesting, is it possible that onAnchorRemoved is called when your camera tilts upwards and the anchor goes out of view, causing the black panther node to hide?

As such, if you try commenting out the code in onAnchorRemoved, would the black panther still be visible? Much thanks!

[kelvkong]

I have commented out the line mBlackPantherNode.setVisible(false); in onAnchorRemoved but the black panther still disappears.

You may download the project here and have a try.