videoTexture: true not working for distant locations

[lemoun]

Hi - I'm looking for a solution to see a location at any distance, even very far away from the device location. More precisely, my use case requires the augmented component to keep the same size on the user screen, the target location being near the user, or very far away. I have searched in the documentation a way to disable the auto-scaling behaviour, but can't find anything. As I understand it, by keeping the default value of maxDistance=0 property of the a-camera component, and by setting videoTexture: true in the a-scene, we should be able to see any distant object. But augmented components keep changing sizes with distance, and far away locations are barely visible. It would require a complex logic to manually scale them to keep the same size.

[nickw1]

Hi @lemoun you would need another type of camera, something other than the perspective camera (based on THREE.PerspectiveCamera) which A-Frame uses by default. The perspective camera is responsible for making nearby objects appear larger than far-away objects. If you look at the A-Frame and three.js documentation on different camera types, you might be able to find something which fits your use-case.

[lemoun]

Thank you @nickw1 for your help. I have searched for the camera type, and it looks like an OrthographicCamera is being initialized with the AFRAME component 'arjs-webcam-texture'. My a-scene property looks like this: arjs='sourceType: webcam; videoTexture: true; debugUIEnabled: false;. What am I missing here ?

[lemoun]

A workaround is to scale the object in the _updatePosition function:

const distanceAbsolute = Math.abs(position.z));
const scaleValue = distanceAbsolute / 10;
this.el.object3D.scale.set(scaleValue, scaleValue, scaleValue);

A cleaner solution would be to define a fixed size object in arjs-webcam-texture component. It would be a great feature to add a parameter fixing the size of the object. There seem to be many use cases where a user wants to instantly see places that are 100 metres away, and others that are 3 or 4 kilometers away.

[nickw1]

@lemoun ok thanks for that - will try and add if I have a moment (might be a little while though, as am quite busy in the immediate future). Alternatively you could use an OrthographicCamera on the scene instead of a PerspectiveCamera, which will allow you to have a scene without perspective.

(Note the OrthographicCamera you found is not for the actual scene, but for rendering the video texture, which needs to be flat and unprojected).

[planetMatrix]

@lemoun I have the same issue as you.

@nickw1 @lemoun Please share where to add OrthographicCamera

many thanks

[IvoPereira]

Hi there!

I wonder if there was any advance on this.

Would you happen to had a time to mess with this @nickw1?

Any advances on implementing this @planetMatrix and @lemoun?

[nickw1]

@IvoPereira I am close to submitting a PR for a revised A-Frame component which hopes to solve some other issues. Once this has been merged I will try and look at this, time permitting.

[IvoPereira]

All good @nickw1, thank you!

Meanwhile I am researching what would be the best way to approach this situation as I haven't been able to have any advance on this.

From the research I've made it looks like there is no way to explicitly use a OrthographicCamera instead of a PerspectiveCamera.

As a mere library user, I was not aware of some of the AR.js and AFrame internals but I've been looking into them in case it might help somebody (or myself later) with some research:

• Where "camera" is set on arjs: https://github.com/AR-js-org/AR.js/blob/66733164a92030bb0453ce3eb7e1dcbd4f12a6f8/aframe/src/system-arjs.js
• How the PerspectiveCamera works: https://observablehq.com/@grantcuster/understanding-scale-and-the-three-js-perspective-camera
• Where a OrtographicCamera is initiatlized when using arjs-webcam-texture: https://github.com/AR-js-org/AR.js/blob/91e73f483b0dd3a7c3a62d773722f7850b37ee1e/aframe/build/aframe-ar-location-only.js#L649
• OrtographicCamera Three.js docs: https://threejs.org/docs/#api/en/cameras/OrthographicCamera
• Example of showing distant places with arjs-webcam-texture BUT without keeping a fixed size (somewhat confused with this one, as if arjs-webcam-texture spawns a OrthographicCamera that supposedly keeps every element with the same size, then how can it not be keeping it here?): https://github.com/AR-js-org/AR.js/blob/master/aframe/examples/location-based/show-arbitrary-distant-places/index.html
• AFrame "a-camera" docs: https://aframe.io/docs/1.3.0/components/camera.html
• "gps-camera": https://github.com/AR-js-org/AR.js/blob/66733164a92030bb0453ce3eb7e1dcbd4f12a6f8/aframe/src/location-based/gps-camera.js
• gps-entity-place source code (one of my goals is to make OrthographicCamera to work with gps-entity-place entities - they seem to look for an element with a gps-camera attribute. Was thinking perhaps we might register a ortho-camera and add the gps-camera to it): https://github.com/AR-js-org/AR.js/blob/master/aframe/src/location-based/gps-entity-place.js
• Some suggestions on how to add a OrthographicCamera to AFrame: https://github.com/aframevr/aframe/issues/3018

Take all this with a grain of salt as I am not at all experienced developing the library internals.

To complement, this is a visual view of what I think is the current behavior vs what we are looking to achieve in this issue (at least in the way I see it).

Current behavior - as long as we move in Z axis, we have a perspective view of our position regarding the object we are looking at

What we expect - as long as we move in Z axis, we keep a predefined distance to the object we are looking at (or perhaps we might think about the chance of possibly adding a factor of how much the scale based on Z)

One of the things I wasn't really able to find (and perhaps that could help trying to implement the new Camera) is exactly where is the PerspectiveCamera initialized in AR.js. Perhaps you might know out of the top of your head where exactly that does happen and could point me out @nickw1?

[lemoun]

Someone shared another work around for this problem on Gitter some time ago. I didn't tried his solution, I'm sticking to the one I proposed earlier for the time being, which is basically what you proposed @IvoPereira, to scale the object relative to the Z value.

[IvoPereira]

Hey there @lemoun. Thank you for following up on this!

The gitter seems like a related issue but not this exact one (I think?). In that it looks like camera FOV does not maintain aspect ratio.

In the case I was considering (and I thought you were in the first post as well) we basically want to keep the same entity scale besides the Z distance from it.

I've tried to use your snippet before but I never came up with a good-enough working solution with it. Would you eventually be able to provide the full component or a more explicit version of this code in your usage?

[nickw1]

@IvoPereira @lemoun sorry will try and look at this in the next few days, I've had little time recently.

[lemoun]

The Gitter solution seems related, by making far away places to be displayed, but I haven't tried his solution so I may be mistaking. I've put the code inside the gps-entity-place AFRAME component. Look at 'autoscale 3d objects' comment:

AFRAME.registerComponent('gps-entity-place', {
      _cameraGps: null,
      schema: {
          longitude: {
              type: 'number',
              default: 0,
          },
          latitude: {
              type: 'number',
              default: 0,
          }
      },
      remove: function() {
          // cleaning listeners when the entity is removed from the DOM
          window.removeEventListener('gps-camera-origin-coord-set', this.coordSetListener);
          window.removeEventListener('gps-camera-update-position', this.updatePositionListener);
      },
      init: function() {
          this.coordSetListener = () => {
              if (!this._cameraGps) {
                  var camera = document.querySelector('[gps-camera]');
                  if (!camera.components['gps-camera']) {
                      console.error('gps-camera not initialized')
                      return;
                  }
                  this._cameraGps = camera.components['gps-camera'];
              }
              this._updatePosition();
          };
  
          this.updatePositionListener = (ev) => {
              if (!this.data || !this._cameraGps) {
                  return;
              }
  
              var dstCoords = {
                  longitude: this.data.longitude,
                  latitude: this.data.latitude,
              };
  
              // it's actually a 'distance place', but we don't call it with last param, because we want to retrieve distance even if it's < minDistance property
              var distanceForMsg = this._cameraGps.computeDistanceMeters(ev.detail.position, dstCoords);
  
              this.el.setAttribute('distance', distanceForMsg);
              this.el.dispatchEvent(new CustomEvent('gps-entity-place-update-positon', { detail: { distance: distanceForMsg } }));
  
              var actualDistance = this._cameraGps.computeDistanceMeters(ev.detail.position, dstCoords, true);
 
              // autoscale 3D object to see a distant location. Works well when the location is < 10km 
              routeDistance.setAttribute('value', `${Math.round(distanceForMsg)} meters`);
              const scaleFactor = distanceForMsg / 10;
              this.el.object3D.scale.set(scaleFactor, scaleFactor, scaleFactor);
              
              if (actualDistance === Number.MAX_SAFE_INTEGER) {
                  this.hideForMinDistance(this.el, true);
              } else {
                  this.hideForMinDistance(this.el, false);
              }
          };
  
          window.addEventListener('gps-camera-origin-coord-set', this.coordSetListener);
          window.addEventListener('gps-camera-update-position', this.updatePositionListener);
  
          this._positionXDebug = 0;
  
          window.dispatchEvent(new CustomEvent('gps-entity-place-added', { detail: { component: this.el } }));
      },
      /**
       * Hide entity according to minDistance property
       * @returns {void}
       */
      hideForMinDistance: function(el, hideEntity) {
          if (hideEntity) {
              el.setAttribute('visible', 'false');
          } else {
              el.setAttribute('visible', 'true');
          }
      },
      /**
       * Update place position
       * @returns {void}
       */
      _updatePosition: function() {
          var position = { x: 0, y: this.el.getAttribute('position').y || 0, z: 0 }
  
          // update position.x
          var dstCoords = {
              longitude: this.data.longitude,
              latitude: this._cameraGps.originCoords.latitude,
          };
  
          position.x = this._cameraGps.computeDistanceMeters(this._cameraGps.originCoords, dstCoords);
  
          this._positionXDebug = position.x;
  
          position.x *= this.data.longitude > this._cameraGps.originCoords.longitude ? 1 : -1;
  
          // update position.z
          var dstCoords = {
              longitude: this._cameraGps.originCoords.longitude,
              latitude: this.data.latitude,
          };
  
          position.z = this._cameraGps.computeDistanceMeters(this._cameraGps.originCoords, dstCoords);

          position.z *= this.data.latitude > this._cameraGps.originCoords.latitude ? -1 : 1;
  
          if (position.y !== 0) {
              var altitude = this._cameraGps.originCoords.altitude !== undefined ? this._cameraGps.originCoords.altitude : 0;
              position.y = position.y - altitude;
          }
          
          // update element's position in 3D world
          this.el.setAttribute('position', position);
          
          // autoscale 3D object to see a distant location. Works well when the location is < 10km 
          const newDstCoords = {
            longitude: this.data.longitude,
            latitude: this.data.latitude,
          };

          const distanceMeters = this._cameraGps.computeDistanceMeters(this._cameraGps.originCoords, newDstCoords);
          routeDistance.setAttribute('value', `${Math.round(distanceMeters)} meters`);
          const scaleFactor = distanceMeters / 10;
          this.el.object3D.scale.set(scaleFactor, scaleFactor, scaleFactor);
      },
  });

[nickw1]

Sorry for the delay on this, unfortunately I've had little time recently, but just a couple of things. I think the solution of @lemoun involving calculating the scaling should work well.

Maybe OrthographicCamera is not what you are after, as it is intended for display of 'flat' objects on a 3D scene, I think what you're after is objects which have a z coordinate (so change their apparent position in the 3D world) but constant width and height?

In case you do want an OrthographicCamera, you should be able to set the A-Frame scene's camera by setting its camera property e.g.

this.el.sceneEl.camera = new THREE.OrthographicCamera(....);

Hope that helps.