AR.js
AR.js copied to clipboard
AR-js-org
Weird Wide Angle effect with AR on some phones
Do you want to request a feature or report a bug? BUG
What is the current behavior? On a Samsung Galaxy 10+ phone, the Aframe AR experience, which uses a marker, presents a skewed and unusual look from the camera and does not show the content on the marker. For example, I placed the marker on the seat cushions of a couch, and pointed the phone at it, and the view on the phone's camera shows the top of the back cushion of the couch. When maneuvering the phone oddly enough to locate the marker, nothing appears on the marker.
Furthermore, on the Samsung, it is noticeable that the image is skewed. If you maneuver the phone so that you are looking at someone's face, their face is slightly skewed and stretched.
On a Google Pixel 3, and other phones, everything works properly. The view is normal, and the marker scans to show the content.
If the current behavior is a bug, please provide the steps to reproduce.
-
Initiate a website with a marker-based Aframe AR experience on a Samsung Galaxy 10+ phone.
-
Immediately you should notice a difference in the camera view on the phone.
In this experience, if you scan the marker, a grid image shows up with an overlay UI. Using the overlay UI to pinpoint a cell on the grid, you can change the grid's cell colour to green by tapping on the screen. This is a test example for a game, which is not complete.
Here is the url for the experience: https://unphased.neocities.org/restaurant/test05a.html
And here is the url for the marker image: https://unphased.neocities.org/restaurant/pattern-marker-test01gb.png
Furthermore, here is a picture showing the Samsung Galaxy 10+ issue on the screen vs what you see on a Google Pixel 3: https://unphased.neocities.org/phone-issue-explained01.png
Please mention other relevant information such as the browser version, Operating System and Device Name
This was tested on the Google Chrome browser and the Firefox browser, using WIndows 10. The issue presented itself using both browsers. The devices tested were the Samsung Galazy 10+, which presented the issue, and also on the Google Pixel 3, and the Galaxy A20 which did not present the issue.
What is the expected behavior?
The expected behaviour is that the view seen on the AR website should be identical if not similar to the view one normally sees when taking a picture on the phone.
If this is a feature request, what is motivation or use case for changing the behavior?
This is not a feature request. It is a bug. I expect it would be difficult for anyone with a Samsung Galaxy 10+ phone to be able to experience marker based Aframe AR . Interestingly, a few months ago , someone explained to me online that one of my experiences appeared stretched. Since it was just the one person, I didn't give it much credence at the time as I was distracted by other things I had to do.
EXTRA INFO, as discussed on the AR topic on the Aframe Slack channel:
It was mentioned that the Samsung Galaxy 10+ phone actually has three back cameras. The first has a wide angle lens, the middle has a standard lens, and the third has a telephoto lens. I verified this to be true.
As a result, I expect that the Aframe AR is choosing the wrong back facing camera, and that in this case is it showing a portion, somehow, of the wide angle lensed camera. Why not the whole thing, I do not know.
I hope this helps!
Hi @Whystler probably, but i can not be sure, because i have not that device for testing, the camera choosen by the app cause the issue can you check the width and height of the screen in pixels?
Hello @kalwalt
It happens for sure on these two phones:
Galaxy Note 10+ https://en.wikipedia.org/wiki/Samsung_Galaxy_Note_10
Galaxy S10e: https://en.wikipedia.org/wiki/Samsung_Galaxy_S_series#Samsung_Galaxy_S10
Is that the information you need?
I am encountering the same problem, on my S21 Ultra, it is choosing the wide angle lens and it is warping my AR experience. I will try deep dive this issue.
Could you try to pinch in, as if zooming out?. I had a similar issue in my Samsung S8 and when I pinch in it zooms out and shows the entire image of the camera.
I've ran to this issue too and I found out that the logic behind selecting the main back camera isn't specific enough and some devices give you the wide lens camera. My Samsung has the wide camera listed in the device list before the normal camera and that's why I'm getting the error.
I've found this piece of hacky code by Googling, actually was thinking of doing this abomination myself before I found it.
They have multiple "back" keywords in an array in multiple languages and they filter out video device labels based on those keywords. Then they sort the left devices alphabetically (ex. "camera 0 back" will be before "camera 4 back", this fixes my Samsung issue).
Here's my code, I've edited it directly in my local arjs js file.
const backCameraKeywords = [
"rear",
"back",
"rück",
"arrière",
"trasera",
"trás",
"traseira",
"posteriore",
"后面",
"後面",
"背面",
"后置", // alternative
"後置", // alternative
"背置", // alternative
"задней",
"الخلفية",
"후",
"arka",
"achterzijde",
"หลัง",
"baksidan",
"bagside",
"sau",
"bak",
"tylny",
"takakamera",
"belakang",
"אחורית",
"πίσω",
"spate",
"hátsó",
"zadní",
"darrere",
"zadná",
"задня",
"stražnja",
"belakang",
"बैक"
];
// get available devices
navigator.mediaDevices.enumerateDevices().then(function(devices) {
const mainBackCamera = devices
.filter(x => x.kind === "videoinput")
.filter(camera => {
const lowercaseLabel = camera.label.toLowerCase();
return backCameraKeywords.some(keyword => lowercaseLabel.includes(keyword));
})
.sort((camera1, camera2) => camera1.label.localeCompare(camera2.label))[0];
var userMediaConstraints = {
audio: false,
video: {
facingMode: 'environment',
focusMode: "continuous",
width: {
ideal: _this.parameters.sourceWidth,
// min: 1024,
// max: 1920
},
height: {
ideal: _this.parameters.sourceHeight,
// min: 776,
// max: 1080
}
}
};
const deviceId = _this.parameters.deviceId || (mainBackCamera ? mainBackCamera.deviceId : null);
if (deviceId) {
userMediaConstraints.video.deviceId = {
exact: deviceId
};
}
I couldn't get userMediaConstraints specific enough for the normal lens to trigger, so I've used the hack to possibly get the main back camera. If that doesn't yield any results, then I just use the old existing logic. CTRL+F for navigator.mediaDevices.enumerateDevices() in your arjs file to find the section with the old logic.
Hello @kalwalt
It happens for sure on these two phones:
Galaxy Note 10+ https://en.wikipedia.org/wiki/Samsung_Galaxy_Note_10
Galaxy S10e: https://en.wikipedia.org/wiki/Samsung_Galaxy_S_series#Samsung_Galaxy_S10
Is that the information you need?
It also occurs for the Samsung Galaxy A51 https://en.wikipedia.org/wiki/Samsung_Galaxy_A51
I've ran to this issue too and I found out that the logic behind selecting the main back camera isn't specific enough and some devices give you the wide lens camera. My Samsung has the wide camera listed in the device list before the normal camera and that's why I'm getting the error.
I've found this piece of hacky code by Googling, actually was thinking of doing this abomination myself before I found it.
They have multiple "back" keywords in an array in multiple languages and they filter out video device labels based on those keywords. Then they sort the left devices alphabetically (ex. "camera 0 back" will be before "camera 4 back", this fixes my Samsung issue).
Here's my code, I've edited it directly in my local arjs js file.
const backCameraKeywords = [ "rear", "back", "rück", "arrière", "trasera", "trás", "traseira", "posteriore", "后面", "後面", "背面", "后置", // alternative "後置", // alternative "背置", // alternative "задней", "الخلفية", "후", "arka", "achterzijde", "หลัง", "baksidan", "bagside", "sau", "bak", "tylny", "takakamera", "belakang", "אחורית", "πίσω", "spate", "hátsó", "zadní", "darrere", "zadná", "задня", "stražnja", "belakang", "बैक" ]; // get available devices navigator.mediaDevices.enumerateDevices().then(function(devices) { const mainBackCamera = devices .filter(x => x.kind === "videoinput") .filter(camera => { const lowercaseLabel = camera.label.toLowerCase(); return backCameraKeywords.some(keyword => lowercaseLabel.includes(keyword)); }) .sort((camera1, camera2) => camera1.label.localeCompare(camera2.label))[0]; var userMediaConstraints = { audio: false, video: { facingMode: 'environment', focusMode: "continuous", width: { ideal: _this.parameters.sourceWidth, // min: 1024, // max: 1920 }, height: { ideal: _this.parameters.sourceHeight, // min: 776, // max: 1080 } } }; const deviceId = _this.parameters.deviceId || (mainBackCamera ? mainBackCamera.deviceId : null); if (deviceId) { userMediaConstraints.video.deviceId = { exact: deviceId }; }I couldn't get userMediaConstraints specific enough for the normal lens to trigger, so I've used the hack to possibly get the main back camera. If that doesn't yield any results, then I just use the old existing logic. CTRL+F for
navigator.mediaDevices.enumerateDevices()in your arjs file to find the section with the old logic.
Hey, Bro, thanks for this smart solution. It worked for me. However, I'm facing another issue related to the focus distance. It seems that the focus is a bit zoomed in. Is there any way to control the focus distance?
Try to remove focusMode constrain from my code, I might have left it there by accident when testing. Or set it to 'none'.
Check this link out, there is info about two focus related constraints and their all possible values: focusMode and focusDistance. Yoink.
I didnt have this issue, so please check yourself... Bro 😉 Its your best bet, but it might not work, the camera API is crap (on purpose for privacy reasons, but still... ).
(This is a delete and redraft)
@ArielJurkowski What file from the github repo do you run as your "local arjs js file"?
@MarkEEaton
I needed a quick and dirty solution, so I just downloaded and edited the final js file directly. I used the 3.3.3 version of aframe-ar-nft.js (here). It seems the new master version has the code with navigator.mediaDevices.enumerateDevices() minified. That wasn't the case with version 3.3.3, but I think my solution should still work if you edit the js file properly.
@ArielJurkowski thanks for this! I got it working locally. I had to downgrade aframe to version 1.2.0. I will test it on some more devices next week.
Thank u very much, this made it for me. At least in the device I'm testing. Had to downgrade to 1.2.0 as well. I will test on more devices and share feedback.
