jsartoolkit5
jsartoolkit5 copied to clipboard
artoolkitx
Make camera param configurable during runtime
As the camera param file seems to contain vital data regarding input aspect ration and camera properties, it would be great to be able to set/modify it.
To be able to do so, one would need to know how the file is constructed. I digged through the C code, but it didn't really tell me a lot. Is there a human readable documentation of the param file somewhere?
Is there a human readable documentation of the param file somewhere?
No idea :/ If you find something out, maybe add it to the docs?
That's… unfortunate. Maybe someone from the @artoolkit team can help out with some info?
Hello @jensarps.
I hope that you are well.
The camera calibration parameters are the same of the OpenCV approach for the pin hole camera model.
http://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html
The code for the calibration code is found under the util folder of ARToolkit5.
https://github.com/artoolkit/artoolkit5/blob/master/util/calib_camera/calib_camera.cpp
The parameters are created from the calibration patterns in the calib() method.
They are the camera intrinsics, distortion coefficients, sizeX and sizeY.
The parameters are generated in the Calib_Camera but the save function itself is called from saveParam() in
https://github.com/artoolkit/artoolkit5/blob/e53c8afe22d9f9604c9f34e2970b1b1d60b671b2/lib/SRC/AR/paramFile.c
That is how they are created and stored, so they can be modified and set to a value if you want to.
My question is, what circumstances is this a good idea? The calibration process is pretty precise and particular to the given camera/lens that the image is being seen through. What is the desired effect when manually tweaking the values?
Hi @Neuroforge,
thanks for getting back to me – and for providing resources.
The reasoning behind this is that we don't have a reliable, fixed camera input on mobile web. Right now, there are two pre-built param files on the jsartoolkit repo. One seems to be a generic file, the other seems to be for an iPhone 5, environment-facing camera.
There's a lot of fragmentation on Android devices, many different camera hardware types being built into phones. Even on iOS it is close to impossible to determine between an iPhone 5 or 6 if they run the same iOS version. Then, the user can request input streams from both the device's cameras (front/back cam). To make things even worse, the WebRTC API allows the user to ask for different resolutions – no idea what this might mean for the future, browsers might modify the camera stream to meet the user's demands. And, on top of all this mess, the user might decide to feed the image processing a smaller image than they received from the camera to improve processing speed.
So, in the end, we might have a guess on what what hardware we are running, and the only thing we really can be sure about are the dimensions of the input stream/image (and all of this is only known during runtime). That's why it would be desirable to set that during runtime. In an ideal world, we'd had a couple of predefined camera data sets, that originate in calibrating a device's camera, and merge those with the image dimensions that are actually being fed to image processing. That would lead to a situation along the lines of, e.g. "Hey, we probably are on a Nexus 9 here, and we're getting a sizeX/sizeY camera input, which we will cut down to sizeX2/sizeY2, so we'll take what we know of the Nexus9 camera and merge the dimension data in, and this will be the camera params that the processing library should use".
(Uh, sorry for the lengthy reply.)
On the native platforms, we can identify the camera, or if not that, at least the device. Then we query the camera calibration server for the calibration parameters. If we can't get anything comparable via WebRTC, then the best bet might be to just create parameters assuming zero distortion:
ARParam myParam;
arParamClear(&myParam, width, height, 4);
Hi guys I'm interested to set the camera parameters manually; Someone Can tell me where modify the saveparam() to resolve my problem. Sorry for my incompetence. Thanks
@ThorstenBux Any update on this? I'm trying to catch up on the state of things re: jsartoolkit and mobile nft marker reco with reasonable ux (which i think means setting camera params at runtime) and not sure which projects are being worked on etc. I have time to contribute to help make this happen.
@rm8x camera params are independent of NFT. I'm currently working on artoolkitX.js which is based on artoolkitX. It will be possible to load camera para on the api with this version. or at least easy to add. nft is a different topic. OpenCV template match might be a place to start. https://docs.opencv.org/3.4/d8/dd1/tutorial_js_template_matching.html
Hei, this is a very relevant issue.. when building AR software, how is it possible that no one is calibrate cameras, is the error for rendering not enough? Im feeling the experience with AR.js is not that good because of this.
Is there any roadmap on how-to integrate the calibration server on artoolkit?
On building native software for android, some cameras might give us this parameters with a syscall. https://developer.android.com/reference/android/hardware/camera2/CameraCharacteristics
Would love to have a better insight into this issue. thanks. great work :)
