jsartoolkit5
jsartoolkit5 copied to clipboard
kalwalt
Filtering feature
This PR try to implement filtering of transpose matrix with ArFilterTransMat method. Probably with this code the interpolation routine that was implemented to smooth and remove the jittering of the model became un-necessary, i'm refering to this part of the code: https://github.com/kalwalt/jsartoolkit5/blob/244b2b23286403e78fa24805b34509dc5a88052f/examples/nft_improved_worker/threejs_worker.js#L179-L191
with this PR simply became:
} else {
sphere.visible = true;
// set matrix of 'root' by detected 'world' matrix
setMatrix(root.matrix, world);
}
filterCutoffFrequency i set to AR_FILTER_TRANS_MAT_CUTOFF_FREQ_DEFAULT that means has a default values of 5.0, in the nftSimple.c example instead is set to 15.0
Build: build with emscripten Docker (latest)
i added a arLog to retrieve the value of the filterCutoffFrequency and it give me 0 so i think the code must be changed. So default value is 0?
AR_FILTER_TRANS_MAT_SAMPLE_RATE_DEFAULT and AR_FILTER_TRANS_MAT_CUTOFF_FREQ_DEFAULT defined in arFilterTransMat.h:
https://github.com/artoolkitx/artoolkit5/blob/5bf0b671ff16ead527b9b892e6aeb1a2771f97be/include/AR/arFilterTransMat.h#L66-L81
as i wrote in the top comment. Maybe i was receiving that value because i haven't added them to ARBindEM.cpp
I will test it! @kalwalt does it work with threejs_gltf_worker example?
I will test it! @kalwalt does it work with threejs_gltf_worker example?
works with all the nft examples but i removed the interpolation routine (that in the javscript code) in a dedicated example main_threejs_filter_worker.html try this and if you want the others. But i think i still need to fix few things to make some comparisons.
@nicolocarpignoli @ThorstenBux I added a WIP example with gpu.js but i think that there are many variables in this approach. I may implement a kernel for the matrix linear interpolation. Probably in this case that is not very recommended and i will remove it. Another idea is to implement a more efficient routine (linear interpolation) inside the C++ code. Probably will result faster?
@nicolocarpignoli @ThorstenBux i think the version with filtering and linear interpolation in the C++ code is much better, I'd like to know what you think :smile: Testing example: examples/nft_improved_worker/main_threejs_filter_worker.html
@nicolocarpignoli yes with the model animation, seems worste. I need to undersand why. Also fps are very slow on my Mobile device. Tested with a Wiko, View model with Android 7.1.2 and Chrome browser 79.0.3945.136
I will also remoove my gpu.js attempt, i think i will try to optimize the code without it, even if it sound interesting.
@nicolocarpignoli i added another matrixLerp, if you want to test. It seems a bit better.
I think that would be optimal to let the user to build his own version with or without the filtering feature. I would do this feature adding some #define to switch the filtering in the C++ code. So in the build process adding -D WITH_FILTERING it will be compiled with the filtering feature.
Testing example: https://kalwalt.github.io/kalwalt-interactivity-AR/nft/nft_threejs_filter_worker_gltf.html
I found in the BabylonAR repository this 'naive' implementation of the Kalman filter, I should try it, it looks very promising!
@kalwalt https://github.com/kalwalt/jsartoolkit5/pull/36#issuecomment-605684207 === this is a good way to go.
The Kalmar becomes a linear function after running for the first time so it can be even implemented in JS for some few relevant points.
Again, finding the right parameters might change by image and perhaps by device. But that I don't know.
If that the case, finding the right parameters might require a search algorithm (I have done some statistics / DS, so that my reference). But then you need to clarify which functions to optimise. I haven't done that job yet. A target success/failure variable is required.
The search can be automated if the right functions are formulated. The search might be slow and data intensive.
There are few more things that can be done to improve jittering that I have been testing. I will make a video and share the link to show you the results when having time. The target identification (short distance only) and homography were very good using the artoolkit algo for NFT. I will check later which algo is being used, I am really curious now. Previous essays I made with a simpler algos were not as good in detection, especially if the target image was a bit blurred, etc.
I will study some C++ shortly...
@kalwalt https://github.com/kalwalt/jsartoolkit5/pull/36#issuecomment-605684207
Nice reference! Thanks!
https://github.com/kalwalt/jsartoolkit5/pull/36#issuecomment-605693282
Yeah... the same here.
Hi @evaristoc recently i added a combined lerp and filtering (with arFilterTransMat) to my JsartoolkitNFT (dev branch) https://github.com/kalwalt/jsartoolkitNFT/pull/35 see if this is better or not. Also take a look at this discussion in the artoolkitx/jsartoolkit5 issue tracker
@kalwalt still jittering. Specially from some positions of the marker :disappointed:
@kalwalt what is exactly arFilterTransMat? It is almost obvious from the name that it refers to the transformation matrix, but what does exactly do? Do you know?
EDIT: first order low pass filter: https://www.electronics-tutorials.ws/filter/filter_2.html: "the low pass filter only allows low frequency signals from 0Hz to its cut-off frequency, ƒc point to pass while blocking those any higher."
It might be close to what it is done when practicing time series analysis in statistics. It looks like it is assuming a certain frequency (a sinusoidal signal) and implementing a linear model (first order) to clean it. It appears to work well for certain positions, likely because it can capture the signal.
@kalwalt still jittering. Specially from some positions of the marker :disappointed:
Hi @evaristoc have you tested which code?
arFilterTransMat is method coming from Artoolkit http://www.artoolkitx.org/artoolkit5/doc/apiref/arFilterTransMat_h/index.html (reference by ARtoolkitX but it is the same) read:
High frequency noise ("jitter") can be an undesirable property in optical tracking systems like ARToolKit. These functions implement a first-order low-pass filter for both the position and orientation components of the pose estimate transformation matrices.
Probably i didn't applyed correctly,It seems that has no effect ( or very low).
Hi @kalwalt I was editing my comment above: EDIT: first order low pass filter: https://www.electronics-tutorials.ws/filter/filter_2.html: "the low pass filter only allows low frequency signals from 0Hz to its cut-off frequency, ƒc point to pass while blocking those any higher."
It might be close to what it is done when practicing time series analysis in statistics. It looks like it is assuming a certain frequency (a sinusoidal signal) and implementing a linear model (first order) to clean it. It appears to work well for certain positions, likely because it can capture the signal.
Hi @kalwalt I was editing my comment above: EDIT: first order low pass filter: https://www.electronics-tutorials.ws/filter/filter_2.html: "the low pass filter only allows low frequency signals from 0Hz to its cut-off frequency, ƒc point to pass while blocking those any higher."
It might be close to what it is done when practicing time series analysis in statistics. It looks like it is assuming a certain frequency (a sinusoidal signal) and implementing a linear model (first order) to clean it. It appears to work well for certain positions, likely because it can capture the signal.
So maybe you can help, to find what is not correct in my approach. I Think it should filter the noise, but of course it should be done with a correct approach/method.
@kalwalt let's see. I expect to find a roof after which we can not improve the method any more, either because affects on performance or because it will mean to redo some aspects of the project.
Right now we are post-processing it.
But of course I want to help!
I would first advice to create a table or something visible of the inputs and outputs, @kalwalt? You want to smooth the signal, so you might need several passes through different filters.
We should evaluate other filters too, as you have been mentioning in other repos.
@kalwalt will later make a sort of list of all the fronts where you are dealing with this (they are a few).
Let me know where you want to deal with this in particular. Here?
Meanwhile if you can point me to the script where you implemented this particular filter, the low pass one? We can probably create a factory for using not one but different kind of filters.
I suspect filters will have to be implemented case by case. I expect the jittering to change for different target images. I even expect that to change between devices, etc. But let's see, I need to research this more.
You can open an issue in this repository and fill all the data there and link to this PR. I prefere here because much control than in artoolkitx/jsartoolkit5. Eventually if yu are interested we have created a new org https://github.com/webarkit. If you are interested to join us, let me know! :slightly_smiling_face:
the arFilterTransMat is used in ARToolKitJS,cpp the code is wrapped in defines:
#if WITH_FILTERING
// code here
#endif
So you are not obliged to compile it, with filtering.
I will be having a look to different examples made with the same library. Example: http://jevois.org/
And keeping this one for further reading: https://answers.opencv.org/question/182075/convert-pose-from-artoolkit-into-opencv-screen-coordinates/
It seems to be some kind of overlap between the two. I have been doing some opencv recently too.