jsartoolkit5
jsartoolkit5 copied to clipboard
artoolkitx
Can't load multiple NFT markers
I'm trying to load multiple NFT markers using artoolkit.addNFTMarker(markerURI, (id) => { /*...*/}) and its throwing an exception (with a number that is different every time it runs) as soon as the second image set starts to load. I tried recompiling jsartoolkit to enable exception catching, but I didn't help a lot.
Did anyone had success with this? I'm using a slightly modified version of one of the Misdake's examples (with jsartoolkit code running on a Worker)
Now I'm just trying to load two, but maybe 5 or 6 in the future. I found that if I create another ARController I can load one marker in each controller. If wonder if it is meant to be used that way? Or should I be able to load multiple markers in the same ARController object?
@nicolocarpignoli had played with it. It is certainly not meant to be by loading ARController multiple times. But I thought @nicolocarpignoli had success with 2.
Hi @ianmethyst thanks for reporting this. What does it meams Uncaught in the log error? Can you post the entire log in specific the Uncaught part? I think that could be a memory issue....
I was able to make it work using AR.js version. Honestly we have to check on that and see differences. I'm sure we can make it work with lean jsartoolkit5 (it should be easier, actually).
I'll upload the complete log with an example project to replicate the error later. I'm not at home right now
Sorry for the delay. Here is the log (which is way too big compared to what is shown in the Chrome DevTools), and a zip with a git repo that contains the example I used when I found this problem. I removed all the files from the jsartoolkit repo that are not related (but kept the directory structure) and added a npm script so you can do npm install && npm run serve to launch a server with the example.
With git diff 417b1ff HEAD js/artoolkit.worker.js you can check the changes introduced that make this error happen. I just copied and pasted the block that initializes the NFT marker, but this time I used the same marker twice (I tried using different markers but got the same result).
localhost-1583890591748.log nft-error.zip
Is there any way I can help with this? I believe there is missing some documentation and some guidance on how to contribute, because I gladly help but I find myself quite lost when I try to navigate the repo. For instance, I don't fully understand what is built with emscripten and what is plain javascript that ends in the final code
We highly appreciate any help. So yes you can certainly help. What is your experience what would you like to contribute?
Sorry @ianmethyst @ThorstenBux i haven't had the time to look Into this. I will try today.
@ianmethyst i will provide an example soon with a different approach, though i would investigate deeply on that error in a near future.
I tested your code @ianmethyst rebuilding the libs with -s ALLOW_MEMORY_GROWTH=1 and i get again that error. We should try if this error also happens outside the WebWorker and understand well why this happens.
We highly appreciate any help. So yes you can certainly help. What is your experience what would you like to contribute?
I have some experience with JavaScript. I use it for frontend and backend (with Node.js) development. I've also been using three.js for a while.
I'd like to start helping fix this issue as it is something we are aware of, but I'd mostly like to contribute with the documentation, because I believe it's the most important part of any FOSS software. I'm gonna open another issue for that, though.
I tested your code @ianmethyst rebuilding the libs with
-s ALLOW_MEMORY_GROWTH=1and i get again that error. We should try if this error also happens outside the WebWorker and understand well why this happens.
Alright, thanks for trying that @kalwalt. I'll remove the Worker from the repo from the example I uploaded and see if it stills throws the exception
@ianmethyst i also tested without the WebWorker and happens the same error. I think we should create a multi handler for Multi NFT, starting from the C++ code. We will go in detail in a dedicated issue.
I'd like to start helping fix this issue as it is something we are aware of, but I'd mostly like to contribute with the documentation, because I believe it's the most important part of any FOSS software. I'm gonna open other issue for that, though.
yes this is fundamental. I improved a bit the docs inside the api but should be enhanced. Actually you can create the docs running npm run create-doc read in the readme.md.
any update on this? or any workaround? thank you for your time, im using artoolkit.min.js and I have same exception. Seem to be blocking in the second call to :
var id = Module._addNFTMarker(arId, prefix); if (callback) callback(id);.
First marker being added succesfully.
@ThorstenBux ThorstenBux @ianmethyst nicolocarpignoli any update? Thank you for your time.
@Memleak1 No any updates from this point, i was focused to solve other part of the code. To handle correctly and efficently a series of NFT markers is required some changes in the C++ code, I think i know how to do but never had the time to implement. :slightly_smiling_face:
Sorry, I was really busy lately and couldn't look into this this before. I just checked the code in the C++ ARToolKit5 repo and found a few things. I don't know C++, so correct me if I'm wrong, but the ARMarkerNFT class inherits from ARMarker that has a load() method and a ARPattern array (patterns). Then, in the .cpp file of ARMarkerNFT, for what I can see, load() is implemented so that the pattern passed as an argument is saved into the first index of the patterns array instead of being loaded into the next index. It's just a guess, but I believe that could be the cause of the exception being thrown.
I'll leave the links to the relevant files: ARPattern.h ARPattern.cpp ARMarker.h ARMarker.cpp ARMarkerNFT.h ARMarkerNFT.cpp
@ianmethyst you looked in the right place, we should implement a sort of iterator as it exist in ARMarkerNFT, but without the threaded part of course.
@kalwalt @ianmethyst, thank you for coming back to me, I tried to run the code to take a look and seem like I cannot build it using VS 2017. Did you manage to make it run? Seems I have issues with the KPM library
@Memleak1 You can use (and you should, to avoid unexpected errors due to OS differences) Docker to build jsartoolkit. It takes the C++ and turns it into JS using Emscripten. If you're trying to build C++ artoolkit5, then you can check this section of its readme.
@ianmethyst yes thats what we do to create an js version, but are you suggesting to make the change on the c++ side and test on the js side? I didnt have any luck building the C++ vs project so far 2017 & 2013.
@Memleak1 please open another issue and post some logs, with detailed informations as much as you can. You can file an issue here or at my repository issue tracker Thank you :slightly_smiling_face:
I can confirm this error. I have been digging through the codebase including the underlying artoolkit5 the past couple of days. It was a crazy journey :)
You cannot call addNFTMarker multiple times on a single AR controller. Once the marker dataset is set, calling this again will throw error. I believe there are a couple of logical flaws in the code. I wonder does anyone able to run multiple markers before.
I fixed a couple of things and finally managed to get something working. If you want, you can use my build here: https://github.com/hiukim/jsartoolkit5/tree/fixnft/build You still cannot call loadNFTMarker multiple times here, but I have extended a new method ar.loadNFTMarkers that accept multiple urls.
ar.loadNFTMarkers([markerURI_1, markerURI_2], function (markerIds) {
// console.log(markerIds[0], markerIds[1]);
});
If you want to build yourself, you need to use this artoolkit5 branch https://github.com/hiukim/artoolkit5/tree/nftfix There is a small fix here as well.
@hiukim Nice! What about performances? How many NFT markers is possible to load with your mod? and why do you modify the artoolkit5 source code? Too many questions but i will try your code.... :smile:
@hiukim amazing! how did you manage to build your visual studio project? I'm having a hard time building it. Regarding the multimarker, seem like so people managed to get it working in A-Frame, I didnt look yet a their solution. I wonder what they made differently... https://github.com/kalwalt/jsartoolkit5/issues/40
@kalwalt @Memleak1 Just to make sure everyone's on the same page and save some time for all the fellows who come along next.
The whole ar process consists of two parts: The detection part and the tracking part. The detection part is trying to find a first match in the loaded markers. After first match, the tracking part comes in and keep track of the match. There are actually two different algorithms behind.
The way it is (or supposed to be) working now is that: At the beginner, ONLY the detection part is running. It try to find a match in all the markers loaded (e.g. through the addNFTMarker. When there is a match, the detection part will stop, and the tracking part comes in. Now the process only keep track of the matched marker, and will not try to detect other markers. When the tracking is lost, the ball is passed back to the detection.
The ar controller as is now cannot track multiple markers. It is supposed be able to accept loading multiple markers for detection though. My fix is for the second part. It's important to point out that all I'm saying is for one AR controller. That's the behaviour of an AR controller. But you can spawn multiple AR controller to track multiple markers.
I hope I explained that clear enough. Now I can come back to the questions :)
Regarding performance, my changes won't hurt the performance because in the tracking part, it is still only tracking one marker. The performance of the detection part will depend on the number of loaded markers and their resolutions.
Regarding why I need to change the artoolkit5 source code. because I think I figured a bug there. If there is anyone here familiar with the source code, it would be great to if he/she can also take a look. I mean, I'm not totally confident too. If anyone interested in, it's good to have another eye to double check. The changes is here: https://github.com/hiukim/artoolkit5/commit/ac5958043f2726b52c409da895535ecd839c5b6d
Regarding tracking multiple markers in another project of aframe. I have also dig into that too. If I understand the code correctly, each anchor spawns their own web workers, and each worker itself spawns an an AR controller. In that case, I'm not surprised if there is a working example that can "track" multiple markers using multiple AR controllers. I'm curious, however, is anyone able to load multiple markers in one AR controller though.
@Memleak1 Regarding building the project. I'm not sure what you mean by building visual studio project. I just follow the README and build with docker.
Hi everybody, I just wanted to add that on AR.js (but should be the same here on jsartoolkitx) I was able to recognise multiple NFT Markers. I have pushed now the branch. There are two images to scan: pinball image and alterra.
Branch and direct link to the example is this one:
https://github.com/AR-js-org/AR.js/blob/multiple-nft-markers/aframe/examples/image-tracking/nft/multi.html
To try it out, you should clone the repo, switch on the branch 'multiple-nft-markers' and run the html file above under an https server.
I have not checked this example in a while, so don't know much about performances. I'm sure it works though. Let me know
I can confirm what you said about thé algorithme @hiukim. If your changes to artoolkit5 are necessary maybe @ThorstenBux may integrate them on the main repository. But it should be tested for all the platforms (Linux, Windows, Mac..), maybe Will be better to create a emscripten branch, and enclose that code between defines. So to prevent for other platform to be compiled. But mine Is only a supposition, i have only took a quick look to your code... @nicolocarpignoli yes i remember that code...🙂