MindAR

[FrankBevr]

Description

Hola 👋

There is MindAr. MindAr is a threeJS libary for image tracking and other AR Features.

Is it possible to integrate it?

Not too familar with TresJs So far, but maybe with some help.
I can skretch something out to make a tiny AR Expierience.
Does someone with more expierience can take a look and some ideas of how to implent it in a really rough way? ^ ^

Best Regards

Suggested solution

React Three Fiber and mindAR integration

MindAR - Repo

Alternative

No response

Additional context

No response

Validations

• [X] I agree to follow this project's Code of Conduct
• [X] Read the Contributing Guidelines.
• [X] Read the docs.
• [X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.

[FrankBevr]

Added Note: I just saw XR is on the road map for next year. Yeay 🎉

[alvarosabu]

Yes! I definitely hope to work on XR in general and implement it with the library but honestly have no idea how to start with it. Haha Let's discuss it here

My biggest question @FrankBevr, do I need a headset to test it?

[FrankBevr]

No, you dont need a headset, thats the beauty in my opinion🌻

If you have 10 minutes, test it

1. gh repo clone hiukim/mind-ar-js
2. cd mind-ar-js
3. node --version

   v18.18.2

4. npm install
5. npx vite --config vite.config.dev.js
6. Open a Webbrowser, go to http://localhost:5173
7. Click on link, which is named Three
8. accept "allow camera", now you see yourself via your webcam
9. grap your phone and navigate to https://tinyurl.com/mindar-frank
10. Hold your phonescreen against your webcam.
11. 🎉
12. Go to Example HTML from Hiukim
13. Spend 3 minutes of reading/looking
14. Now you know more than before

In General:

The trick with AR is to understand. There is the virtual camera and there is a real camera. You overlay the real camera(webcam) with the virtual camera(ThreeJS Camera). The tricky part is to calucalte the position of the real camera 60 times per second.

Alias "Where is my picture" "There is my picture" "Where is my picture" "There is my picture" "Where is my picture" "There is my picture" 60 times a second. After each frame adjust the virtual camera a bit

More webar libaries:

8th Wall
Zappar
Babylon AR ...

Issue of Intregation

MindAR uses his own thing of ThreeJS Camera.

See this line

The same in blue via react, this line

I don't know yet if thats a actually issue or not, in the react-three-fiber example he just dont use it. See this line , I think he rips the controller from mindAR out and attach it to his own camera somehow, this line But yeah this code is a bit cumbersome to read, especially if i see updateMatrix, yeah sure completly naturally to update Matrix manually 😅🙈

OverAndOut, cheers ✌️

[alvarosabu]

@FrankBevr We started to play around with XR just recently here https://github.com/Tresjs/XR

I see potential as a community package, wdyt?

[alvarosabu]

I'm gonna close due to inactivity. Feel free to reopen if someone wants to work on it. But IHMO should be a community package