PoGo-UWP
PoGo-UWP copied to clipboard
ST-Apps
(Question) My progress with AR
Hey ST-Apps and crew!
Since starting work on AR, I have completed the framework that displays the camera feed and renders on top of it using DirectX. You can currently switch to AR mode and view a test object. What remains is hooking into orientation and the catchable pokemon array.
I can make a pull request so you guys can verify things are going the way you prefer them to go, or I can wait a few more days until its all done. What do you prefer? ( Its my first collaboration :p )
Hey, just one question. We have 3D pokemon models and want to place them in the Pokemon view, do you how to do that?? Nice job!
@Kaankehh I know how to do that, and it's a massive job that I won't be doing right now. Version 1 of the AR code will probably be super rough but it will be something we can iterate over and improve. Also maybe you can add those models to the VS project, as non-deployable assets for now so others can look at them.
@MadoxLabs I wanted to stream the pokemon models, the app is getting too big to put it in the library. I can give you a link if you want so you can take a look
@Kaankehh Sure. I made the assumption that its rigged models that you want to see animated. If its just static models, thats a lot easier. :p
@MadoxLabs No actually a static model is fine now haha, if you can turn it in the Pokemon view it is enough for now, animation can be added later on. http://www.models-resource.com/3ds/pokemonxy/ These are like exactly the same ones as in Pokemon GO. I suck in 3D modeling xD dont know how you do that haha
@Kaankehh I played around with the bulbasaur model. The FBX file is an old version that Blender doesn't handle. I converted it up to the 2013 FBX format, and it loads but the UVs need a lot of tweaking to fix up the textures. The DAE files it comes with are in even worse shape for textures.
If there is someone that is more knowledgeable with these models, FBX and/or Blender/3DS/Maya, please jump in here :)
Woah, this is great! Does your code allows to render 3D models even on top of XAML? Because we may even try adding 3D models to the Whole game!
I'm leaving for two days off, I'll get to you when I'll be back.
@ST-Apps Ya you could render on top of XAML. The way it works is that I have a rendering class that is able to act as a source for an image element, and I add a topmost image element that covers the whole screen. The scene gets rendered onto a transparent background. That screenshot shows the object at 50% transparency because i was messing around. It normally appears solid.
I'm using SharpDX which is a C++ interop wrapper. (first time using it :p ) Currently this code does the bare minimum to do a simple render. If you want to do 3d everywhere, we probably need more robust code, I have my own rendering engine that started out as C++/DirectX and I have ported to C#/XNA and JS/WebGL but since XNA is dead, porting it to SharpDX would take a while.
@MadoxLabs Admirable job you did there ! We all apriciate the work you do and everyone understands it will obv. tak a while. So are you saying that with your code, our UI guys can do 3D everywhere (or will be able to) or have I missinterpreted it ?
You could do 3D everywhere - You have to add that topmost fullscreen image to each view which will use the rendering class as a source. Maybe we could have a single image that sits on top of every view, and views change underneath it, but it sounds like too much shenanigans to me :p
@MadoxLabs I have some experience in Cinema4D so I can help you with the 3D stuff ^^ I can also do the animations, but I don't have any clue about the programming side, so I don't know what exactly you will need ;-) I also got some texture issues but i could fix them:
Awesome! At this point I dont know what SharpDX needs either. If it still uses the X file format for models, we'll need a converter. Hopefully we can just use FBX. Is it possible to bake all the textures into a single texture?
I'm also having trouble with the geo positioning - its not as fine grained as I was hoping, and it seems to go through spurts of not getting updates. I have it updating every 500ms which is hopefully good enough, but if you are walking fast, it will appear choppy.
I have no idea if this will murder your data, but the app currently updates the map so it must do the same thing.
Edit to add: @Lithagon I have an FBX parser/importer that i wrote for WebGL. http://madoxlabs.github.io/WebGL/mxPipeline/
If you can check that the models render in that tool, we can probably modify it for SharpDX. Instructions are on there - drop a file on the Importer tab, drop textures on the texture box, go to Viewer tab to see the render.
@MadoxLabs I think that it will murder your battery faster than your phone xD
But it is nice that you 2 know about this D stuff, now we can give Niantic the real "FUCK YOUUU " feeling! xD
@MadoxLabs I can export .x files, but i never used, maybe you can try using this one: https://dl.dropboxusercontent.com/u/1267589/Pokemon/Bulbasaur.x
@Lithagon I have no way to test X files right now without a ton of coding :p I'll get to it though
@MadoxLabs Hello, i'm a 3D modeller and i previously did this Pokemon trainer
i tried to import the fbx in your link, but can't seem to make it work from maya 2016
My tool only works with 2013 FBX formats - 7.3 ascii and 7.4 binary to be exact. (It says it on the page :p ) Maybe the 2016 format isnt compatible? Sweet model!
@MadoxLabs When you export in FBX from Maya you can choose Ascii or Binary and FBX year; so I exported in Ascii 2013 fbx, but nothing. There are no other options about the format
That's odd. I was using Maya2013 when I wrote that and it was ok. Do you want to send me a test file to debug with? [email protected]
It could be that, as a pro modeler, you have goofy stuff going on that I wasnt aware of so teh parser chokes on it. :p
Hey - the tool throws an error when I load that FBX saying "mesh is not a triangle list". It does end up with something that renders, but it obviously messed up.
Blender doesnt do ascii FBX so I used Autodesk FBX converter to convert it to Binary. In doing so, it pulled out a texture file - so I guess you had a texture baked in there? After that, I could load it in Blender to triangulate the faces. Loading the mesh and texture in my tool afterwards resulted in:
So it works! It just needs triangle faces and separate textures.
ooh indeed i didn't triangulate when i exported and left embedded texture on.
Good thing it works!