xviz
xviz copied to clipboard
aurora-opensource
XVIZ live conversion + visualization
Hello, I want to generate .glb files from sensor data and directly send it to Uberviewer to visualize data in real time. I can generate .glb files but I need some ideas to visualize this data with a frequency. If it possible, what would be the steps to achieve this?
Here's what I've done and found for live mode and the results:
- We do need 0-frame.json for timestamps (why the hell we need timestamp in live mode?)
- We also need a 1-frame.glb file which contains meta data information, which makes sense.
- I edited streetscape.gl/examples/get-started/src/log-from-live.js file and added "session_type: 'live'"
- Also starting server command changes as: "./modules/server/bin/babel-xvizserver -d data/generated/kitti/2011_09_26/2011_09_26_drive_0005_sync --port 8081 format = JSON_STRING --live"
- From the side I see, converting some data to xviz format takes considerable time in js, which causes to latency for live mode.. How people solved this issue when they want to play their data in live mode? Calling js for every frame is very expensive.
I'd appreciate if you provide an example or specification for live mode operations. Thanks for your interest.
It is a fair to say that a live mode has partial support, as you have experienced.
To address your bullet points
- This is simply due to the current way the Providers and components work. Since live mode in the server isn't fully supported this is the current situation. Your choice of language helps emphasize your frustration, but does little else. The reality is live is not fully supported and does need improved. This could be done in many ways. An alternate provider could be created, or a flag could be provided to ignore the index.
- Actually, we are moving to a more flexible metadata in the live case because you may not know ahead of time what streams are available. We are looking to make metadata messages valid at any time but have not finalized it yet.
- Nice :) The server then needs to respect this and change behavior accordingly.
- Javascript is not a good choice for production work, we know this. Ideally it would be in C++ but for various reasons we have chosen to make XVIZ available with the Javascript examples and specification first, and over time add additional languages.
In general, a live mode is extremely useful, but large amounts of data exists off-line that are easier to work with and equally valuable. Priorities and resources is the name of the game.
Let me see if I can help remove some complexity for you.
You are working with the existing components, which is great build an understanding of the system, but to make it work for you I expect you to replace components that don't fit. The layers try to isolate a responsibility. XVIZSource/XVIXSink operate on a named data source, it could be from file or something else but assumes some structure w/o saying memory or file or socket or db.
The next layer is the XVIZReader/Writer. These conform to the structure you mention of index, metadata, data (0-frame.json, 1-frame.json, {2,*}-frame.??? - respectively). You could create a component that returned some minimally viable data where it is not relevant to you instance.
The next layer is the XVIZProvider, which is very concrete about Index, Metadata, Message and does not care if the underlying interface follows the convention (0-frame.json, etc) since it relies on the XVIZReader.
What this means is you could create an XVIZProvider and XVIZReader that was entirely independent of the existance of an index or metadata (just send valid but largely ignored values as necessary).
I would love to get XVIZ to a level of maturity where what you want came out of the box ready to use, but we are building to that. In the mean time we are trying to provide appropriate levels of abstraction such that most elements can be worked around while still leveraging other layers provided.
Hope this helps give you some ideas. Ping me on slack or post a PR and I can provided assistence.
Hello Eugin, I followed same the steps you mentioned. but when I run yarn start-live-local, The UI comes up but says Missing Data:
How were you able to visualize .glb files in real time? Right now latency is not matter of concern for me.
Anyways., I am able to visualize now. But the .glb files are generated in realtime, so for testing I kept 1 file in the data folder and later started copying 10 files in the folder. Only first frame is displayed after that nothing.
How to send the glb files one after the other through xviz server?
Hello! Actually I am not working on this issue for a while but I was exactly at that point. I also wanted to send glb files to xviz server in real time but I could only display the data that I put before I start streaming.. If you figure out how to solve this and display continously, the problem will be pretty much solved. Next thing will be synchronizing the data which seems difficult while using js for xviz conversion.. Hopefully python support will solve this issue.
Hello Engin, Thanks for the reply. I am able to display it continuously. But yes synchronization is the next task now which is difficult. I wanted to know how are you converting the data into XVIZ ? I meant how did you integrate the conversion and the server part?
Hello,
I am actually doing those separately. I convert json files to xviz glb files. Then I start streaming and finally visualize.
Is there any difference between sending glb and json directly in live mode?
Hello,
I am actually doing those separately. I convert json files to xviz glb files. Then I start streaming and finally visualize.
Hello, Can you Offer you solution with more information? My work is similary with you;
@VijeetBenni hello, Can you tell me your solution? I want to do real-time visualization.
@VijeetBenni Hello， How do I send GLB files one by one through the xviz server?Have you solved the problem，Could you tell me your solution？
@heshiwen1989 I recommend you guyz to join Slack channel and ask your questions in detail. You can also find similar issues of people and answers to these issues. https://join.slack.com/t/streetscapexviz/shared_invite/zt-71l8ybk0-DKJHSCamkgZ81L7QmhLoHw
@VijeetBenni how you fix the UI missing data issue for live mode. i also meet the same issue and cannot fix it. thanks a lot, looking forward your relply
