PhoneVR
PhoneVR copied to clipboard
Hardware encoding and Latency minimization via FFMPEG
Hardware Encoding Support and Network hand-wave latency reduction
- [x] Research low-latency Protocols: Done: RTP best for this application.
- [x] Research Available libraries
- [x] PJ-SIP, FFMPEG
- [x] PJ-SIP - Tests: Status: Failed no pj-media Support.
- [ ] Implement FFMPEG
- [ ] Implement FFMPEG on Desktop
- [x] Protocols Selection: RTP (ultra-low latency) + NVENC/NVDEC Encoder (h.264 Container format)
- [ ] Code Implementation:
- [x] Implement ffmpeg as ffmpeg-cli commandLineInterface integration
- [x] Stage0 - Research D3DSharedResourceHandle Streaming
- [x] Test1: using Inprocess pipes to get pipe render to ffmpeg for encoding. Status: Failed - WritePipe() Speed too low (~15 fps)
- [x] Test2: Piping D3DSharedResourceHandle to ffmpeg.exe process in background. Status Failed - Hard to maintain with FFMPEG-master updates, better to implement libav*
- [x] ~Bypass Mobile negotiation, for Development~
- [x] ~Modify FFMPEG fopen_inputs() to get D3DSharedResourceHandle~
- [x] Stage0 - Research D3DSharedResourceHandle Streaming
- [ ] Implement ffmpeg as libav* libraries, if Test1 & 2 Both fail.
- [ ] Stage1 - Static Content Streaming
- [ ] Stream Static testfile.mp4, with RTP to localhost, for Dev
- [x] Bypass Mobile negotiation, for Development
- [x] Code FFMPEG Init()
- [x] Code FFMPEG fopen_inputs()
- [x] Code FFMPEG fopen_outputs()
- [x] Code FFMPEG transcode()
- [x] Code FFMPEG transcode_init()
- [x] Code FFMPEG transcode_step() <--- Main Loop
- [ ] Code FFMPEG stop_stream(), Outputs and Inputs
- [x] Code FFMPEG deInit() <--- safe exit + Memory Release
- [ ] Stream Static testfile.mp4, with RTP to localhost, for Dev
- [ ] Stage2 - Dynamic Content Streaming
- [ ] Modify FFMPEG fopen_inputs() to get D3DSharedResourceHandle
- [x] Research methodology
- [x] Modify FFMPEG fopen_inputs() to take D3DSharedResourceHandle as Input instead of Static file
- [ ] Modify FFMPEG fopen_inputs() to get D3DSharedResourceHandle
- [ ] Stage1 - Static Content Streaming
- [x] Implement ffmpeg as ffmpeg-cli commandLineInterface integration
- [ ] Implement FFMPEG on Mobile End
- [ ] Add shared libraries (.so) to Android project.
- [ ] Research into available FFMPEGAndroid ports/wrappers
- [ ] Test3: Port Wrapper Implementation test on Android
- [ ] If Test3 not successful, implement FFMPEG libav* code.
- [ ] Implement FFMPEG on Desktop
Fixes #44 and #52