How to Capture during Streaming?( Need detailed example.. )

[dwsmart32]

Hi, Im student who is studying your API. I have some question about your API

I saw the example of processing Image(per frames) while streaming #810

I use rtmpCamera2 and want to capture a photo (per second, fps=1) and want to do Object Detection.

I have Object Detection code but,

But I can't understand how to use takephoto during streaming and setfps continuously , even though I saw your answers.

I thought the streaming video's content is saved in rtmpCamera2 instance continuosly.(if wrong plz correct me.)

But If I use "rtmpCamera2.getGlInterface().setImage(imageStreamObject);", then where does the original video contents go??

Actually I cannot utilize your takephoto function. Can I define

" public class MainActivity extends AppCompatActivity implements SurfaceHolder.Callback, View.OnClickListener { private RtmpCamera2 rtmpCamera2; ... (about streaming code...) ... ... rtmpCamera2.getGlInterface().takePhoto(new TakePhotoCallback() { @Override public void onTakePhoto(Bitmap bitmap) {...}}); } "

in Mainactivity? (I have only one class which is MainActivity and it is controlling start streaming or stop.)

couly U give me some detailed example how to takephoto from streaming content? and also setting fps also.

Thank you for offering exciting API :)

thanx a lot :)

• also I am struggling updating dependency and use new function in your API.
• I used 2.1.4 dependency and synced as in your readme.md, but setImage function doesn't work..
• Do U have a special Idea to solve this problem?

[pedroSG94]

Replace ontakePhoto code:

    ImageStreamObject imageStreamObject = new ImageStreamObject();
    imageStreamObject.load(bitmap);
    rtmpCamera2.getGlInterface().setImage(imageStreamObject);
    rtmpCamera2.getGlInterface().setStreamObjectSize(50, 50);

To:

    ImageObjectFilterRender imageObjectFilterRender = new ImageObjectFilterRender();
    rtmpCamera2.getGlInterface().setFilter(imageObjectFilterRender);
    imageObjectFilterRender.setImage(bitmap);
    imageObjectFilterRender.setScale(50, 50);

You have example of all filters in OpenGl example activities: https://github.com/pedroSG94/rtmp-rtsp-stream-client-java/blob/master/app/src/main/java/com/pedro/rtpstreamer/openglexample/OpenGlRtmpActivity.java

[dwsmart32]

thanx a lot. But Im still struggling to code capture Image from my streaming video and render(Image Object detetcion using tflite ML model) it.

1. if I define and use rtmpCamera2.getglInterface().takephtoho method, then where does the Image saved in? rtmpCamera2 instance? or ImageObjectFilter Instance?
2. I need bitmap type of Image for my ML Model as input from my streaming. But I can't understand what is Filter's role in this library(especially ImageObjectFilter's role)
3. I saw your answer #992. Is it impossible capture Image from my streaming and render it for my own purpose? I want to capture image from my video streaming (in some FPS) and wanna detect Object with that inputImage(bitmap or byte also).

Thanx for your help :)

[pedroSG94]

Hello,

1 - As you can see, that method (rtmpCamera2.getGlInterface().takePhoto) require an interface that provide you access to a bitmap (it is not saved in disk, it is a object in memory). That bitmap is the image that you must save yourself or use as you want.

2 - ImageObjectFilter is an object used to add an overlay image to stream on fly. Could be usedful to add a watermark or any info to stream and modify on fly as required. All clases with package com.pedro.encoder.input.gl.render.filters have the same purpose.

3 - I think that this issue is not related. This library only can push video and audio but reproduce an external stream is not available. In the issue metioned, the user want reproduce a stream and redirect it to other server.

[dwsmart32]

umm.. I decided to get h264 data first for takephoto parameter

but, Im struggling getting h264 data from streaming video.

how can I get a h264 data from a rtmpCamera2 instance? I think rtmpCamera2.getVideodata can give me a h264 data from streaming but I dont know what parameters should I put in rtmpCamera2.getVideodata function.

I read there must be "ByteBuffer " "MediaCodec.BufferInfo ". as parameters type.

But is empty ByteBuffer and bufferinfo okay for parameter?

[pedroSG94]

You will need to modify RtmpCamera2 class and create a callback in getH264DataRtp method. If you are using gradle you can copy that class and add the modification.