Is it possible to redirect the stream to another application ?

[Darkroll76]

Hi,

Currently, scrpcy opens independent windows to render video stream but I'm trying to integrate it inside an application. Is there any way to do it? By redirecting the stream?

It could be a Unity App or Java App or C# UWP App but I would like to be able to share android-based VR headset screen (Oculus Go, Quest, PICO, etc) inside a desktop application.

Thanks for your time, Killian

[iiasceri]

https://github.com/barry-ran/QtScrcpy

[Darkroll76]

https://github.com/barry-ran/QtScrcpy

Thanks, I will try if I could find what I need with this project. For now, it just seems to offer the same features.

[rom1v]

Currently, scrpcy opens independent windows to render video stream but I'm trying to integrate it inside an application. Is there any way to do it?

The proper way would be to split scrcpy in two parts: libscrcpy and the UI (https://github.com/Genymobile/scrcpy/issues/707#issuecomment-520941669). But this is not done.

Another way could be to use only the .jar on your side, and write your client which mimic the protocol used between the client and the server (but this protocol should be considered private, i.e. it may change at any time).

Alternatively, you could fork current master and replace some parts…

[LBensman]

That's pretty much what I did - just take the Java portion that already sends back H.264 stream (with initial custom header), and just handle H.264 stream inside your desktop application (or where ever you want to consume it). Basically reuse the server jar, but dump viewer in favor of your own stream handling and visualization.

On a separate note, @rom1v , did you notice from screenshots above that QtScrcpy seems to be supporting audio? I know you tried USBAudio (I'm looking into it now myself), but quick look (really, probably not even 2 min spent on it) at QtScrcpy source, I couldn't see much for audio handling. Have you looked into it and see if it something more solid than USBAudio?

[rom1v]

did you notice from screenshots above that QtScrcpy seems to be supporting audio?

I don't think so. They're just shortcut for volume up/down on the device. Also see #612.

Have you looked into it and see if it something more solid than USBAudio?

The real solution will be the Playback Capture API introduced in Android Q. See #933.

[LBensman]

Yeah, I know about Playback Capture API (I've been all over the threads, esp #14 one) - unfortunately it is insufficient for what I need...

[rom1v]

unfortunately it is insufficient for what I need...

Why?

[LBensman]

I mean it would "help somewhat" in that it will allow for a small sliver of audio to be captured, but it is a co-operative solution between apps and the capturing app (the app has to explicitly allow for its audio to be captured), and, as far as I understand, would only work with a small subset of apps and none of the system sounds. It seems PCAPI was designed for a very very narrow use case like streaming of game playing, rather than general-purpose API. In other words, in scrcpy analogy, only being able to capture screen when a co-operating game is being played, but otherwise getting a blank screen in all other cases - much more limited and thus much less useful.

[rom1v]

I did not investigated yet, but by calling private API with shell permissions, it is maybe possible to capture the audio from any app (I'm not sure).

For example, the MediaProjection API requires to get an authorization from the user, but scrcpy does not.

[Darkroll76]

Hi @LBensman ,

That's pretty much what I did - just take the Java portion that already sends back H.264 stream (with initial custom header), and just handle H.264 stream inside your desktop application (or where ever you want to consume it). Basically reuse the server jar, but dump viewer in favor of your own stream handling and visualization.

Could you help me with that part? Should it work if I'm just launching server .jar? Because, I guess, it requires to modify the scrcpy code, but It could be very helpful if you help me where.

I want to display the video stream in a Unity app, I can handle H.264 stream from the "receiver" side, but I'm just stuck trying to find a way to modify scrcpy to send back H264 to it.

Thanks,

[rom1v]

A proper solution could be to implement an option --serve, which would wait for a client to connect before starting the mirroring, and it would forward the video stream.

For example, it would be used like this:

scrcpy --serve=tcp:localhost:1234

That way, once implemented, anyone could receive the raw video stream.

It's not a priority for me, but if you work on something that requires it, you could implement it that way and submit a pull request. 😉

[Darkroll76]

Yes, this solution could be great and easy to use. Thanks for the suggestion!

I can for sure understand that isn't a priority for you ;).

Since I'm not an expert in java development, I'm not sure I will be able to do it properly but I'm still going to start investigating the code and try (If anyone who is reading this post wants to join ;) ).

Have you any tips? I have read the developer readme but it would be helpful to know which IDE (for example) should be used to edit your project. I'm using Android Studio for the server-side but which IDE are you using for the client development and to build scrcpy.exe?

Thanks for your time,

[rom1v]

Since I'm not an expert in java development

This feature has to be implemented on the client-side, which is in C :wink:

Basically, when you receive a packet, it is decoded + recorded (if necessary): https://github.com/Genymobile/scrcpy/blob/39356602ed472cc3f533e36ae04a110b247c29e0/app/src/stream.c#L81-L97

We can, in addition, write it to a socket.

But in fact, it could even be written earlier (without even parsing it):

https://github.com/Genymobile/scrcpy/blob/39356602ed472cc3f533e36ae04a110b247c29e0/app/src/stream.c#L229

I have read the developer readme but it would be helpful to know which IDE (for example) should be used to edit your project.

Whatever you want. I use vi.

[Darkroll76]

Hi,

I don't know if I should create another topic about creating an --serve option to forward video stream. I just want to be sure I'm doing the right thing since I only started to manage and understand the scrcpy client source code.

First in scrcpy/app/scr/cli.c, I added the option {"serve", required_argument, NULL, OPT_SERVE}, then into the switch I added

case OPT_SERVE:
                opts->serve = optarg;
                break;

Then in scrcpy/app/scr/scrcpy.c : scrcpy(const struct scrcpy_options *options)

if (!stream_start(&stream)) {
        goto end;
    }
    stream_started = true;

if (!serve_init(options->serve)){
        goto end;
    }

I would like to split the args 'tcp:localhost:1234' and use net_connect function from net.c to create a socket. Then the idea is to send the packet through this socket each time process_frame is called. But in code, I don't know the best way I should do it. Create another class? Or just create a function serve_packet(AVPacket *packet) in stream.c ?

Thanks again for your time, Killian

[qaisbayabani]

https://github.com/qaisbayabani/SCRCPY-C-Sharp-Client is a demo to show a self made application for scrcpy output

[Darkroll76]

Hi @qaisbayabani !

Thanks for the demo ! I tried but I got a java.io.IOException: Connection refused Any idea ?

[qaisbayabani]

i have made 100% working clients for both c# and java is available on request if the party needs it so deperetly a one page 100 line code.

[qaisbayabani]

mr darkroll u r using an emulator? are you connecting your phoen directly via USB? and confirm u are using scrcpy 1.12.1 version?

adb commands must be reconfigured according to your own environment other things will work fine. i have deleted this demo repository because i have developed a flawless full alpha version now which is working 100% fine soon i will load it to repository

if you could get me screenshots of output window than may i could understand that will be a thanks for your response.

[overvox]

i have made 100% working clients for both c# and java is available on request if the party needs it so deperetly a one page 100 line code.

Hey, any chance you could share the C# Code or are you still working on a functioning version?

[Darkroll76]

@qaisbayabani I'm interested to see it too.

[qaisbayabani]

i have made 100% working clients for both c# and java is available on request if the party needs it so deperetly a one page 100 line code.

Hey, any chance you could share the C# Code or are you still working on a functioning version?

the version mr.darkroll was downloaded is also a working one.

[qaisbayabani]

mr darkroll u r using an emulator? are you connecting your phoen directly via USB? and confirm u are using scrcpy 1.12.1 version?

adb commands must be reconfigured according to your own environment other things will work fine. i have deleted this demo repository because i have developed a flawless full alpha version now which is working 100% fine soon i will load it to repository

if you could get me screenshots of output window than may i could understand that will be a thanks for your response.

is it working or still error

[qaisbayabani]

mr darkroll u r using an emulator? are you connecting your phoen directly via USB? and confirm u are using scrcpy 1.12.1 version? adb commands must be reconfigured according to your own environment other things will work fine. i have deleted this demo repository because i have developed a flawless full alpha version now which is working 100% fine soon i will load it to repository if you could get me screenshots of output window than may i could understand that will be a thanks for your response.

is it working or still error

as i guess u were using an emulator u should use a phone and tell the room is it working at ur side

[Darkroll76]

@qaisbayabani I'm using it with a VR Headset based on Android. I still have a Connection refused with the first demo you posted. It's not directly connected by USB but I added the adb tcpip 5555 and adb connect command in your script. I was using 1.12.1 then I changed it by the new one (1.13).

My version allows forwarding the packet received on another socket. In your demo, you directly push the scrcpy-server and connect to it. It is very helpful to understand how you integrated it into a C# solution.

[qaisbayabani]

https://github.com/qaisbayabani/SCRCPY-C-Sharp-Client it covers. multithreading a single file program.cs converting smooth capture from scrcpy-server to c# client adb commands using c# server port opening listening to stream and forwarding the stream to other port. opencv capture from network stream and disply H264 stream from android screen capture. cross plateform stuff. enjoy

[qaisbayabani]

Mr. Overvox

Hey, any chance you could share the C# Code or are you still working on a functioning version?

its here

[Darkroll76]

@qaisbayabani I don't have the connection refused any more. But It's still not working for me. I always have a gray screen with "Empty" outputs. I'm using it on Windows with an Android VR Headset.

[qaisbayabani]

@qaisbayabani I don't have the connection refused any more. But It's still not working for me. I always have a gray screen with "Empty" outputs. I'm using it on Windows with an Android VR Headset. i have prepared it and tested with android mobile phone and working perfect. may be its needs fine tunning for VR Headset. try to attach VR with USB Cable may be a Network ports speed issue. try it tell me if its working on android phone as its working fine here on 50 sets i have tried form andorid 6.0 to 9.0.

[Darkroll76]

@qaisbayabani i have prepared it and tested with android mobile phone and working perfect. may be its needs fine tunning for VR Headset. try to attach VR with USB Cable may be a Network ports speed issue. try it tell me if its working on android phone as its working fine here on 50 sets i have tried form andorid 6.0 to 9.0.

I tried using USB Cable on VR Headset (Android 8) and my phone too (Android 10). I got the "Third is Gneeal" error and receiving "Empty" outputs each time.

[hgedik]

@qaisbayabani hi,

I have Android 9 (samsung galaxy s8+). I tried over wifi and usb cable. And same result. just "empty" outputs.