Sending messages from server (HL2) to client (PC)

[anjelomerte]

Hey, yet another question:) Thanks for being so active on your issues!

Is there a way to send messages from the server to the client? I see that e.g. when remotely creating scenes in Unity from the client, a result array is "pulled", so there seems to be some communication in this direction implemented.

In particular, I would like to start and stop the streaming from the server side. For that, the transmission of a simple timestamp would be sufficient. Is this possible without modification of the underlying C++ plugin?

Edit: Just read that TCP port 3816 seems to be dedicated to that purpose and 4byte integers can be transmitted

[jdibenes]

Hi, There's no support for sending messages from HL2 to the PC in the same format as they are sent from PC to HL2. You can try using the HL2->PC 4-byte message queue by itself but maybe it would be simpler to create a new command to poll the server every few milliseconds and determine whether to stop/start the stream or do nothing. In either case, no modification of the C++ plugin is necessary, just need to change the RemoteUnityScene C# script and hl2ss_rus.py.

[anjelomerte]

Hey jdibenes,

thanks for the reply and sorry for my late reply, sth got inbetween. Anyways, your suggestion with polling the server seemed very reasonable which is what I'm currently digging into. I have managed to implement some custom commands for now to test behavior. The client can send some stuff (e.g. text) to the server (confirmed). However, I am struggling to read the response e.g. in the follwing setting:

# Create command buffer and append command(s)
buffer = command_buffer() 
buffer.tell_connect_success()

 # Send command(s) in buffer to the Unity app and receive response (4 byte unsigned integer per command)
client.push(buffer)
response = client.pull(buffer)

# Evaluate response
print(response[0])

In line response = client.pull(buffer) the program simply freezes. Why could that be? How can I read the uint response by the server? I need to be able to read the response to e.g. pack a timestamp as an integer. Thanks a lot!

[jdibenes]

That code looks OK. How are you implementing the tell_connect_success method?

[anjelomerte]

I tried to follow your code examples. In the module we would have a class command_buffer(hl2ss.umq_command_buffer) like so:

# Custom command buffer -------------------------------------------------------
class command_buffer(hl2ss.umq_command_buffer):
    # Command structure
    # id:     u32 (4 bytes)
    # size:   u32 (4 bytes)
    # params: size bytes

    # Indicate successful connection
    def connect_success(self):
        # Command id: 0xFFFFFFFD
        # Command params: empty byte data (id enough to indicate purpose)
        self.add(0xFFFFFFFD, b"") 

Then we define some methods to call like so:

# Message functions -----------------------------------------------------------
# Inform server about succuessful connect
def inform_connect():
    # Create command buffer and append command(s)
    buffer = command_buffer() 
    buffer.connect_success()

     # Send command(s) in buffer to the Unity app and receive response (4 byte unsigned integer per command)
    client.push(buffer)
    # response = client.pull(buffer)

    # Evaluate response
    # print("Received response:")
    # print(response[0])

[anjelomerte]

It simply was a problem of my network. For some reason it allowed messages from client to server but not the other way around. On another network it worked. Thanks a lot for your input!

[Zulex]

@anjelomerte

Could you detail a bit more on how you achieved this? I would like to send one of my Unity object coordinates together with the image frames (so they are synced in time), and I am deciding on what the best method would be.

@jdibenes, as we need more than 4 bytes, I assume we should be adjusting the DLL. Would you suggest creating something like the headpose function but then feeding the objects position from Unity? Any hints on where to start?

[anjelomerte]

As recommended by @jdibenes, I implemented polling from the client side. This means every x ms the client sends a message to the server and the server responds with a 4byte message (as with any other message). I only distinguished between two states though (0 and non-zero). However, I guess you could send a header with the number of bytes first and then evaluate the correspoding number of messages after that. Maybe there is a better way though. I also didn't explore @jdibenes 2nd suggestion:

You can try using the HL2->PC 4-byte message queue by itself