Coherence icon indicating copy to clipboard operation
Coherence copied to clipboard

Add third party plugin support

Open McManning opened this issue 4 years ago • 2 comments

I'd like to be able to support third party plugins using Coherence as a bridge for extra communication between Blender / Unity - sending custom messages back and forth using the same protocol.

Slightly related to #15

Initial Draft / Use Cases

1. I want to send custom messages between apps

Let's say someone wants to create a more complicated integration between their water simulation plugin in Blender and Unity's interaction with that simulation.

In Unity, a developer can send and listen to custom messages through Coherence:

Coherence.SendCustomMessage<MyDataType>(MY_CUSTOM_ID, payload); 
Coherence.OnCustomMessage<MyDataType>(MY_CUSTOM_ID, handler);
...
public void handler(string ID, MyDataType payload) {
    // do work with payload
}

MY_CUSTOM_ID would be some unique message ID for the third party plugin (like a "updateWaterSim" or something). The payload T would need to be a serializable struct that follows all the rules of our interop structs (no variable length arrays, complex data types, yadda yadda).

In Blender-land, the DLL would just forward the message off to Python for handling. The third party Blender addon would register a listener for the message and act accordingly.

import coherence from somewhere

def register():
  coherence.on_custom_message(MY_CUSTOM_ID, handler)

def unregister():
  coherence.remove_on_custom_message(MY_CUSTOM_ID, handler)

def handler(ID: str, payload_ptr):
  payload = ctypes.cast(payload_ptr, MyDataType).value
  # do things with payload.foo / payload.bar
  # send a response message
  coherence.send_custom_message(MY_CUSTOM_ID_2, new_data)

Plugin developers would need to use ctypes to get a serializable struct to/from Unity.

Internally, this is sent as an RpcMessage.CUSTOM and the target is MY_CUSTOM_ID.

2. I want to send large arrays of data between apps

I'd expect the same API to support arrays of structs, even if internally it's implemented slightly different:

Coherence.SendCustomMessage<MyDataType[]>(MY_CUSTOM_ID, payload); 
Coherence.OnCustomMessage<MyDataType[]>(MY_CUSTOM_ID, handler);
...
public void handler(string ID, MyDataType[] payload) {
    // do work with payload
}

def handler(ID: str, payload_ptr, payload_length):
  payload = ctypes.cast(payload_ptr, MyDataType * payload_length).value
  # do things with payload.foo / payload.bar
  # send a response message
  coherence.send_custom_message(MY_CUSTOM_ID_2, new_data)

The rest is up to the third party developer to figure out how to quickly shape their data.

McManning avatar Feb 23 '21 15:02 McManning

More use cases:

3. I want a custom representation of mesh data in Unity

Let's say I want a mesh from Blender to transfer as a volumetric in Unity. Rather than trying to convert and transfer on the Blender side - I'd rather write a Unity plugin that integrates with Coherence to grab the geometry data coming from Blender updates, feed that into a compute buffer, and run a voxelize / SDF builder compute shader to spit out a Texture3D I can use for whatever in Unity (VFX graph emitters/attractors/repulsors, shaped volumetric lighting, raymarched material shaders, etc).

McManning avatar Mar 18 '21 14:03 McManning

Started on this on the plugin-architecture branch.

McManning avatar Apr 13 '21 02:04 McManning

Resolved as part of the new C++ core

McManning avatar Nov 16 '22 22:11 McManning