siddhi
siddhi copied to clipboard
How to write a Kryo serializer for SiddhiAppRuntime?
As a first attempt, I wrote the following (non-working) code:
package com.my.alerting.siddhi
import com.esotericsoftware.kryo.io.{Input, Output}
import com.esotericsoftware.kryo.{Kryo, Serializer}
import io.siddhi.core.SiddhiAppRuntime
class SiddhiAppRuntimeSerializer extends Serializer[SiddhiAppRuntime] {
override def write(kryo: Kryo, output: Output, runtime: SiddhiAppRuntime): Unit = {
val runtimeSnapshot = runtime.snapshot()
output.writeInt(runtimeSnapshot.length)
output.writeBytes(runtimeSnapshot)
}
override def read(kryo: Kryo, input: Input, `type`: Class[SiddhiAppRuntime]): SiddhiAppRuntime = {
val runtimeSnapshotLength = input.readInt()
val runtimeSnapshot = input.readBytes(runtimeSnapshotLength)
val runtime = `type`.newInstance()
runtime.restore(runtimeSnapshot)
runtime
}
}
But I realize that I cannot create a new runtime instance like this val runtime = type.newInstance(), so I am trying something else:
override def write(kryo: Kryo, output: Output, runtime: SiddhiAppRuntime): Unit = {
val app = runtime.getSiddhiApp.toString
val runtimeSnapshot = runtime.snapshot()
output.writeString(app)
output.writeInt(runtimeSnapshot.length)
output.writeBytes(runtimeSnapshot)
}
override def read(kryo: Kryo, input: Input, `type`: Class[SiddhiAppRuntime]): SiddhiAppRuntime = {
val app = input.readString()
val runtimeSnapshotLength = input.readInt()
val runtimeSnapshot = input.readBytes(runtimeSnapshotLength)
val manager = new SiddhiManager()
val runtime = manager.createSiddhiAppRuntime(app)
runtime.restore(runtimeSnapshot)
runtime
}
However the above approach also fails because runtime.getSiddhiApp.toString does not return the whole Siddhi App code as I was expecting, so I cannot use that string for creating a SiddhiAppRuntime using the manager. Another concern is having to create a SiddhiManager within the read method...which I would prefer to avoid if possible.
Please, can anyone help on this?
FYI, I posted the same question in https://stackoverflow.com/questions/60529065/how-to-write-a-kryo-serializer-for-siddhiappruntime.
Also, as an aside note, I think it should be possible to recover the full siddhi app code (as a string) from the runtime object, what do you guys think @mohanvive ?
As an update, this is the best thing I have come up with so far:
import com.esotericsoftware.kryo.io.{Input, Output}
import com.esotericsoftware.kryo.{Kryo, Serializer}
import io.siddhi.core.{SiddhiAppRuntime => SiddhiRuntime}
// A simple wrapper necessary because of https://github.com/siddhi-io/siddhi/issues/1630
case class SiddhiAppRuntime(app: String, runtime: SiddhiRuntime)
class SiddhiAppRuntimeSerializer() extends Serializer[SiddhiAppRuntime] with Serializable {
override def write(kryo: Kryo, output: Output, appRuntime: SiddhiAppRuntime): Unit = {
val runtimeSnapshot = appRuntime.runtime.snapshot()
output.writeString(appRuntime.app)
output.writeInt(runtimeSnapshot.length)
output.writeBytes(runtimeSnapshot)
}
override def read(kryo: Kryo, input: Input, `type`: Class[SiddhiAppRuntime]): SiddhiAppRuntime = {
val app = input.readString()
val runtimeSnapshotLength = input.readInt()
val runtimeSnapshot = input.readBytes(runtimeSnapshotLength)
// Isn't there a way to avoid this??
val siddhiManager = new SiddhiManager()
val runtime = siddhiManager.createSiddhiAppRuntime(app)
runtime.restore(runtimeSnapshot)
SiddhiAppRuntime(app, runtime)
}
}
The main problem with this approach is that I rely on a SiddhiManager object in order to be able to recreate the runtime from a string representation of the Siddhi app, and before restoring the snapshot.
How could I avoid having to create a SiddhiManager in the read method? I guess I could rely on a global singleton object since, indeed, Siddhi seems to recommend using one SiddhiManager per JVM. However, I wonder whether this could result in contention issues (imagine a lot of threads using different runtimes).