kotlinx.serialization
kotlinx.serialization copied to clipboard
Kotlin
Serialize & Deserialize Kotlin Delegates
What is your use-case and why do you need this feature? Right now there is no way to serialize kotlin delegates and even if there is , its not easy ! I use Jetpack Compose , If I use type like MutableState and specify a custom serializer I have to type .value for each property everywhere in my code where I used it , which is just very annoying and code doesn't look good
Describe the solution you'd like I would like an easy annotation / way to serialize delegated properties , easy way to enforce serialization for delegated properties
There's no built-in way to do so, but you can write a custom serializer that uses getters and setters of the delegates.
Maybe we can implement @SerializeByGetterAndSetter annotation
@sandwwraith
There's no built-in way to do so, but you can write a custom serializer that uses getters and setters of the delegates.
It seems that the compiler is actually completely ignoring properties with delegates for serialization, even those with a custom serializer annotation @Serializable(with = ...).
My use case is the serialization of a graph with nodes intended to look like this (simplified):
class TreeNode private constructor(override val id: ReferenceableID) : Referenceable() {
@Serializable(with = LazyReference.Serializer::class)
var parent: TreeNode? by LazyReference() // <- stores the node's ID internally
var children: List<TreeNode> = mutableListOf()
constructor(id: ReferenceableID, initialParent: TreeNode? = null) : this(id) {
parent = initialParent
}
}
When the children of a parent node are deserialized, each of their parent properties refers to the immediate parent node, which has not been constructed yet. To deal with this cycle during deserialization, I delay parent resolving: I use a context which remembers deserialized nodes via a HashMap<ReferenceableID, Referenceable>. LazyReference then uses this context later to find the node by its ID.
The closest I could get with the current implementation is via a separate backing property:
class TreeNode private constructor(override val id: ReferenceableID) : Referenceable() {
private var _parentID: ReferenceableID? = null // <-- will be serialized
var parent: TreeNode? by LazyReference(this::_parentID) // <-- will not be serialized
var children: List<TreeNode> = mutableListOf()
constructor(id: ReferenceableID, initialParent: TreeNode? = null) : this(id) {
parent = initialParent
}
}
Could you consider making the compiler honor @Serializable(with = ...) for properties with delegates and not ignore those?
You would need to write a surrogate , that's how I did it , I had nodes inside which were delegates by mutable state , since I am using jetpack compose to render a big map so nodes contained a self reference
So I had to recursively convert the nodes to surrogates and register surrogate classes in serializersModule and I serialized and deserialized surrogate classes instead of actual nodes
@OliverO2
I wrote the surrogate for parent class instead of the class that was being delegated.
Good to hear that surrogates worked for you. The downside of both approaches is boilerplate which we are all trying to avoid. If I'm not overlooking something, this could all be avoided by letting the serialization compiler plugin accept getters and setters instead of insisting on backing fields.
Some thoughts on a possible implementation:
As explained in the section Delegated properties – Translation rules, the compiler generates an auxiliary property for each delegated property like this:
private val parent$delegate: LazyReference
var parent: TreeNode? by parent$delegate
To serialize the delegated property in the above example:
- The corresponding auxiliary property (
parent$delegate) would be serialized. - The delegated property's deserializer (the one for
parent) would return the delegate (aLazyReferenceobject). - The delegate would be assigned to the auxiliary property (
parent$delegate). - The delegated property (
parent) would be left as is.
As long as the delegate itself is serializable, all of this could work without any additional annotation.
I guess they don't want delegated properties to be serializable by default
They don't have to be. That could depend on the delegate class being annotated with @Serializable.
A lot of classes from other libs and jetpack compose won't have serializable annotation over them
So how would this be fixed , would we have to implement a serializer for the delegate class ourselves ?
You might want to look at Deriving external serializer for another Kotlin class (experimental). However, it will often be unfeasible to serialize classes that were not designed with serialization in mind.
@wakaztahir Can you please provide an example of serializable class with delegates that can be used with Jetpack Compose? I'm investigating the issue and so far it seems that delegates should only be used to obtain State<T> instance
I think the original discussion about this started here on Slack: https://kotlinlang.slack.com/archives/C7A1U5PTM/p1628060200011800
I am using mutable state inside classes by delegating it , MutableState<T>
Yes , In Jetpack Compose MutableState<T> / State<T> are the only delegating classes , I am also obtaining State<T> instances only with delegates but in general delegating with classes that use generics to return same type as the parameter is common , I probably exaggerated
There's no built-in way to do so, but you can write a custom serializer that uses getters and setters of the delegates.
Maybe we can implement
@SerializeByGetterAndSetterannotation
I find that for more complex serializations I need a "delegate" (which can be a "private" member class) that is just a simple class with properties (and the ability to be constructed from the actual type - toDelegate(), as well as the reverse - Delegate.toActual(). It needs a fairly trival custom serializer. It would be good to have an annotation to allow this custom serializer to be automatically generated.
Jetpack compose also includes mutable state list , which is the snapshot state list inheriting from list of course And mutable state map
Thought I'd mention these classes because they are used a lot and sould be easily serializable like a normal list yet I have to cast it or if I use them as properties then provide a serializer either surrogate / custom
I would see this working best in combination with the explicit backing field keep (when applied to delegates).
I've developed most of my classes to use property delegates for the sake of autosaving and listenable properties and such. I'm really excited about the potential of kotlinx.serialization but this issue is somewhat killing it for me.
I seem to have two choices:
- Keep using my property delegates but use custom serializers. This roughly doubles the amount of code I have to write to define a class.
- Design my classes to have internal data models that don't use property delegates, make sure everything is bound correctly, then build and serialize my classes based on those internal models.
A @SerializeByGetterAndSetter annotation would be a lifesaver.
Serializing delegate properties is actually much simpler. It is not even necessary to introduce an extra annotation. Almost everything pretty much works out of the box right now:
The Kotlin compiler transforms a delegate property declared like this:
var parent: TreeNode? by ReferenceDelegate()
into these two properties:
var `parent$delegate` = ReferenceDelegate() // (1) auxiliary property
var parent: TreeNode? by `parent$delegate` // (2) accessor property
Actually, serialization for the two transformed properties works as intended: (1) serializes correctly, (2) is ignored for serialization.
The only thing that is missing currently:
- Have the serialization compiler plugin recognize the auxiliary property (1) as serializable if its delegate class (
ReferenceDelegateabove) is serializable.
Here is a completely working code example demonstrating the above: Gist: Serializing delegate properties with kotlinx.serialization
Note: Names enclosed in backticks are not supported by the Android runtime.
Is there a chance of having this seemingly simple solution implemented, or getting a PR accepted?
EDIT: Inconsistent delegate name corrected.
@OliverO2 I 100% agree with you. It would be logical for the kotlinx.serialization compiler to recognize if an auxiliary property is serializable. The current behavior can continue to be the default for non-serializable delegates.
Serialization here seems highly straightforward. It is possible to get a reference to a property delegate itself with KProperty0.getDelegate at runtime already.
What I'm curious about is how the delegation process (the "by" keyword) actually works and if that can be done at runtime or it requires compiler magic
@Serializable
class NameDelegate {
val name = "rex"
operator fun getValue(thisRef: Any?, property: KProperty<*>) = name
}
@Serializable
class Dog {
val name by NameDelegate()
}
val dog = Dog()
// Serializing a delegate is easy
// (this is symbolic of what the kotlinx.serialization compiler might setup)
val json = buildJsonObject {
val wasAccessible = dog::name.isAccessible
dog::name.isAccessible = true
put(dog::name.name, JsonPrimitive((dog::name.getDelegate() as NameDelegate).name))
dog::name.isAccessible = wasAccessible
}
val newdog = Dog()
// deserializing a delegate is hard
// this causes an error because val cannot be reassigned.
// Also, this doesnt even set up the delegate using the "by" mechanism
newdog.name = json["name"]!!.jsonPrimitive.content
So my question is how deeply embedded into the kotlin compilation process does this feature addition have to be?
@mgroth0 To make this work correctly (transparently) it would need to be implemented into the compiler plugin. It would be worth to note that it would also need to deal with operator provideDelegate on the deserialization side (especially if it is a read-only delegate).
I've looked into the serialization compiler plugin. The point where it decides which properties to serialize seems to be the compiler's frontend (resolving) phase. At that time, the plugin sees the accessor property, which it correctly decides not to serialize. But it does not see the synthetic auxiliary property (...$delegate), which seems to be created in the compiler's backend phase (though I could not spot where exactly).
To find out where to pick up the auxiliary property for serialization (backend IR) code generation, one would need a proper understanding of the interactions between various compiler components (frontend, extension points, backend), in particular the sequencing of those. And there is basically no documentation, so it is quite time-consuming to figure this out. The required change might still be straightforward.
In the meantime, we can still have serializable delegates by adding the auxiliary property manually, as shown above. It's one line of extra boilerplate per delegated property.
There's no built-in way to do so, but you can write a custom serializer that uses getters and setters of the delegates.
Maybe we can implement
@SerializeByGetterAndSetterannotation
This would be also handy in case of serializing and deserializing inline properties in case if their set call with parameters passed from serialized input in constructor is required.
I'll just leave a note here that this issue is connected to the upcoming "explicit backing fields" feature (see https://github.com/Kotlin/KEEP/issues/278). It will provide a more explicit mechanism than property delegation. That is, when "explicit backing fields" is implemented, one can consider a delegated property like this:
var property: Type by createDelegate() // CASE (1)
simply to be a shorthand notation for a more verbose explicit backing field declaration like this:
var property: Type // CASE (2)
field = createDelegate()
get() = field.getValue(this, ::property)
set(value) { field.setValue(this, ::property, value) }
The serialization design will have to take into account those two cases and what happens when the case (1) declaration is expanded into the case (2) declaration.
missing this feature very much
here's something I tried , thinking derived serializer might set the stateful property
@Serializable
abstract class Something(open val prop: String)
@Serializable
class StatefulSomething : Something("") {
override var prop: String by mutableStateOf("")
}
@OptIn(ExperimentalSerializationApi::class)
@Serializer(forClass = StatefulSomething::class)
object StatefulSomethingSerializer
val json = Json {
serializersModule = SerializersModule {
polymorphic(Something::class) {
subclass(StatefulSomething::class)
}
}
}
@Test
fun testStatefulSerialization() {
val state = StatefulSomething()
state.prop = "hello-world1"
assertEquals("hello-world1",state.prop)
val encoded = json.encodeToString(state)
assertEquals("{\"prop\":\"hello-world1\"}", encoded)
state.prop = "something-else"
val encoded2 = json.encodeToString(state)
assertEquals("{\"prop\":\"something-else\"}", encoded2)
// This test fails
val decoded = json.decodeFromString<StatefulSomething>(StatefulSomethingSerializer, encoded)
assertEquals("hello-world", decoded.prop)
}
