Kotlin
Performance of deserializer for small JSON string is poor as compared to GSON
Describe the bug For small stringified JSON, converting it to data class for small number of iterations is poor as compared to GSON.
To Reproduce
{
"accountNumberMaxLength": 16,
"configMessage": {
"invalidAmountGenericMessage": "Invalid amount",
"maxPaymentAmountMessage": "Amount can't be more than",
"minPaymentAmountMessage": "Amount should be at least ₹1",
"requestMoneyDefaultRemarks": "Requested via",
"sendMoneyDefaultRemarks": "Paid via"
},
"amountLimits": {
"sendMoneyMccBased": {
"5960": 200000,
"6211": 500000,
"6300": 200000,
"6529": 200000,
"7322": 200000
},
"sendMoneyVerifiedMerchant": 200000,
"sendMoneyDefaultLimit": 100000,
"requestMoneyDefaultLimit": 100000,
"minPaymentAmount": 1
},
"contactSearchQueryMaxLength": 50,
"ifscLength": 11,
"maxBankAccountPerUser": 3,
"maxPaymentAmount": 1000000,
"recipientNameMaxLength": 50,
"upiIdMaxLength": 50,
"vpaHandles": [
"@ybl",
"@paytm",
"@axl",
"@ibl",
"@apl",
"@okaxis",
"@okhdfcbank",
"@okicici",
"@oksbi",
"@axisb",
"@upi"
]
}
Above is the JSON string. In loop I try to deserialise it using gson & json and see the time it takes. I am observing that for smaller iterations gson outperforms kotlinx serialization.
Sample code
val range = (1..10000)
val gsonTime = measureTimeMillis {
range.forEach { _ ->
gson.fromJson(
cachedConfigJson,
object : TypeToken<DefaultConfig>() {}.type
)
}
}
val kotlinxTime = measureTimeMillis {
range.forEach { _ ->
if (cachedConfigJson != null) {
json.decodeFromString<DefaultConfig>(cachedConfigJson)
}
}
}
Timber.d("Ujjwal: gsonTime: $gsonTime, kotlinxTime: $kotlinxTime")
Here is the data that I got
For count = 1 Ujjwal: gsonTime: 2, kotlinxTime: 31 Ujjwal: gsonTime: 4, kotlinxTime: 15 Ujjwal: gsonTime: 3, kotlinxTime: 11 Ujjwal: gsonTime: 2, kotlinxTime: 9 Ujjwal: gsonTime: 3, kotlinxTime: 12 Ujjwal: gsonTime: 3, kotlinxTime: 9 Ujjwal: gsonTime: 3, kotlinxTime: 7 Ujjwal: gsonTime: 3, kotlinxTime: 2 Ujjwal: gsonTime: 3, kotlinxTime: 13 Ujjwal: gsonTime: 1, kotlinxTime: 11 Ujjwal: gsonTime: 2, kotlinxTime: 10
For count = 10 Ujjwal: gsonTime: 10, kotlinxTime: 48 Ujjwal: gsonTime: 11, kotlinxTime: 38 Ujjwal: gsonTime: 22, kotlinxTime: 52 Ujjwal: gsonTime: 31, kotlinxTime: 54 Ujjwal: gsonTime: 11, kotlinxTime: 40 Ujjwal: gsonTime: 23, kotlinxTime: 52 Ujjwal: gsonTime: 23, kotlinxTime: 26 Ujjwal: gsonTime: 19, kotlinxTime: 31 Ujjwal: gsonTime: 19, kotlinxTime: 40 Ujjwal: gsonTime: 24, kotlinxTime: 43
For count = 100 Ujjwal: gsonTime: 79, kotlinxTime: 219 Ujjwal: gsonTime: 131, kotlinxTime: 264 Ujjwal: gsonTime: 109, kotlinxTime: 78 Ujjwal: gsonTime: 142, kotlinxTime: 179 Ujjwal: gsonTime: 138, kotlinxTime: 161 Ujjwal: gsonTime: 121, kotlinxTime: 118 Ujjwal: gsonTime: 143, kotlinxTime: 136 Ujjwal: gsonTime: 124, kotlinxTime: 143 Ujjwal: gsonTime: 173, kotlinxTime: 113 Ujjwal: gsonTime: 169, kotlinxTime: 153
For count = 1000 Ujjwal: gsonTime: 1694, kotlinxTime: 2190 Ujjwal: gsonTime: 1210, kotlinxTime: 612 Ujjwal: gsonTime: 895, kotlinxTime: 643 Ujjwal: gsonTime: 644, kotlinxTime: 751 Ujjwal: gsonTime: 688, kotlinxTime: 663 Ujjwal: gsonTime: 837, kotlinxTime: 295 Ujjwal: gsonTime: 843, kotlinxTime: 488 Ujjwal: gsonTime: 702, kotlinxTime: 466 Ujjwal: gsonTime: 600, kotlinxTime: 532 Ujjwal: gsonTime: 705, kotlinxTime: 379
For count = 10000 Ujjwal: gsonTime: 4630, kotlinxTime: 2908 Ujjwal: gsonTime: 3585, kotlinxTime: 1710 Ujjwal: gsonTime: 2768, kotlinxTime: 1752 Ujjwal: gsonTime: 3043, kotlinxTime: 1251 Ujjwal: gsonTime: 2581, kotlinxTime: 1185 Ujjwal: gsonTime: 2697, kotlinxTime: 1173 Ujjwal: gsonTime: 2564, kotlinxTime: 1146 Ujjwal: gsonTime: 2525, kotlinxTime: 1155 Ujjwal: gsonTime: 2329, kotlinxTime: 1172 Ujjwal: gsonTime: 2508, kotlinxTime: 1139
Expected behavior
The behaviour for kotlinxTime should be better in all scenarios
Environment
- Kotlin version: [e.g. 1.9.22]
- Library version: [e.g. 1.6.2]
- Kotlin platforms: [e.g. Android]
- Gradle version: [e.g. 8.5]
- GradlePlugin = "8.2.1"
There is a lot of noise/jitter in your measurements. Did you use a proper microbenchmarking harnas such as JMH for the measurements? (and did you make sure the system was not under load in other ways). If I look at the performance (averages) it seems that kotlinx.serializization scales better with larger numbers, but has a bigger constant overhead (this really needs to be graphed out). From my perspective that is a legitimate choice unless the constant overhead is significant in real-world cases (hence the need to have a proper harnas).
Other things to consider:
- You're including the Gson TypeAdapter lookup and kotlinx.serialization KSerializer lookup in the loop. This will skew numbers. Look it up once outside the measurement. If you want to benchmark that, do it separately from actual deserialization.
- The kotlinx.serialization loop has a top-level conditional whereas Gson does not. Eliminate all conditionals and benchmark only the operation in question.
- Use JMH, as @pdvrieze says. Warmup iterations, ideally on a machine with little else running and with fixed-clock CPUs.
- Test data sources other than string. Generally you are not reading from a string but from a source of bytes. Thus, how UTF-8 decoding occurs as part of the process (whether amortized or up-front) plays a factor.
Am I correct that count is your range's upper bound? E.g. for count = 10 it was range = (0..10)? Did you restart the program between measurements? In that case, I think the differences in time may be attributed to various JVM startup mechanisms — loading classes, JIT compilation, etc. Generally, I can imagine that kotlinx.serialization does more initialization of various caches and configurations, and this is not necessarily a problem, as most applications (Android and server-side as well) typically live long enough for these effects to be insignificant. Your measurements for larger counts kinda confirm that.
If you want to measure actual deserialization performance, take a look at @pdvrieze and @JakeWharton advises, JMH is the best tool to make proper measurements.
However, if you want to measure application cold startup time, and you're interested in that particular metric, I imagine you would need other tools, as measureTimeMillis is also not sufficient for that case.
