diffplug
Concurrent spotlessJava tasks with eclipse hanging on empty p2 repository
With spotless gradle plugin 7.0.4 we are seeing the spotlessJava tasks keeps hanging in our CI environments. further investigation showed this seems like a concurrency issue. jstack gives me this stacktrace with two entries (for each active spotless java task) similar to this:
"Execution worker Thread 6" #117 [1841467] prio=5 os_prio=0 cpu=2438.51ms elapsed=100.17s tid=0x00007fcc0400e4e0 nid=1841467 in Object.wait() [0x00007fcc415e5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait0([email protected]/Native Method)
- waiting on <no object reference available>
at java.lang.Object.wait([email protected]/Object.java:366)
at java.lang.Object.wait([email protected]/Object.java:339)
at dev.equo.solstice.p2.LockFile.<init>(LockFile.java:40)
- locked <0x00000000d2a38d00> (a java.lang.Object)
at dev.equo.solstice.p2.P2Client.<init>(P2Client.java:72)
at dev.equo.solstice.p2.P2Model.queryRaw(P2Model.java:105)
at dev.equo.solstice.p2.P2Model.query(P2Model.java:136)
at com.diffplug.spotless.extra.EquoBasedStepBuilder.lambda$build$0(EquoBasedStepBuilder.java:128)
at com.diffplug.spotless.extra.EquoBasedStepBuilder$$Lambda/0x00007fcd9932a9d0.get(Unknown Source)
at com.diffplug.spotless.JarState$Promised.get(JarState.java:73)
at com.diffplug.spotless.extra.EquoBasedStepBuilder$EquoStep.state(EquoBasedStepBuilder.java:193)
at com.diffplug.spotless.extra.EquoBasedStepBuilder.lambda$build$cc29b8bc$1(EquoBasedStepBuilder.java:144)
at com.diffplug.spotless.extra.EquoBasedStepBuilder$$Lambda/0x00007fcd9932c700.apply(Unknown Source)
at com.diffplug.spotless.FormatterStepSerializationRoundtrip.stateSupplier(FormatterStepSerializationRoundtrip.java:57)
at com.diffplug.spotless.FormatterStepSerializationRoundtrip$HackClone$$Lambda/0x00007fcd9972f3a8.get(Unknown Source)
at com.diffplug.spotless.ThrowingEx.get(ThrowingEx.java:67)
at com.diffplug.spotless.FormatterStepSerializationRoundtrip$HackClone.writeObject(FormatterStepSerializationRoundtrip.java:113)
at java.lang.invoke.LambdaForm$DMH/0x00007fcd9813c000.invokeSpecial([email protected]/LambdaForm$DMH)
at java.lang.invoke.LambdaForm$MH/0x00007fcd98018c00.invoke([email protected]/LambdaForm$MH)
at java.lang.invoke.Invokers$Holder.invokeExact_MT([email protected]/Invokers$Holder)
at jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl([email protected]/DirectMethodHandleAccessor.java:154)
at jdk.internal.reflect.DirectMethodHandleAccessor.invoke([email protected]/DirectMethodHandleAccessor.java:103)
at java.lang.reflect.Method.invoke([email protected]/Method.java:580)
at java.io.ObjectStreamClass.invokeWriteObject([email protected]/ObjectStreamClass.java:1072)
at java.io.ObjectOutputStream.writeSerialData([email protected]/ObjectOutputStream.java:1529)
at java.io.ObjectOutputStream.writeOrdinaryObject([email protected]/ObjectOutputStream.java:1451)
at java.io.ObjectOutputStream.writeObject0([email protected]/ObjectOutputStream.java:1194)
at java.io.ObjectOutputStream.writeObject([email protected]/ObjectOutputStream.java:358)
at com.diffplug.spotless.ConfigurationCacheHackList.writeObject(ConfigurationCacheHackList.java:76)
at java.lang.invoke.LambdaForm$DMH/0x00007fcd9813c000.invokeSpecial([email protected]/LambdaForm$DMH)
this is reproducible when I cleanup the ~/.m2/repository/dev/equo/p2-data/ folder locally and run our build again.
In general we got biten by this dynamic p2 lookups and I would love to see the resolution of those artifacts would be wired into the gradle dependency resolution instead of being completely hidden deep down in the spotless logic. This currently blocks us from updating to newer spotless gradle plugin versions.
Looking more at the code. It seems we query p2 just for getting a list of jars we need to have on the classpath. I noticed there is already a cacheDirectory property that would allow to prepopulate that local p2 repo upfront with some custom (gradle) logic.
Ideally I would like to skip the p2 quering alltogether and provide the jars required by the eclipse formatter manually. I wish they would just provide a bom.
@nedtwigg Do you by chance have an idea how to address this? I was thinking moving away from the eclipse formatter and use something else. but would love to avoid that overhead :)
like a concurrency issue
what about changing this plugin to sequentiell execution to avoid this. Its very fast anyways so is could be an tmp fix, not adding up to much time.
Having this issue in rewrite the same, putting the workers.max to 2 fixes the concurrency issues. This might be an good compromis for this as well.
