opengrok icon indicating copy to clipboard operation
opengrok copied to clipboard

bump xstream to 1.4.21

Open vladak opened this issue 1 year ago • 3 comments

fixes https://osv.dev/vulnerability/GHSA-hfq9-hggm-c56q

vladak avatar Nov 12 '24 14:11 vladak

Looks like this needs some work in the suggester:

[INFO] Running org.opengrok.suggest.popular.impl.ChronicleMapAdapterTest
Error:  Tests run: 4, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 2.381 s <<< FAILURE! - in org.opengrok.suggest.popular.impl.ChronicleMapAdapterTest
Error:  org.opengrok.suggest.popular.impl.ChronicleMapAdapterTest.testResize  Time elapsed: 0.254 s  <<< ERROR!
com.thoughtworks.xstream.converters.ConversionException: 
unable to convert node named=org.apache.lucene.util.BytesRef
---- Debugging information ----
message             : unable to convert node named=org.apache.lucene.util.BytesRef
class               : net.openhft.chronicle.map.VanillaChronicleMap
required-type       : net.openhft.chronicle.map.VanillaChronicleMap
converter-type      : net.openhft.xstream.converters.VanillaChronicleMapConverter
line number         : -1
version             : 1.4.21
-------------------------------
	at net.openhft.xstream.converters.AbstractChronicleMapConverter.unmarshal(AbstractChronicleMapConverter.java:133)
	at net.openhft.xstream.converters.VanillaChronicleMapConverter.unmarshal(VanillaChronicleMapConverter.java:28)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:74)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:68)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:52)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:136)
	at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1468)
	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1445)
	at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1334)
	at net.openhft.chronicle.map.JsonSerializer.putAll(JsonSerializer.java:70)
	at net.openhft.chronicle.map.AbstractChronicleMap.putAll(AbstractChronicleMap.java:64)
	at org.opengrok.suggest.popular.impl.chronicle.ChronicleMapAdapter.resize(ChronicleMapAdapter.java:139)
	at org.opengrok.suggest.popular.impl.ChronicleMapAdapterTest.testResize(ChronicleMapAdapterTest.java:88)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

Error:  org.opengrok.suggest.popular.impl.ChronicleMapAdapterTest.dataNotLostAfterResizeTest  Time elapsed: 0.028 s  <<< ERROR!
com.thoughtworks.xstream.converters.ConversionException: 
unable to convert node named=org.apache.lucene.util.BytesRef
---- Debugging information ----
message             : unable to convert node named=org.apache.lucene.util.BytesRef
class               : net.openhft.chronicle.map.VanillaChronicleMap
required-type       : net.openhft.chronicle.map.VanillaChronicleMap
converter-type      : net.openhft.xstream.converters.VanillaChronicleMapConverter
line number         : -1
version             : 1.4.21
-------------------------------
	at net.openhft.xstream.converters.AbstractChronicleMapConverter.unmarshal(AbstractChronicleMapConverter.java:133)
	at net.openhft.xstream.converters.VanillaChronicleMapConverter.unmarshal(VanillaChronicleMapConverter.java:28)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convert(TreeUnmarshaller.java:74)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:68)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.convertAnother(TreeUnmarshaller.java:52)
	at com.thoughtworks.xstream.core.TreeUnmarshaller.start(TreeUnmarshaller.java:136)
	at com.thoughtworks.xstream.core.AbstractTreeMarshallingStrategy.unmarshal(AbstractTreeMarshallingStrategy.java:32)
	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1468)
	at com.thoughtworks.xstream.XStream.unmarshal(XStream.java:1445)
	at com.thoughtworks.xstream.XStream.fromXML(XStream.java:1334)
	at net.openhft.chronicle.map.JsonSerializer.putAll(JsonSerializer.java:70)
	at net.openhft.chronicle.map.AbstractChronicleMap.putAll(AbstractChronicleMap.java:64)
	at org.opengrok.suggest.popular.impl.chronicle.ChronicleMapAdapter.resize(ChronicleMapAdapter.java:139)
	at org.opengrok.suggest.popular.impl.ChronicleMapAdapterTest.dataNotLostAfterResizeTest(ChronicleMapAdapterTest.java:67)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

vladak avatar Nov 12 '24 14:11 vladak

It seems that the recent version of xtream got more strict in terms of (de)serialization and now demands explicit converter, in our case for the Lucene's BytesRef. The converter needs to be implemented and plugged into xstream using the registerConverter() API as described on http://x-stream.github.io/javadoc/index.html

Looking at com.thoughtworks.xstream.converters.extended no such thing is there.

Some inspiration can be found e.g. on https://github.com/x-stream/xstream/issues/293

vladak avatar Nov 13 '24 15:11 vladak

The XStream object is created inside ChronicleMap code and there does not seem to be a way how to access it or modify its converters. This would be possible if the putAll() method allowed to pass the converters to the JsonSerializer in AbstractChronicleMap/VanillaChronicleMap however this is not the case as it uses just the empty list (decompiled code) :

default void putAll(File fromFile) throws IOException {
        synchronized(this) {
            JsonSerializer.putAll(fromFile, this, Collections.emptyList());
        }
    }

vladak avatar Nov 13 '24 15:11 vladak