exist icon indicating copy to clipboard operation
exist copied to clipboard

[upgrade] Upgrade to Saxon v11

Open dizzzz opened this issue 3 years ago • 5 comments

Saxon10 introduces some significant API changes. This PR consists of

  • upgrade to saxon v10.x
  • upgrade exist-salon-regex (separate PR)
  • code changes to match updated APIs

with the regex lib update, issues appear in Saxon, looks a bit similar to https://saxonica.plan.io/issues/4684

Error 
  SXXP0003   Error reported by XML parser: Unable to parse document: Unable to parse document
java.lang.NullPointerException
	at net.sf.saxon.tree.linked.LinkedTreeBuilder.endDocument(LinkedTreeBuilder.java:175)
	at net.sf.saxon.event.ProxyReceiver.endDocument(ProxyReceiver.java:121)
	at net.sf.saxon.event.ProxyReceiver.endDocument(ProxyReceiver.java:121)
	at net.sf.saxon.event.ProxyReceiver.endDocument(ProxyReceiver.java:121)
	at net.sf.saxon.event.ReceivingContentHandler.endDocument(ReceivingContentHandler.java:270)
	at org.exist.xslt.EXistDbXMLReader.parse(EXistDbXMLReader.java:99)
	at net.sf.saxon.event.Sender.sendSAXSource(Sender.java:439)
	at net.sf.saxon.event.Sender.send(Sender.java:142)
	at net.sf.saxon.style.StylesheetModule.sendStylesheetSource(StylesheetModule.java:153)
	at net.sf.saxon.style.StylesheetModule.loadStylesheetModule(StylesheetModule.java:113)
	at net.sf.saxon.style.UseWhenFilter.processIncludeImport(UseWhenFilter.java:319)
	at net.sf.saxon.style.UseWhenFilter.startElement(UseWhenFilter.java:251)
	at net.sf.saxon.event.Stripper.startElement(Stripper.java:114)
	at net.sf.saxon.event.CommentStripper.startElement(CommentStripper.java:50)
	at net.sf.saxon.event.ReceivingContentHandler.startElement(ReceivingContentHandler.java:379)
	at org.exist.util.serializer.ReceiverToSAX.startElement(ReceiverToSAX.java:105)
	at org.exist.storage.serializers.XIncludeFilter.startElement(XIncludeFilter.java:233)
	at org.exist.storage.serializers.NativeSerializer.serializeToReceiver(NativeSerializer.java:273)
	at org.exist.storage.serializers.NativeSerializer.serializeToReceiver(NativeSerializer.java:275)
	at org.exist.storage.serializers.NativeSerializer.serializeToReceiver(NativeSerializer.java:132)
	at org.exist.storage.serializers.Serializer.toSAX(Serializer.java:873)
	at org.exist.xslt.EXistDbXMLReader.parse(EXistDbXMLReader.java:97)
	at net.sf.saxon.event.Sender.sendSAXSource(Sender.java:439)
	at net.sf.saxon.event.Sender.send(Sender.java:142)
	at net.sf.saxo
Error in 2.xsl:
  XTSE0165  The XML parser reported one validation error
Error 
  SXXP0003   Error reported by XML parser: Unable to parse document: Unable to parse
  document. Caused by org.xml.sax.SAXException: The XML parser reported one validation
  error. Caused by net.sf.saxon.trans.XPathException: The XML parser reported one validation error
Error in 1.xsl:
  XTSE0165  The XML parser reported one validation error

org.xmldb.api.base.XMLDBException: exerr:ERROR Unable to set up transformer: The XML parser reported one validation error [at line 4, column 8]

	at org.exist.xmldb.LocalXPathQueryService.execute(LocalXPathQueryService.java:203)
	at org.exist.xmldb.LocalXPathQueryService.doQuery(LocalXPathQueryService.java:160)
	at org.exist.xmldb.LocalXPathQueryService.lambda$query$0(LocalXPathQueryService.java:126)
	at org.exist.xmldb.function.LocalXmldbFunction.apply(LocalXmldbFunction.java:48)
	at org.exist.xmldb.AbstractLocal.withDb(AbstractLocal.java:265)
	at org.exist.xmldb.LocalXPathQueryService.query(LocalXPathQueryService.java:124)
	at org.exist.xmldb.LocalXPathQueryService.query(LocalXPathQueryService.java:114)
	at org.exist.xquery.TransformTest.execQuery(TransformTest.java:78)
	at org.exist.xquery.TransformTest.transform(TransformTest.java:63)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:33)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:235)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:54)
Caused by: org.exist.xquery.XPathException: exerr:ERROR Unable to set up transformer: The XML parser reported one validation error [at line 4, column 8]
	at org.exist.xquery.functions.transform.Transform.createHandler(Transform.java:400)
	at org.exist.xquery.functions.transform.Transform.eval(Transform.java:197)
	at org.exist.xquery.BasicFunction.eval(BasicFunction.java:73)
	at org.exist.xquery.InternalFunctionCall.eval(InternalFunctionCall.java:62)
	at org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:58)
	at org.exist.xquery.LetExpr.eval(LetExpr.java:110)
	at org.exist.xquery.LetExpr.eval(LetExpr.java:110)
	at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
	at org.exist.xquery.PathExpr.eval(PathExpr.java:279)
	at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71)
	at org.exist.xquery.XQuery.execute(XQuery.java:373)
	at org.exist.xquery.XQuery.execute(XQuery.java:295)
	at org.exist.xmldb.LocalXPathQueryService.execute(LocalXPathQueryService.java:200)
	... 37 more
Caused by: org.xml.sax.SAXException: The XML parser reported one validation error
net.sf.saxon.trans.XPathException: The XML parser reported one validation error
	at net.sf.saxon.event.ReceivingContentHandler.startElement(ReceivingContentHandler.java:389)
	at org.exist.util.serializer.ReceiverToSAX.startElement(ReceiverToSAX.java:105)
	at org.exist.storage.serializers.XIncludeFilter.startElement(XIncludeFilter.java:233)
	at org.exist.storage.serializers.NativeSerializer.serializeToReceiver(NativeSerializer.java:273)
	at org.exist.storage.serializers.NativeSerializer.serializeToReceiver(NativeSerializer.java:275)
	at org.exist.storage.serializers.NativeSerializer.serializeToReceiver(NativeSerializer.java:132)
	at org.exist.storage.serializers.Serializer.toSAX(Serializer.java:873)
	at org.exist.xslt.StylesheetResolverAndCompiler.compileTemplates(StylesheetResolverAndCompiler.java:155)
	at org.exist.xslt.StylesheetResolverAndCompiler.templates(StylesheetResolverAndCompiler.java:105)
	at org.exist.xslt.StylesheetResolverAndCompiler.newTransformerHandler(StylesheetResolverAndCompiler.java:133)
	at org.exist.xquery.functions.transform.Transform.createHandler(Transform.java:390)
	... 49 more
Caused by: net.sf.saxon.trans.XPathException: The XML parser reported one validation error
	at net.sf.saxon.event.Sender.sendSAXSource(Sender.java:483)
	at net.sf.saxon.event.Sender.send(Sender.java:142)
	at net.sf.saxon.style.StylesheetModule.sendStylesheetSource(StylesheetModule.java:153)
	at net.sf.saxon.style.StylesheetModule.loadStylesheetModule(StylesheetModule.java:113)
	at net.sf.saxon.style.UseWhenFilter.processIncludeImport(UseWhenFilter.java:319)
	at net.sf.saxon.style.UseWhenFilter.startElement(UseWhenFilter.java:251)
	at net.sf.saxon.event.Stripper.startElement(Stripper.java:114)
	at net.sf.saxon.event.CommentStripper.startElement(CommentStripper.java:50)
	at net.sf.saxon.event.ReceivingContentHandler.startElement(ReceivingContentHandler.java:379)
	... 59 more

more

[INFO] Running xquery.xquery3.XQuery3Tests
[ERROR] Tests run: 10, Failures: 1, Errors: 1, Skipped: 2, Time elapsed: 39.786 s <<< FAILURE! - in xquery.xinclude.XIncludeTests
[ERROR] XIncludeTests.XInclude applied before XSLT step  Time elapsed: 6.85 s  <<< ERROR!
org.exist.xquery.XPathException: java:java.lang.NullPointerException  [at line 175, column 27]

[ERROR] XIncludeTests.XInclude applied before XSLT step 2  Time elapsed: 0.201 s  <<< FAILURE!
org.opentest4j.MultipleFailuresError: 
Multiple Failures (2 failures)
	org.exist.xquery.XPathException: java:java.lang.NullPointerException  [at line 175, column 27]
	org.exist.xquery.XPathException: java:java.lang.NullPointerException  [at line 175, column 27]

and

[INFO] Running org.exist.xquery.functions.validate.JaxpXsdCatalogTest
Error 
  SXXP0003   Error reported by XML parser: Unable to parse document: Unable to parse document
java.lang.NullPointerException
	at net.sf.saxon.tree.linked.LinkedTreeBuilder.endDocument(LinkedTreeBuilder.java:175)
	at net.sf.saxon.event.ProxyReceiver.endDocument(ProxyReceiver.java:121)
	at net.sf.saxon.event.ProxyReceiver.endDocument(ProxyReceiver.java:121)
	at net.sf.saxon.event.ProxyReceiver.endDocument(ProxyReceiver.java:121)
	at net.sf.saxon.event.ReceivingContentHandler.endDocument(ReceivingContentHandler.java:270)
	at org.exist.xslt.EXistDbXMLReader.parse(EXistDbXMLReader.java:99)
	at net.sf.saxon.event.Sender.sendSAXSource(Sender.java:439)
	at net.sf.saxon.event.Sender.send(Sender.java:142)
	at net.sf.saxon.style.StylesheetModule.sendStylesheetSource(StylesheetModule.java:153)
	at net.sf.saxon.style.StylesheetModule.loadStylesheetModule(StylesheetModule.java:113)
	at net.sf.saxon.style.UseWhenFilter.processIncludeImport(UseWhenFilter.java:319)
	at net.sf.saxon.style.UseWhenFilter.startElement(UseWhenFilter.java:251)
	at net.sf.saxon.event.Stripper.startElement(Stripper.java:114)
	at net.sf.saxon.event.CommentStripper.startElement(CommentStripper.java:50)
	at net.sf.saxon.event.ReceivingContentHandler.startElement(ReceivingContentHandler.java:379)
	at org.exist.util.serializer.ReceiverToSAX.startElement(ReceiverToSAX.java:105)
	at org.exist.storage.serializers.XIncludeFilter.startElement(XIncludeFilter.java:233)
	at org.exist.storage.serializers.NativeSerializer.serializeToReceiver(NativeSerializer.java:270)
	at org.exist.storage.serializers.NativeSerializer.serializeToReceiver(NativeSerializer.java:272)
	at org.exist.storage.serializers.NativeSerializer.serializeToReceiver(NativeSerializer.java:129)
	at org.exist.storage.serializers.Serializer.toSAX(Serializer.java:852)
	at org.exist.xslt.EXistDbXMLReader.parse(EXistDbXMLReader.java:97)
	at net.sf.saxon.event.Sender.sendSAXSource(Sender.java:439)
	at net.sf.saxon.event.Sender.send(Sender.java:142)
	at net.sf.saxo
Error in 2.xsl:
  XTSE0165  The XML parser reported one validation error
Error 
  SXXP0003   Error reported by XML parser: Unable to parse document: Unable to parse
  document. Caused by org.xml.sax.SAXException: The XML parser reported one validation
  error. Caused by net.sf.saxon.trans.XPathException: The XML parser reported one validation error
Error in 1.xsl:
  XTSE0165  The XML parser reported one validation error
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 5.271 s <<< FAILURE! - in org.exist.xquery.TransformTest
[ERROR] org.exist.xquery.TransformTest.transform  Time elapsed: 3.924 s  <<< ERROR!
org.xmldb.api.base.XMLDBException: exerr:ERROR Unable to set up transformer: The XML parser reported one validation error [at line 4, column 8]
	at org.exist.xquery.TransformTest.execQuery(TransformTest.java:78)
	at org.exist.xquery.TransformTest.transform(TransformTest.java:63)
Caused by: org.exist.xquery.XPathException: exerr:ERROR Unable to set up transformer: The XML parser reported one validation error [at line 4, column 8]
	at org.exist.xquery.TransformTest.execQuery(TransformTest.java:78)
	at org.exist.xquery.TransformTest.transform(TransformTest.java:63)
Caused by: org.xml.sax.SAXException: The XML parser reported one validation error
	at org.exist.xquery.TransformTest.execQuery(TransformTest.java:78)
	at org.exist.xquery.TransformTest.transform(TransformTest.java:63)
Caused by: net.sf.saxon.trans.XPathException: The XML parser reported one validation error
	at org.exist.xquery.TransformTest.execQuery(TransformTest.java:78)
	at org.exist.xquery.TransformTest.transform(TransformTest.java:63)

and

[ERROR] Tests run: 6, Failures: 0, Errors: 5, Skipped: 1, Time elapsed: 5.627 s <<< FAILURE! - in org.exist.xquery.functions.transform.TransformTest
[ERROR] org.exist.xquery.functions.transform.TransformTest.transformReindexTransform  Time elapsed: 3.43 s  <<< ERROR!
org.exist.xquery.XPathException: exerr:ERROR Unable to set up transformer: null [at line 6, column 9]
	at org.exist.xquery.functions.transform.TransformTest.transform1(TransformTest.java:308)
	at org.exist.xquery.functions.transform.TransformTest.transformReindexTransform(TransformTest.java:269)
Caused by: java.lang.NullPointerException
	at org.exist.xquery.functions.transform.TransformTest.transform1(TransformTest.java:308)
	at org.exist.xquery.functions.transform.TransformTest.transformReindexTransform(TransformTest.java:269)

[ERROR] org.exist.xquery.functions.transform.TransformTest.transformReindexTransform_with_comment  Time elapsed: 0.023 s  <<< ERROR!
org.exist.xquery.XPathException: exerr:ERROR Unable to set up transformer: null [at line 6, column 9]
	at org.exist.xquery.functions.transform.TransformTest.transform1(TransformTest.java:308)
	at org.exist.xquery.functions.transform.TransformTest.transformReindexTransform_with_comment(TransformTest.java:279)
Caused by: java.lang.NullPointerException
	at org.exist.xquery.functions.transform.TransformTest.transform1(TransformTest.java:308)
	at org.exist.xquery.functions.transform.TransformTest.transformReindexTransform_with_comment(TransformTest.java:279)

[ERROR] org.exist.xquery.functions.transform.TransformTest.twoNodesCountDescendants  Time elapsed: 0.007 s  <<< ERROR!
org.exist.xquery.XPathException: exerr:ERROR Unable to set up transformer: null [at line 6, column 9]
	at org.exist.xquery.functions.transform.TransformTest.transform_twoNodesCountDescendants(TransformTest.java:361)
	at org.exist.xquery.functions.transform.TransformTest.twoNodesCountDescendants(TransformTest.java:299)
Caused by: java.lang.NullPointerException
	at org.exist.xquery.functions.transform.TransformTest.transform_twoNodesCountDescendants(TransformTest.java:361)
	at org.exist.xquery.functions.transform.TransformTest.twoNodesCountDescendants(TransformTest.java:299)

[ERROR] org.exist.xquery.functions.transform.TransformTest.keys  Time elapsed: 0.327 s  <<< ERROR!
org.exist.xquery.XPathException: exerr:ERROR Unable to set up transformer: null [at line 14, column 8]
	at org.exist.xquery.functions.transform.TransformTest.keys(TransformTest.java:219)
Caused by: java.lang.NullPointerException
	at org.exist.xquery.functions.transform.TransformTest.keys(TransformTest.java:219)

[ERROR] org.exist.xquery.functions.transform.TransformTest.transformReindexTransform_with_two_comments  Time elapsed: 0.005 s  <<< ERROR!
org.exist.xquery.XPathException: exerr:ERROR Unable to set up transformer: null [at line 6, column 9]
	at org.exist.xquery.functions.transform.TransformTest.transform1(TransformTest.java:308)
	at org.exist.xquery.functions.transform.TransformTest.transformReindexTransform_with_two_comments(TransformTest.java:289)
Caused by: java.lang.NullPointerException
	at org.exist.xquery.functions.transform.TransformTest.transform1(TransformTest.java:308)
	at org.exist.xquery.functions.transform.TransformTest.transformReindexTransform_with_two_comments(TransformTest.java:289)

and

WARNING] Tests run: 24, Failures: 0, Errors: 0, Skipped: 6, Time elapsed: 79.486 s - in org.exist.backup.XMLDBRestoreTest
[INFO] 
[INFO] Results:
[INFO] 
[ERROR] Failures: 
[ERROR]   Multiple Failures (2 failures)
	org.exist.xquery.XPathException: java:java.lang.NullPointerException  [at line 175, column 27]
	org.exist.xquery.XPathException: java:java.lang.NullPointerException  [at line 175, column 27]
[ERROR] Errors: 
[ERROR]   java:java.lang.NullPointerException  [at line 175, column 27]
[ERROR]   TransformTest.transform:63->execQuery:78 » XMLDB exerr:ERROR Unable to set up ...
[ERROR]   TransformTest.keys:219 » XPath exerr:ERROR Unable to set up transformer: null ...
[ERROR]   TransformTest.transformReindexTransform:269->transform1:308 » XPath exerr:ERRO...
[ERROR]   TransformTest.transformReindexTransform_with_comment:279->transform1:308 » XPath
[ERROR]   TransformTest.transformReindexTransform_with_two_comments:289->transform1:308 » XPath
[ERROR]   TransformTest.twoNodesCountDescendants:299->transform_twoNodesCountDescendants:361 » XPath
[INFO] 
[ERROR] Tests run: 5031, Failures: 1, Errors: 7, Skipped: 101

dizzzz avatar Sep 23 '21 08:09 dizzzz

@dizzzz Something along these lines came up in today's Community Call - about upgrading from Saxon 9 to 10. Is this superseded by the PRs @adamretter mentioned? Or if this can stay, does it need to wait for eXist 6 to be merged?

joewiz avatar Nov 29 '21 19:11 joewiz

As discussed in todays community call this is considered a breaking change because it may break existing XSL transformations due to breaking changes in Saxon 10.

line-o avatar Nov 29 '21 20:11 line-o

Although, by reading through the change list I was not able to spot a breaking change (source: https://www.saxonica.com/html/documentation10/changes/v10/).

line-o avatar Nov 29 '21 20:11 line-o

see #4093

dizzzz avatar Nov 30 '21 14:11 dizzzz

I remember again... With the current state I have to contact the saxon guys to check on some NPEs; some of them look similar to issues fixed in 9.x

dizzzz avatar Dec 01 '21 09:12 dizzzz