languagetool [en] Maven broken build, LanguageTool-20240216-snapshot.zip testrules error

From Maven:

[INFO]
[ERROR] Errors:
[ERROR]   DynamicMorfologikLanguageTest.test:35 » Runtime java.lang.RuntimeException: Could not check sentence (language: Testlang): <sentcontent>test</sentcontent>
[INFO]
[ERROR] Tests run: 115, Failures: 0, Errors: 1, Skipped: 8
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for LanguageTool 6.4-SNAPSHOT:
[INFO]
[INFO] LanguageTool ....................................... SUCCESS [  0.114 s]
[INFO] LanguageTool Style and Grammar Checker Core ........ SUCCESS [ 12.641 s]
[INFO] English module for LanguageTool .................... FAILURE [ 43.148 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  56.166 s
[INFO] Finished at: 2024-02-17T09:43:38Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0:test (default-test) on project language-en:

Testrules (first part) for standalone LanguageTool 2024-02-16:

<snip>\LanguageTool-6.4-SNAPSHOT>testrules en
Running XML pattern tests...
LanguageTool version 6.4-SNAPSHOT (2024-02-16 18:55:08 +0100, a71f133)
Known languages: [Arabic, Asturian, Belarusian, Breton, Catalan, Catalan (Valencian), Catalan (Balearic), Danish, German, German (Germany), German (Austria), German (Swiss), Simple German, Greek, English, English (US), English (GB), English (Australian), English (Canadian), English (New Zealand), English (South African), Esperanto, Spanish, Spanish (voseo), Persian, French, French (Canada), French (Switzerland), French (Belgium), Irish, Galician, Italian, Japanese, Khmer, Dutch, Dutch (Belgium), Polish, Portuguese, Portuguese (Portugal), Portuguese (Brazil), Portuguese (Angola preAO), Portuguese (Moçambique preAO), Romanian, Russian, Slovak, Slovenian, Swedish, Tamil, Tagalog, Ukrainian, Chinese, Crimean Tatar, Testlanguage]
Running XML validation for en/grammar.xml...
Running XML validation for en/style.xml...
Running XML validation for /org/languagetool/rules/en/remote-rule-filters.xml...
Running pattern rule tests for English (org.languagetool.language.English)...
Exception in thread "main" java.lang.RuntimeException: Could not activate rules
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:319)
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:278)
        at org.languagetool.MultiThreadedJLanguageTool.<init>(MultiThreadedJLanguageTool.java:94)
        at org.languagetool.MultiThreadedJLanguageTool.<init>(MultiThreadedJLanguageTool.java:84)
        at org.languagetool.MultiThreadedJLanguageTool.<init>(MultiThreadedJLanguageTool.java:67)
        at org.languagetool.MultiThreadedJLanguageTool.<init>(MultiThreadedJLanguageTool.java:51)
        at org.languagetool.rules.patterns.PatternRuleTest.createToolForTesting(PatternRuleTest.java:210)
        at org.languagetool.rules.patterns.PatternRuleTest.runTestForLanguage(PatternRuleTest.java:192)
        at org.languagetool.rules.patterns.PatternRuleTest.runGrammarRulesFromXmlTestIgnoringLanguages(PatternRuleTest.java:184)
        at org.languagetool.rules.patterns.PatternRuleTest.main(PatternRuleTest.java:927)
Caused by: java.io.IOException: Cannot load or parse input stream of '/org/languagetool/rules/en/grammar.xml'
        at org.languagetool.rules.patterns.PatternRuleLoader.getRules(PatternRuleLoader.java:79)
        at org.languagetool.Language.getPatternRules(Language.java:705)
        at org.languagetool.JLanguageTool.activateDefaultPatternRules(JLanguageTool.java:624)
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:312)
        ... 9 more
Caused by: java.lang.RuntimeException: Could not create filter class using constructor public org.languagetool.rules.en.EnglishSuppressMisspelledSuggestionsFilter() throws java.io.IOException

Feb 17 '24 09:02 MikeUnwalla

I am unable to reproduce the error that you received:

 Results:
[INFO] 
[WARNING] Tests run: 115, Failures: 0, Errors: 0, Skipped: 8
[INFO] 
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for LanguageTool 6.4-SNAPSHOT:
[INFO] 
[INFO] LanguageTool ....................................... SUCCESS [  2.614 s]
[INFO] LanguageTool Style and Grammar Checker Core ........ SUCCESS [02:18 min]
[INFO] English module for LanguageTool .................... SUCCESS [05:33 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  07:59 min
[INFO] Finished at: 2024-02-17T11:34:41+01:00
[INFO] ------------------------------------------------------------------------

Caused by: java.io.IOException: Cannot load or parse input stream of '/org/languagetool/rules/en/grammar.xml' ^ This typically indicates a syntax error in the XML code. Without knowing much more about what you are seeing, I can only suggest that the surefire-report for EnglishPatternRuleTest might indicate where the error occurs (if there is indeed a syntax error).

Feb 17 '24 10:02 evan-defran-lt

I tried Maven again. This is (I think) the applicable Maven message:

[INFO] Running org.languagetool.DynamicMorfologikLanguageTest
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.051 s <<< FAILURE! - in org.languagetool.DynamicMorfologikLanguageTest
[ERROR] org.languagetool.DynamicMorfologikLanguageTest.test  Time elapsed: 0.05 s  <<< ERROR!

I attach org.languagetool.DynamicMorfologikLanguageTest.txt: org.languagetool.DynamicMorfologikLanguageTest.txt

Feb 17 '24 12:02 MikeUnwalla

@jaumeortola, do you know anything about this?

Feb 19 '24 08:02 evan-defran-lt

I cannot reproduce the problem on Linux. According to the attached file, it is something related to file paths on Windows: Caused by: java.nio.file.InvalidPathException: Illegal char <:> at index 65: ...

@MikeUnwalla Have you tried mvn clean install? This will run tests for all languages. You can run mvn clean install -DskipTests to skip the tests.

Feb 19 '24 09:02 jaumeortola

@jaumeortola, I used: mvn clean --projects languagetool-language-modules/en --also-make test

I have got a meeting in a couple of minutes. Let me know if you want me to run mvn clean install -DskipTests , and I will do it in a couple of hours.

Feb 19 '24 09:02 MikeUnwalla

I would try mvn clean install, yes.

Feb 19 '24 09:02 jaumeortola

@jaumeortola, mvn clean install -DskipTests gives a build success.

But, if I do not skip the tests (use: mvn clean install), the build fails. I attach the report: org.languagetool.DynamicMorfologikLanguageTest.txt

Also, don't forget the testrules failure. I downloaded the snapshot, unizpped it, and then got the testrules failure.

Feb 19 '24 11:02 MikeUnwalla

I got the same error

'[ERROR] Errors: [ERROR] DynamicMorfologikLanguageTest.test:35 ? Runtime java.lang.RuntimeException: Could not check sentence (language: Testlang): test'

when execute command 'mvn clean test' for just cloned repository via command 'git clone --depth 5 https://github.com/languagetool-org/languagetool.git'. I used a Windows Command Prompt.

Feb 20 '24 17:02 juzyz

I think that I have found the cause of the problem (for the GUI):

        ... 33 more
Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.nio.file.NoSuchFileException: \en\english.dict

Files that were in 6.3 are not in 6.4 snapshot (2024-02-20):

Feb 21 '24 09:02 MikeUnwalla

Files that were in 6.3 are not in 6.4 snapshot (2024-02-20):

Those files were moved to an external dependency (english-pos-dict). From the directory where you execute java -jar languagetool.jar, they should be in /libs/english-pos-dict.jar. Can you check that this file is there? You could even unzip the file (make a copy, rename it with a zip extension, and unzip), and verify that the files are there.

Feb 21 '24 10:02 jaumeortola

The directory/libsis not in my clone. I did not find english-pos-dict.jar with a Windows search (but Windows search almost useless).

Feb 21 '24 10:02 MikeUnwalla

This file can be found in the snapshots: https://internal1.languagetool.org/snapshots/ (download a snapshot and decompress).

Locally, it can be found at languagetool/languagetool-standalone/target/LanguageTool-6.4-SNAPSHOT/LanguageTool-6.4-SNAPSHOT/ after doing mvn clean install -DskipTests.

Feb 21 '24 10:02 jaumeortola

Your problem is related to file paths on Windows. Later, I will be able to check it on Windows. Maybe @fabrichter or @SteVio89 have some idea.

Caused by: java.nio.file.InvalidPathException: Illegal char <:> at index 65: T:\GitHubClone\languagetool\languagetool-language-modules\en\file:\C:\Users\Mike\.m2\repository\org\languagetool\english-pos-dict\0.3\english-pos-dict-0.3.jar!\org\languagetool\resource\en\hunspell\en_US.dict
	at java.base/sun.nio.fs.WindowsPathParser.normalize(WindowsPathParser.java:182)
	at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:153)
	at java.base/sun.nio.fs.WindowsPathParser.parse(WindowsPathParser.java:77)
	at java.base/sun.nio.fs.WindowsPath.parse(WindowsPath.java:92)
	at java.base/sun.nio.fs.WindowsFileSystem.getPath(WindowsFileSystem.java:232)
	at java.base/java.nio.file.Path.of(Path.java:147)
	at java.base/java.nio.file.Paths.get(Paths.java:69)
	at org.languagetool.rules.spelling.morfologik.MorfologikSpellerRule.initSpellers(MorfologikSpellerRule.java:226)
	at org.languagetool.rules.spelling.morfologik.MorfologikSpellerRule.match(MorfologikSpellerRule.java:127)
	at org.languagetool.JLanguageTool.checkAnalyzedSentence(JLanguageTool.java:1447)
	at org.languagetool.JLanguageTool.access$1600(JLanguageTool.java:73)
	at org.languagetool.JLanguageTool$TextCheckCallable.getOtherRuleMatches(JLanguageTool.java:2051)
	... 39 more

Feb 21 '24 10:02 jaumeortola

The problem is not only related to file paths on Windows. There are testrules errors with the snapshots.

Although english-pos-dict.jar is in the GUI that I made with Maven, testrules gives the error about the missing file \en\english.dict:

T:\GitHubClone\languagetool\languagetool-standalone\target\LanguageTool-6.4-SNAPSHOT\LanguageTool-6.4-SNAPSHOT>testrules en
Running XML pattern tests...
LanguageTool version 6.4-SNAPSHOT (2024-02-21 10:56:43 +0000, eb53325)
Known languages: [Arabic, Asturian, Belarusian, Breton, Catalan, Catalan (Valencian), Catalan (Balearic), Danish, German, German (Germany), German (Austria), German (Swiss), Simple German, Greek, English, English (US), English (GB), English (Australian), English (Canadian), English (New Zealand), English (South African), Esperanto, Spanish, Spanish (voseo), Persian, French, French (Canada), French (Switzerland), French (Belgium), Irish, Galician, Italian, Japanese, Khmer, Dutch, Dutch (Belgium), Polish, Portuguese, Portuguese (Portugal), Portuguese (Brazil), Portuguese (Angola preAO), Portuguese (Moçambique preAO), Romanian, Russian, Slovak, Slovenian, Swedish, Tamil, Tagalog, Ukrainian, Chinese, Crimean Tatar, Testlanguage]
Running XML validation for en/grammar.xml...
Running XML validation for en/style.xml...
Running XML validation for /org/languagetool/rules/en/remote-rule-filters.xml...
Running pattern rule tests for English (org.languagetool.language.English)...
Exception in thread "main" java.lang.RuntimeException: Could not activate rules
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:319)
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:278)
        at org.languagetool.MultiThreadedJLanguageTool.<init>(MultiThreadedJLanguageTool.java:94)
        at org.languagetool.MultiThreadedJLanguageTool.<init>(MultiThreadedJLanguageTool.java:84)
        at org.languagetool.MultiThreadedJLanguageTool.<init>(MultiThreadedJLanguageTool.java:67)
        at org.languagetool.MultiThreadedJLanguageTool.<init>(MultiThreadedJLanguageTool.java:51)
        at org.languagetool.rules.patterns.PatternRuleTest.createToolForTesting(PatternRuleTest.java:210)
        at org.languagetool.rules.patterns.PatternRuleTest.runTestForLanguage(PatternRuleTest.java:192)
        at org.languagetool.rules.patterns.PatternRuleTest.runGrammarRulesFromXmlTestIgnoringLanguages(PatternRuleTest.java:184)
        at org.languagetool.rules.patterns.PatternRuleTest.main(PatternRuleTest.java:927)
Caused by: java.io.IOException: Cannot load or parse input stream of '/org/languagetool/rules/en/grammar.xml'
        at org.languagetool.rules.patterns.PatternRuleLoader.getRules(PatternRuleLoader.java:79)
        at org.languagetool.Language.getPatternRules(Language.java:705)
        at org.languagetool.JLanguageTool.activateDefaultPatternRules(JLanguageTool.java:624)
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:312)
        ... 9 more
Caused by: java.lang.RuntimeException: Could not create filter class using constructor public org.languagetool.rules.en.EnglishSuppressMisspelledSuggestionsFilter() throws java.io.IOException
        at org.languagetool.rules.patterns.RuleFilterCreator.getFilter(RuleFilterCreator.java:54)
        at org.languagetool.rules.patterns.XMLRuleHandler.setRuleFilter(XMLRuleHandler.java:684)
        at org.languagetool.rules.patterns.PatternRuleHandler.createRules(PatternRuleHandler.java:779)
        at org.languagetool.rules.patterns.PatternRuleHandler.createRules(PatternRuleHandler.java:806)
        at org.languagetool.rules.patterns.PatternRuleHandler.createRules(PatternRuleHandler.java:806)
        at org.languagetool.rules.patterns.PatternRuleHandler.createRules(PatternRuleHandler.java:806)
        at org.languagetool.rules.patterns.PatternRuleHandler.endElement(PatternRuleHandler.java:502)
        at java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:618)
        at java.xml/com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.endNamespaceScope(XMLDTDValidator.java:1985)
        at java.xml/com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.handleEndElement(XMLDTDValidator.java:1934)
        at java.xml/com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.endElement(XMLDTDValidator.java:865)
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1728)
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2899)
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:605)
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:542)
        at java.xml/com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:889)
        at java.xml/com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:825)
        at java.xml/com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
        at java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1224)
        at java.xml/com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:637)
        at java.xml/com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:326)
        at java.xml/javax.xml.parsers.SAXParser.parse(SAXParser.java:197)
        at org.languagetool.rules.patterns.PatternRuleLoader.getRules(PatternRuleLoader.java:76)
        ... 12 more
Caused by: java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:79)
        at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:483)
        at org.languagetool.rules.patterns.RuleFilterCreator.getFilter(RuleFilterCreator.java:47)
        ... 34 more
Caused by: java.lang.ExceptionInInitializerError
        at org.languagetool.language.English.createDefaultTagger(English.java:120)
        at org.languagetool.Language.getTagger(Language.java:440)
        at org.languagetool.rules.AbstractSuppressMisspelledSuggestionsFilter.<init>(AbstractSuppressMisspelledSuggestionsFilter.java:39)
        at org.languagetool.rules.en.EnglishSuppressMisspelledSuggestionsFilter.<init>(EnglishSuppressMisspelledSuggestionsFilter.java:29)
        at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:67)
        ... 37 more
Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.nio.file.NoSuchFileException: \en\english.dict
        at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:5020)
        at org.languagetool.rules.spelling.morfologik.MorfologikSpeller.getDictionaryWithCaching(MorfologikSpeller.java:97)
        at org.languagetool.tagging.BaseTagger.<init>(BaseTagger.java:88)
        at org.languagetool.tagging.en.EnglishTagger.<init>(EnglishTagger.java:42)
        at org.languagetool.tagging.en.EnglishTagger.<clinit>(EnglishTagger.java:38)
        ... 42 more
Caused by: java.nio.file.NoSuchFileException: \en\english.dict
        at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:85)
        at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
        at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
        at java.base/sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:236)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:380)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:432)
        at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:422)
        at java.base/java.nio.file.Files.newInputStream(Files.java:160)
        at morfologik.stemming.Dictionary.read(Dictionary.java:63)
        at org.languagetool.rules.spelling.morfologik.MorfologikSpeller$1.load(MorfologikSpeller.java:57)
        at org.languagetool.rules.spelling.morfologik.MorfologikSpeller$1.load(MorfologikSpeller.java:50)
        at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3571)
        at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2313)
        at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2190)
        at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2080)
        at com.google.common.cache.LocalCache.get(LocalCache.java:4012)
        at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4035)
        at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5011)
        at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:5018)
        ... 46 more
Running disambiguator rule tests...
Running disambiguation tests for English...
Exception in thread "main" java.lang.RuntimeException: Could not activate rules
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:319)
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:278)
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:338)
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:244)
        at org.languagetool.tagging.disambiguation.rules.DisambiguationRuleTest.testDisambiguationRulesFromXML(DisambiguationRuleTest.java:71)
        at org.languagetool.tagging.disambiguation.rules.DisambiguationRuleTest.main(DisambiguationRuleTest.java:246)
Caused by: java.io.IOException: Cannot load or parse input stream of '/org/languagetool/rules/en/grammar.xml'
        at org.languagetool.rules.patterns.PatternRuleLoader.getRules(PatternRuleLoader.java:79)
        at org.languagetool.Language.getPatternRules(Language.java:705)
        at org.languagetool.JLanguageTool.activateDefaultPatternRules(JLanguageTool.java:624)
        at org.languagetool.JLanguageTool.<init>(JLanguageTool.java:312)
        ... 5 more
Caused by: java.lang.RuntimeException: Could not create filter class using constructor public org.languagetool.rules.en.EnglishSuppressMisspelledSuggestionsFilter() throws java.io.IOException
        at org.languagetool.rules.patterns.RuleFilterCreator.getFilter(RuleFilterCreator.java:54)
        at org.languagetool.rules.patterns.XMLRuleHandler.setRuleFilter(XMLRuleHandler.java:684)
        at org.languagetool.rules.patterns.PatternRuleHandler.createRules(PatternRuleHandler.java:779)
        at org.languagetool.rules.patterns.PatternRuleHandler.createRules(PatternRuleHandler.java:806)
        at org.languagetool.rules.patterns.PatternRuleHandler.createRules(PatternRuleHandler.java:806)
        at org.languagetool.rules.patterns.PatternRuleHandler.createRules(PatternRuleHandler.java:806)
        at org.languagetool.rules.patterns.PatternRuleHandler.endElement(PatternRuleHandler.java:502)
        at java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.endElement(AbstractSAXParser.java:618)
        at java.xml/com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.endNamespaceScope(XMLDTDValidator.java:1985)
        at java.xml/com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.handleEndElement(XMLDTDValidator.java:1934)
        at java.xml/com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.endElement(XMLDTDValidator.java:865)
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanEndElement(XMLDocumentFragmentScannerImpl.java:1728)
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2899)
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:605)
        at java.xml/com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:542)
        at java.xml/com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:889)
        at java.xml/com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:825)
        at java.xml/com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
        at java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1224)
        at java.xml/com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:637)
        at java.xml/com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl.parse(SAXParserImpl.java:326)
        at java.xml/javax.xml.parsers.SAXParser.parse(SAXParser.java:197)
        at org.languagetool.rules.patterns.PatternRuleLoader.getRules(PatternRuleLoader.java:76)
        ... 8 more
Caused by: java.lang.reflect.InvocationTargetException
        at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:79)
        at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:483)
        at org.languagetool.rules.patterns.RuleFilterCreator.getFilter(RuleFilterCreator.java:47)
        ... 30 more
Caused by: java.lang.ExceptionInInitializerError
        at org.languagetool.language.English.createDefaultTagger(English.java:120)
        at org.languagetool.Language.getTagger(Language.java:440)
        at org.languagetool.rules.AbstractSuppressMisspelledSuggestionsFilter.<init>(AbstractSuppressMisspelledSuggestionsFilter.java:39)
        at org.languagetool.rules.en.EnglishSuppressMisspelledSuggestionsFilter.<init>(EnglishSuppressMisspelledSuggestionsFilter.java:29)
        at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:67)
        ... 33 more
Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.nio.file.NoSuchFileException: \en\english.dict
        at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:5020)
        at org.languagetool.rules.spelling.morfologik.MorfologikSpeller.getDictionaryWithCaching(MorfologikSpeller.java:97)
        at org.languagetool.tagging.BaseTagger.<init>(BaseTagger.java:88)
        at org.languagetool.tagging.en.EnglishTagger.<init>(EnglishTagger.java:42)
        at org.languagetool.tagging.en.EnglishTagger.<clinit>(EnglishTagger.java:38)
        ... 38 more
Caused by: java.nio.file.NoSuchFileException: \en\english.dict
        at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:85)
        at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
        at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
        at java.base/sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:236)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:380)
        at java.base/java.nio.file.Files.newByteChannel(Files.java:432)
        at java.base/java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:422)
        at java.base/java.nio.file.Files.newInputStream(Files.java:160)
        at morfologik.stemming.Dictionary.read(Dictionary.java:63)
        at org.languagetool.rules.spelling.morfologik.MorfologikSpeller$1.load(MorfologikSpeller.java:57)
        at org.languagetool.rules.spelling.morfologik.MorfologikSpeller$1.load(MorfologikSpeller.java:50)
        at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3571)
        at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2313)
        at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2190)
        at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2080)
        at com.google.common.cache.LocalCache.get(LocalCache.java:4012)
        at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:4035)
        at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:5011)
        at com.google.common.cache.LocalCache$LocalLoadingCache.getUnchecked(LocalCache.java:5018)
        ... 42 more
Running XML bitext pattern tests...
Bitext pattern tests successful.
Validating false-friends.xml...
Validation successfully finished.

Feb 21 '24 11:02 MikeUnwalla

I can reproduce the error on Windows, and it can be easily solved by disabling or removing the test in en/DynamicMorfologikLanguageTest.java. Does it mean that DynamicMorfologikLanguage.java doesn't work properly on Windows? Maybe nobody uses a server with configuration on Windows. Is it okay to just disable the test? Do we need to address the problem? @fabrichter @SteVio89

Feb 23 '24 06:02 jaumeortola

For the record, I do not use a server with configuration on Windows. Thus, the defects cannot only be related to a server with configuration on Windows. The problem with standalone LT cannot be related to a server with configutation on Windows because no server is involved (download, unzip, testrules, fail).

Feb 23 '24 08:02 MikeUnwalla

I disabled the test here. Please check if it solves the issue for you. https://github.com/languagetool-org/languagetool/commit/b5c89420fded0fc69e26a7343c4ae546a973a95c

The problem with standalone LT cannot be related to a server with configutation on Windows because no server is involved (download, unzip, testrules, fail).

I couldn't reproduce the problem with standalone LT (downloading a snapshot).

Feb 23 '24 09:02 jaumeortola

My version of Windows: Windows 10 Pro, version 22H2.

I synched my repository to get your change.

When I run mvn clean install, I get error messages about the command line version (as best I remember, I did not get these messages 2 days ago). Surefire report attached: org.languagetool.commandline.MainTest.txt

I ran mvn clean install -DskipTests to make a new GUI. Testrules fails: Caused by: com.google.common.util.concurrent.UncheckedExecutionException: java.nio.file.NoSuchFileException: \en\english.dict

For the latest snaphot (https://internal1.languagetool.org/snapshots/LanguageTool-20240222-snapshot.zip,) testrules gives me an error message: Caused by: java.nio.file.NoSuchFileException: \en\english.dict Given that you made the change this morning, there is some weirdness going on. Why don't you get a testrules error message for the snapshot?

Feb 23 '24 11:02 MikeUnwalla

@jaumeortola,

Is the testrules error caused by testrules looking for files that have been moved? (And thus, the solution is to remove the now redundant checks for the existence of the files?)

My reasoning: I downloaded and unzipped https://internal1.languagetool.org/snapshots/LanguageTool-20240225-snapshot.zip. I unzipped english-pos-dict.jar and put the files into \org\languagetool\resource\en. Except for an unrelated error, testrules is happy:

T:<snip>\LanguageTool-6.4-SNAPSHOT>testrules en
Running XML pattern tests...
LanguageTool version 6.4-SNAPSHOT (2024-02-25 18:59:28 +0100, 77fff17)
Known languages: [Arabic, Asturian, Belarusian, Breton, Catalan, Catalan (Valencian), Catalan (Balearic), Danish, German, German (Germany), German (Austria), German (Swiss), Simple German, Greek, English, English (US), English (GB), English (Australian), English (Canadian), English (New Zealand), English (South African), Esperanto, Spanish, Spanish (voseo), Persian, French, French (Canada), French (Switzerland), French (Belgium), Irish, Galician, Italian, Japanese, Khmer, Dutch, Dutch (Belgium), Polish, Portuguese, Portuguese (Portugal), Portuguese (Brazil), Portuguese (Angola preAO), Portuguese (Moçambique preAO), Romanian, Russian, Slovak, Slovenian, Swedish, Tamil, Tagalog, Ukrainian, Chinese, Crimean Tatar, Testlanguage]
Running XML validation for en/grammar.xml...
Running XML validation for en/style.xml...
Running XML validation for /org/languagetool/rules/en/remote-rule-filters.xml...
Running pattern rule tests for English (org.languagetool.language.English)...
Check rule id uniqueness for English...
Loaded 6117 rules...
Check that sentence start tag is not included in <marker>....
Check that <unify-ignore> is not at the start of <unify>....
Check parenthesis and back references in synthesis matches...
Checking regexp syntax of 6088 rules for English...
Checking messages for 'TBD' etc of 6088 rules for English...
Checking example sentences of 6088 rules for English...
Testing rule 100...
Exception in thread "main" java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error analyzing sentence: '<S> Children[child/NNS,B-NP-plural|E-NP-plural] are[be/VBP,B-VP] also[also/RB,B-ADJP] more[more/RP,many/JJR,much/JJR,I-ADJP] susceptible[susceptible/JJ,I-ADJP],[,/,,,/PCT,,/,,O] with[with/IN,with/RP,B-PP] two[two/CD,two/JJ,two/NN,B-NP-singular|E-NP-singular]-[-/null,O] to[to/IN,to/TO,B-VP] four-year-olds[four-year-old/NNS,four-year-olds/NNS,B-NP-plural|E-NP-plural] having[having/NN:UN,have/VBG,B-VP] the[the/DT,B-NP-plural] highest[high/JJS,I-NP-plural] rates[rate/NNS,rate/VBZ,E-NP-plural] of[of/IN,B-PP] infection[infection/NN:UN,B-NP-singular|E-NP-singular].[./.,</S>./PCT,<P/>,O]' with rule EN_SPLIT_WORDS_HYPHEN[1]
        at org.languagetool.rules.patterns.PatternRuleTest.testGrammarRulesFromXML(PatternRuleTest.java:549)
        at org.languagetool.rules.patterns.PatternRuleTest.runTestForLanguage(PatternRuleTest.java:202)
        at org.languagetool.rules.patterns.PatternRuleTest.runGrammarRulesFromXmlTestIgnoringLanguages(PatternRuleTest.java:184)
        at org.languagetool.rules.patterns.PatternRuleTest.main(PatternRuleTest.java:927)
Caused by: java.util.concurrent.ExecutionException: java.lang.RuntimeException: Error analyzing sentence: '<S> Children[child/NNS,B-NP-plural|E-NP-plural] are[be/VBP,B-VP] also[also/RB,B-ADJP] more[more/RP,many/JJR,much/JJR,I-ADJP] susceptible[susceptible/JJ,I-ADJP],[,/,,,/PCT,,/,,O] with[with/IN,with/RP,B-PP] two[two/CD,two/JJ,two/NN,B-NP-singular|E-NP-singular]-[-/null,O] to[to/IN,to/TO,B-VP] four-year-olds[four-year-old/NNS,four-year-olds/NNS,B-NP-plural|E-NP-plural] having[having/NN:UN,have/VBG,B-VP] the[the/DT,B-NP-plural] highest[high/JJS,I-NP-plural] rates[rate/NNS,rate/VBZ,E-NP-plural] of[of/IN,B-PP] infection[infection/NN:UN,B-NP-singular|E-NP-singular].[./.,</S>./PCT,<P/>,O]' with rule EN_SPLIT_WORDS_HYPHEN[1]
        at java.base/java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.base/java.util.concurrent.FutureTask.get(FutureTask.java:191)
        at org.languagetool.rules.patterns.PatternRuleTest.testGrammarRulesFromXML(PatternRuleTest.java:547)
        ... 3 more
Caused by: java.lang.RuntimeException: Error analyzing sentence: '<S> Children[child/NNS,B-NP-plural|E-NP-plural] are[be/VBP,B-VP] also[also/RB,B-ADJP] more[more/RP,many/JJR,much/JJR,I-ADJP] susceptible[susceptible/JJ,I-ADJP],[,/,,,/PCT,,/,,O] with[with/IN,with/RP,B-PP] two[two/CD,two/JJ,two/NN,B-NP-singular|E-NP-singular]-[-/null,O] to[to/IN,to/TO,B-VP] four-year-olds[four-year-old/NNS,four-year-olds/NNS,B-NP-plural|E-NP-plural] having[having/NN:UN,have/VBG,B-VP] the[the/DT,B-NP-plural] highest[high/JJS,I-NP-plural] rates[rate/NNS,rate/VBZ,E-NP-plural] of[of/IN,B-PP] infection[infection/NN:UN,B-NP-singular|E-NP-singular].[./.,</S>./PCT,<P/>,O]' with rule EN_SPLIT_WORDS_HYPHEN[1]
        at org.languagetool.rules.patterns.PatternRuleMatcher.match(PatternRuleMatcher.java:103)
        at org.languagetool.rules.patterns.PatternRule.match(PatternRule.java:218)
        at org.languagetool.rules.patterns.PatternRuleTest.match(PatternRuleTest.java:843)
        at org.languagetool.rules.patterns.PatternRuleTest.testCorrectSentences(PatternRuleTest.java:811)
        at org.languagetool.rules.patterns.PatternRuleTest.lambda$testGrammarRulesFromXML$2(PatternRuleTest.java:536)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: java.lang.NullPointerException: Cannot invoke "org.languagetool.rules.spelling.morfologik.MorfologikMultiSpeller.isMisspelled(String)" because "speller" is null
        at org.languagetool.rules.spelling.morfologik.MorfologikSpellerRule.isMisspelled(MorfologikSpellerRule.java:291)
        at org.languagetool.rules.spelling.morfologik.MorfologikSpellerRule.isMisspelled(MorfologikSpellerRule.java:278)
        at org.languagetool.rules.AbstractSuppressMisspelledSuggestionsFilter.isMisspelled(AbstractSuppressMisspelledSuggestionsFilter.java:83)
        at org.languagetool.rules.AbstractSuppressMisspelledSuggestionsFilter.acceptRuleMatch(AbstractSuppressMisspelledSuggestionsFilter.java:54)
        at org.languagetool.rules.patterns.RuleFilterEvaluator.runFilter(RuleFilterEvaluator.java:46)
        at org.languagetool.rules.patterns.PatternRuleMatcher.createRuleMatch(PatternRuleMatcher.java:195)
        at org.languagetool.rules.patterns.PatternRuleMatcher.lambda$match$1(PatternRuleMatcher.java:86)
        at org.languagetool.rules.patterns.AbstractPatternRulePerformer.matchFrom(AbstractPatternRulePerformer.java:161)
        at org.languagetool.rules.patterns.AbstractPatternRulePerformer.doMatch(AbstractPatternRulePerformer.java:75)
        at org.languagetool.rules.patterns.PatternRuleMatcher.match(PatternRuleMatcher.java:85)
        ... 8 more
Testing rule 200...
Testing rule 300...
<snip>
Testing rule 5700...
Testing rule 5800...
Testing rule 5900...
Running disambiguator rule tests...
Running disambiguation tests for English...
100...
200...
300...
400...
500...
600...
700...
800...
900...
1000...
1003 rules tested (3968ms)
Disambiguator tests successful.
Running XML bitext pattern tests...
Bitext pattern tests successful.
Validating false-friends.xml...
Validation successfully finished.

Feb 26 '24 09:02 MikeUnwalla

@jaumeortola wrote: I couldn't reproduce the problem with standalone LT (downloading a snapshot).

With LT snapshot 2024-02-27, the problem remains: testrules fails. The disambiguation rules and the grammar rules are not checked. Thus, if I make a rule, I cannot be sure that the rule is correct.

@languagetool-org/developers, can you reproduce the testrules problem on Windows?

Feb 28 '24 08:02 MikeUnwalla

@MikeUnwalla I get the error mentioned in the issue #10397 under Windows (using mvn clean test).

Mar 14 '24 14:03 milekpl

@MikeUnwalla: Does the problem persist? I asked a couple of people to run the tests on Windows, but they couldn't reproduce the problem.

Mar 14 '24 15:03 jaumeortola

Could this be related to the version of Java? @milekpl which version do you use?

Mar 14 '24 15:03 danielnaber

@danielnaber semeru (IBM JDK version 19.0.2). I have also openjdk 21 installed but it didn't work for me. Which version should I use?

Mar 14 '24 16:03 milekpl

Could you try whether it works with Java 11?

Mar 14 '24 16:03 danielnaber

@danielnaber it does! mvn clean test and mvn clean install work. But testrules does NOT (I made sure it uses Java 11). Here's my stderr log;

error_log.txt

Mar 14 '24 21:03 milekpl

and the file english-pos-dict.jar is found in my libs dir.

Mar 14 '24 21:03 milekpl

@jaumeortola, sorry for my late reply. I was on vacation.

With snapshot 2024-03-21, I get the testrules error message.

Mar 22 '24 11:03 MikeUnwalla

Java 8 is on my computer. @danielnaber wrote about Java 11.

@jaumeortola, which version of Java is necessary? 'The three-minute introduction' on https://dev.languagetool.org/development-overview tells me to download the standalone version of LT, but it does not tell me about Java.

Mar 22 '24 13:03 MikeUnwalla

Fixed in LanguageTool 6.4 (https://languagetool.org/download/LanguageTool-6.4.zip). Thank you.

Mar 29 '24 12:03 MikeUnwalla

languagetool languagetool copied to clipboard

[en] Maven broken build, LanguageTool-20240216-snapshot.zip testrules error

languagetool
languagetool copied to clipboard