Logos 10 crashes after opening biblical word study for "worship"
Reproduction steps:
- Launch Logos 10 (v36.1.0.6) with wine-devel 9.12 (or 9.16)
- Open a biblical word study for "worship" (some words cause this crash, others don't).
- Expand the Hebrew words section
- Application crashes with the error below
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"Application: Logos.exe\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"CoreCLR Version: 8.0.724.31311\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L".NET Version: 8.0.7\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"Description: The application requested process termination through System.Environment.FailFast.\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"Message: Unhandled exception in async work\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"Description: The process was terminated due to an unhandled exception.System.Globalization.CultureNotFoundException: Culture is not supported. (Parameter 'name')\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"en-us-posix is an invalid culture identifier.\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at System.Globalization.CultureInfo.GetCultureInfo(String name)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at System.Globalization.CultureData.IcuEnumCultures(CultureTypes types)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at Libronix.Utility.CultureInfoUtility.CreateSpecificCulture(LanguageName language)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at Libronix.Globalization.Culture.CreateSpecificCulture()\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at Libronix.Globalization.Culture.get_FormatCultureInfo()\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at Libronix.Globalization.Culture.CreateStringComparer(CaseSensitivity caseSensitivity)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at Libronix.Globalization.Culture.get_StringComparerIgnoreCase()\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at Libronix.DigitalLibrary.StopListUtility.IsStopLemma(LemmaReference l)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at System.Linq.Enumerable.SkipWhileIterator[TSource](IEnumerable`1 source, Func`2 predicate)+MoveNext()\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at Libronix.Utility.EnumerableUtility.TrimEndWhere[T](IEnumerable`1 seq, Func`2 fnPredicate)+MoveNext()\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at Libronix.Utility.EnumerableUtility.ToReadOnlyCollection[T](IEnumerable`1 seq)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at Libronix.DigitalLibrary.StopListUtility.TrimLemmas(IEnumerable`1 lemmas)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at Libronix.DigitalLibrary.Guides.Sections.TranslationSectionUtility.GetJoinedLemmasForTranslation(ReverseInterlinearResource resource, String strWord, ReferenceRange referenceRange, IWorkState state)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at Libronix.DigitalLibrary.Guides.Sections.TranslationSectionUtility.GenerateDataPointsAndSparklineForTranslation(String word, IEnumerable`1 reverseInterlinears, ResourceLists resourceLists, ReferenceRange referenceRange, IWorkState state, ReadOnlyCollection`1& dataPoints, SparklineDat"...
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at Libronix.DigitalLibrary.Guides.Sections.WordLemmaTranslationSectionSource.GenerateData(WordGuideKey keyWord, LibraryCatalog libraryCatalog, ResourceLists resourceLists, ReferenceRange referenceRange, IWorkState state)\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at Libronix.DigitalLibrary.Guides.Sections.WordLemmaTranslationSectionSource.GenerateData(WordGuideKey keyWord, String strResourceOverride, String strResourceAbbreviatedTitle, ResourceManager resourceManager, VersifiedResourceManager versifiedResourceManager, ReverseInterlinearManager "...
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at Libronix.DigitalLibrary.Guides.Sections.TranslationSectionBase.DoWork(TranslationSectionBase section)+MoveNext()\r\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at Libronix.Utility.Threading.AsyncWorkerTask`1.EnumMoveNext()\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L"Stack:\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at System.Environment.FailFast(System.String, System.Exception)\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at Libronix.Utility.ApplicationUtility.FailFast(System.String, System.Exception)\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at Libronix.Utility.Threading.AsyncWorkerTask`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].EnumMoveNext()\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at Libronix.Utility.Threading.AsyncWorkerTask`1[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].ContinueExecution(System.Object)\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at Libronix.Utility.Threading.GroupedThreadPool.ExecuteNextCallback(System.Object)\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at System.Threading.QueueUserWorkItemCallback.Execute()\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at System.Threading.ThreadPoolWorkQueue.Dispatch()\n"
2024-10-05 19:12:07 INFO: 03d0:err:eventlog:ReportEventW L" at System.Threading.PortableThreadPool+WorkerThread.WorkerThreadStart()\n"
See #156, which has some reports of this same issue. Those posting had a few issues going on, so it would be good to keep this open separately.
Another way to crash with the same error is to open the "Reverse Interlinear Pane" in a Bible.
Thanks, that should make it very easy to test.
Commenting for visibility. I also experienced this yesterday evening, but for various other words.
Looking into this in more depth...
Found:
- This appears to be the failing GetCultureInfo call https://github.com/dotnet/runtime/blob/v8.0.7/src/libraries/System.Private.CoreLib/src/System/Globalization/CultureData.Icu.cs#L475
- Which is populated from https://github.com/dotnet/runtime/blob/v8.0.7/src/libraries/System.Private.CoreLib/src/System/Globalization/CultureData.Icu.cs#L453
- (probably) https://github.com/dotnet/runtime/blob/2aade6beb02ea367fd97c4070a4198802fe61c03/src/native/libs/System.Globalization.Native/pal_locale.c#L156
- uloc_countAvailable defined in icu repo
Open questions:
- (unlikely) Is it possible this is a problem with the .NET runtime under wine? Similar to: https://github.com/dotnet/runtime/blob/6d23f64952d0f242c4dcc7cbb230a619be930309/src/native/libs/System.Globalization.Native/pal_locale.m#L720 . There are several issues listed there for en-us-posix https://github.com/dotnet/runtime/issues?q=%22en-us-posix%22%20
- (general) What's a good way to debug Logos w/out violating the terms (https://www.logos.com/terms 4b "Reverse engineer, disassemble, decompile, make any attempt to discover or use the source code of any Services, or make any attempt to “unlock” or circumvent the digital copyright protection of any Services;")
Failed attempts at producing a different result:
- changing language in Logos to Spanish, same backtrace, (en-us-posix)
- set LC_ALL="ja_JP" and LANG="ja_JP.UTF8", same backtrace
- enabled WINEDBG=+keyboard, same result, no relevant logs
- installing locales-all package, same result
- removing generated C/POSIX locale files (DO NOT DO THIS ON YOUR SYSTEM) - still was detected as supported
- tried inspecting running envs for all processes with "Logos" in the name, all include the LANG env var. Checking to make sure it isn't falling back to posix via https://www.gnu.org/software/libc/manual/html_node/Standard-Locales.html
- removing the en_US_POSIX locale from icudtl.dll via
icupkgtool - (long shot, didn't expect to work) copying the en_US_POSIX locale to en-us-posix via
icupkgtool andmv
Observations:
- en-us-posix seems to be the source of a number of bugs in icu, however none of these issues seem to relate to our issue at this time https://unicode-org.atlassian.net/issues/?jql=text%20~%20%22en-us-posix%22
@thw26 would it be possible to document how the icu release was created (or update script)?
I'm stuck regarding looking into this issue due to my inability to get an icu I compile to be loaded by logos. I don't understand how icu shared library is getting generated. I'm not that familiar with windows build systems, but did figure some of it out. New to building on windows but have done a fair bit of development on linux systems. It should be noted I'm using MSVC build tools v2019 and windows 11 to compile. Trying again with linux and gcc..
unknown source
- C:/windows/system32/icu.dll (there is no .dll with roughly that name, nor roughly that size I could find doing a clean build of icu, what is this file?) Looking at icuuc.dll, it looks like icu.dll contains the common icu library (icuuc just forwards to it).
found correlation between:
- icu/icu4c/sources/lib/icuin76.dll and C:/windows/system32/icuin.dll
- icu/icu4c/sources/lib/icuout76.dll and C:/windows/system32/icuout.dll
- icu/icu4c/sources/data.out/build/icudt76l/ and C:/windows/globalization/ICU/*.res
- C:/winows/globalization/ICU/icudtl.dat and icu/icu4c/sources/data/tmp/icudt761.dat
We copied the v75.1 release from https://github.com/unicode-org/icu/releases into https://github.com/FaithLife-Community/icu/releases, but I see that we have icu-win.tar.gz, while I don't see that exact file in the unicode-org repo's release. @thw26, can you add any details to where our file comes from vs what's available in the original repo? It seems like it would be good to rule out any issues that could have gotten introduced by our release.
Once the file is downloaded, we just uncompress it and drop its contents straight into drive_c:
def install_icu_data_files(app=None):
releases_url = "https://api.github.com/repos/FaithLife-Community/icu/releases" # noqa: E501
json_data = network.get_latest_release_data(releases_url)
icu_url = network.get_latest_release_url(json_data)
# icu_tag_name = utils.get_latest_release_version_tag_name(json_data)
if icu_url is None:
logging.critical("Unable to set LogosLinuxInstaller release without URL.") # noqa: E501
return
icu_filename = os.path.basename(icu_url)
network.logos_reuse_download(
icu_url,
icu_filename,
config.MYDOWNLOADS,
app=app
)
drive_c = f"{config.INSTALLDIR}/data/wine64_bottle/drive_c"
utils.untar_file(f"{config.MYDOWNLOADS}/icu-win.tar.gz", drive_c)
The installer checks to see if the ICU license file exists or not before installing the ICU files:
icu_license_path = f"{config.WINEPREFIX}/drive_c/windows/globalization/ICU/LICENSE-ICU.txt" # noqa: E501
if not utils.file_exists(icu_license_path):
wine.install_icu_data_files(app=app)
so you could theoretically short-circuit that process by touching that file, then going back and dropping in any other ICU files after the installation completes to test how they work. Or just simply replace the ones we install with other versions.
Looked into it a little more
Two zip differences (between upstream icu and ours):
- ICU packages 32bit/64 bit separately, we publish both (one in system32, the other in syswow64), good thinking!
- ICU packages with renaming on (all functions smurfed to end in the version number), we don't (corresponding to the --disable-renaming flag on ./configure)
- (trivial) they publish as a .dll, we publish as a .dat (both are supported by icu)
- different directory structure (as noted on the FaithLife-Community/icu release)
- metaZones.rs,timezoneTypes.res,windowsZones.res and zoneinfo64.res copied from build process (not present in upstream package). Guessing there was a need for these, so they were added
- new dll called icu.dll (appears to be a combined implementation, unsure how to build this)
- icuuc.dll doesn't contain implementations, just pointing to icu.dll
$ dumpbin /exports icu-win\windows\syswow64\icuuc.dll
...
534 215 utf8_nextCharSafeBody (forwarded to icu.utf8_nextCharSafeBody)
535 216 utf8_prevCharSafeBody (forwarded to icu.utf8_prevCharSafeBody)
536 217 utrace_format (forwarded to icu.utrace_format)
537 218 utrace_functionName (forwarded to icu.utrace_functionName)
538 219 utrace_getFunctions (forwarded to icu.utrace_getFunctions)
539 21A utrace_getLevel (forwarded to icu.utrace_getLevel)
540 21B utrace_setFunctions (forwarded to icu.utrace_setFunctions)
541 21C utrace_setLevel (forwarded to icu.utrace_setLevel)
542 21D utrace_vformat (forwarded to icu.utrace_vformat)
- icuin.dll doesn't contain implementations either, just points to icu.dll
$ dumpbin /exports icu-win\windows\syswow64\icuin.dll
...
408 197 utrans_openU (forwarded to icu.utrans_openU)
409 198 utrans_register (forwarded to icu.utrans_register)
410 199 utrans_setFilter (forwarded to icu.utrans_setFilter)
411 19A utrans_toRules (forwarded to icu.utrans_toRules)
412 19B utrans_trans (forwarded to icu.utrans_trans)
413 19C utrans_transIncremental (forwarded to icu.utrans_transIncremental)
414 19D utrans_transIncrementalUChars (forwarded to icu.utrans_transIncrementalUChars)
415 19E utrans_transUChars (forwarded to icu.utrans_transUChars)
416 19F utrans_unregisterID (forwarded to icu.utrans_unregisterID)
I'm not familiar enough with cpp or windows build systems to figure out how to build the combined library
We store it in .tar.gz since we have an untar command already made.
utils.untar_file(f"{config.MYDOWNLOADS}/icu-win.tar.gz", drive_c)
Sure I realize that, was more wondering how icu.dll/icuuc.dll and icuin.dll were created. Were they copied from a windows install of Logos?
Checked against my windows logos install and confirmed icu.dll/icuin.dll/icuuc.dll all matched the format we ship. (as documented here https://learn.microsoft.com/en-us/windows/win32/intl/international-components-for-unicode--icu-#version-1903-may-2019-update)
I attempted to open the icudtl.dat that Logos for windows ships with:
- increased the STRING_STORE_SZIE in tools/toolsutil.h by 100x (can't open the file without it)
- building the icupkg tool on linux
$ bin/icupkg ~/QubesIncoming/Win11Logos/icudtl.dat
icupkg: input .dat item name "icudt72l/aa.res" does not start with "icudtl/"
- I would if they were building with ICU version 72 (rather than the 76 we ship with). Unsure if it matters
Tried to run Logos on linux with the dlls from the windows install, failed during logos startup (same thing happens if I use the .dlls from nuget)
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L"Stack:\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Globalization.CalendarData+<>c.<GetCalendarInfo>b__32_0(System.Span`1<Char>, System.String, System.Globalization.CalendarId, System.Globalization.CalendarDataType)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at Interop.CallStringMethod[[System.__Canon, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Globalization.CalendarId, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[System.Globalization.Cal"...
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Globalization.CalendarData.GetCalendarInfo(System.String, System.Globalization.CalendarId, System.Globalization.CalendarDataType, System.String ByRef)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Globalization.CalendarData.IcuLoadCalendarDataFromSystem(System.String, System.Globalization.CalendarId)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Globalization.CalendarData.LoadCalendarDataFromSystemCore(System.String, System.Globalization.CalendarId)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Globalization.CalendarData..ctor(System.String, System.Globalization.CalendarId, Boolean)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Globalization.CultureData.GetCalendar(System.Globalization.CalendarId)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Globalization.CultureData.DateSeparator(System.Globalization.CalendarId)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Globalization.DateTimeFormatInfo.InitializeOverridableProperties(System.Globalization.CultureData, System.Globalization.CalendarId)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Globalization.CultureInfo.get_DateTimeFormat()\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.Controls.DateTimeHelper.GetDateFormat(System.Globalization.CultureInfo)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.Controls.DateTimeHelper.GetCurrentDateFormat()\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.Controls.Calendar..cctor()\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Runtime.CompilerServices.RuntimeHelpers.RunClassConstructor(System.RuntimeTypeHandle)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at MS.Internal.WindowsBase.SecurityHelper.RunClassConstructor(System.Type)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.DependencyProperty.FromName(System.String, System.Type)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.Baml2006.Baml2006Reader.WriteTypeConvertedInstance(Int16, Int32)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.Baml2006.Baml2006Reader.Process_PropertyCustom()\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.Baml2006.Baml2006Reader.Process_OneBamlRecord()\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.Baml2006.Baml2006Reader.Process_BamlRecords()\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.Baml2006.Baml2006Reader.Read()\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.Markup.WpfXamlLoader.TransformNodes(System.Xaml.XamlReader, System.Xaml.XamlObjectWriter, Boolean, Boolean, Boolean, System.Xaml.IXamlLineInfo, System.Xaml.IXamlLineInfoConsumer, MS.Internal.Xaml.Context.XamlContextStack`1<System.Windows.Markup.WpfXamlFrame>, System.W"...
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.Markup.WpfXamlLoader.Load(System.Xaml.XamlReader, System.Xaml.IXamlObjectWriterFactory, Boolean, System.Object, System.Xaml.XamlObjectWriterSettings, System.Uri)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.Markup.WpfXamlLoader.LoadBaml(System.Xaml.XamlReader, Boolean, System.Object, System.Xaml.Permissions.XamlAccessLevel, System.Uri)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.Markup.XamlReader.LoadBaml(System.IO.Stream, System.Windows.Markup.ParserContext, System.Object, Boolean)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.Application.LoadBamlStreamWithSyncInfo(System.IO.Stream, System.Windows.Markup.ParserContext)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at MS.Internal.AppModel.AppModelKnownContentFactory.BamlConverterCore(System.IO.Stream, System.Uri, Boolean, Boolean, Boolean, Boolean, System.Windows.Markup.XamlReader ByRef, Boolean)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at MS.Internal.AppModel.MimeObjectFactory.GetObjectAndCloseStreamCore(System.IO.Stream, MS.Internal.ContentType, System.Uri, Boolean, Boolean, Boolean, Boolean, System.Windows.Markup.XamlReader ByRef, Boolean)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.ResourceDictionary.set_Source(System.Uri)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.Baml2006.WpfSharedBamlSchemaContext+<>c.<Create_BamlProperty_ResourceDictionary_Source>b__344_0(System.Object, System.Object)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.Baml2006.WpfKnownMemberInvoker.SetValue(System.Object, System.Object)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at MS.Internal.Xaml.Runtime.ClrObjectRuntime.SetValue(System.Xaml.XamlMember, System.Object, System.Object)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at MS.Internal.Xaml.Runtime.ClrObjectRuntime.SetValue(System.Object, System.Xaml.XamlMember, System.Object)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at MS.Internal.Xaml.Runtime.PartialTrustTolerantRuntime.SetValue(System.Object, System.Xaml.XamlMember, System.Object)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Xaml.XamlObjectWriter.SetValue(System.Object, System.Xaml.XamlMember, System.Object)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Xaml.XamlObjectWriter.Logic_ApplyPropertyValue(MS.Internal.Xaml.Context.ObjectWriterContext, System.Xaml.XamlMember, System.Object, Boolean)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Xaml.XamlObjectWriter.Logic_DoAssignmentToParentProperty(MS.Internal.Xaml.Context.ObjectWriterContext)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Xaml.XamlObjectWriter.Logic_AssignProvidedValue(MS.Internal.Xaml.Context.ObjectWriterContext)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Xaml.XamlObjectWriter.WriteEndMember()\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Xaml.XamlWriter.WriteNode(System.Xaml.XamlReader)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.Markup.WpfXamlLoader.TransformNodes(System.Xaml.XamlReader, System.Xaml.XamlObjectWriter, Boolean, Boolean, Boolean, System.Xaml.IXamlLineInfo, System.Xaml.IXamlLineInfoConsumer, MS.Internal.Xaml.Context.XamlContextStack`1<System.Windows.Markup.WpfXamlFrame>, System.W"...
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.Markup.WpfXamlLoader.Load(System.Xaml.XamlReader, System.Xaml.IXamlObjectWriterFactory, Boolean, System.Object, System.Xaml.XamlObjectWriterSettings, System.Uri)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.Markup.WpfXamlLoader.LoadBaml(System.Xaml.XamlReader, Boolean, System.Object, System.Xaml.Permissions.XamlAccessLevel, System.Uri)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.Markup.XamlReader.LoadBaml(System.IO.Stream, System.Windows.Markup.ParserContext, System.Object, Boolean)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at System.Windows.Application.LoadComponent(System.Object, System.Uri)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at LDLS4.OurApp.InitializeComponent()\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at LDLS4.OurApp.InitializeAndRun(System.Collections.ObjectModel.ReadOnlyCollection`1<System.String>, Libronix.Utility.Windows.Threading.SingleInstanceManager, Libronix.Utility.Scope)\n"
2024-10-17 17:59:07 INFO: 0110:err:eventlog:ReportEventW L" at LDLS4.OurApp.Main(System.String[])\n"
2024-10-17 17:59:11 DEBUG: Getting wine environment.
Did we use microsoft's icu as https://github.com/FaithLife-Community/LogosLinuxInstaller/issues/22#issuecomment-2155510687 suggests or unicode's?
I think we need to get @tbleher into this conversation.
A victory! Figured out how to compile the icu.dll as windows does! (WIP branch here: https://github.com/ctrlaltf24/microsoft-icu/commit/b3368b7ffb1484cd63c22559a9091fea6a177713 ) (no longer blocked)
Still needed:
- [ ] icuuc.dll and icuin.dll pointing to icu.dll (no idea how they did this)
- [ ] Remove CPP exports from icu.dll (presently this code exports them, while MS Win11's doesn't) (technically works without this, but would be cleaner to remove them. There is a pre-processor flag to remove exporting them, haven't quite figured out where to put it yet)
- [X] Right now the ICU data is loaded from a .dll as built from msbuilt, but we want it to pull from a .dat, we may need a compiler flag for this. Looks like the dll provided by windows neither has a dependency on the icudt.dll nor does it export the symbol. Fixed in https://github.com/ctrlaltf24/microsoft-icu/commit/be13a055844748ec31d8062c310a19e7b53e9476
- [ ] Start debugging and fixing this bug!
Finally tested removing en_US_POSIX from the icudt.dat, and interestingly enough some other locale failed...continuing to debug...
I'm trying to understand the bigger picture here. Maybe someone can help me out. We added separate ICU files because of #22. But now we've discovered that our added ICU files include (and make use of) en-us-posix language/culture, which is unrecognized by .NET shipped with Logos. Am I summarizing correctly?
If so, could it be that we "overshot" somehow and used a version of ICU that is too new for Logos? I see that Windows' current version is 72.1.0.3, while we're using 75.1. Maybe if we went went back to 72.1 it would be in sync better with Logos?
I'm trying to understand the bigger picture here. Maybe someone can help me out. We added separate ICU files because of #22.
Correct (these files are bundled in newer versions of windows
But now we've discovered that our added ICU files include (and make use of)
en-us-posixlanguage/culture, which is unrecognized by.NETshipped with Logos. Am I summarizing correctly?
Almost, I'm not sure what's causing this error specifically. It should be noted upstream windows does package en-us-posix in their icu data. I'm not sure what the difference is between us and windows. For now, I'm swapping the icu dlls in development to assist in the diagnosis of this bug.
If so, could it be that we "overshot" somehow and used a version of ICU that is too new for Logos? I see that Windows' current version is
72.1.0.3, while we're using75.1. Maybe if we went went back to72.1it would be in sync better with Logos?
While that is possible, I'm compiling against window's 72 and still getting errors.
Progress...
Intelligence gathering
- Inspected the icudtl.dat provided as a part of windows, it does ship with EN_US_POSIX, bgc and bgc_IN
Tests ran
- made uloc_countAvailable return 0 to trick it into thinking there are no installed locales -> .NET exception, couldn't handle empty list
- made uloc_countAvailable return 1 -> PASS! "worship" loaded successfully! This isn't a real fix, but does give us hope we can fix it on our end.
- made uloc_countAvailable return 66 (right before bgc is loaded) -> PASS! "worship" loaded successfully! This isn't a real fix.
- made uloc_countAvailable return 67 (including bgc but not bgc_IN) -> FAIL. Earlier hypothesis may be wrong
- compiled microsoft's icu v72 got "bgc is an invalid culture identifier.\r\n" when trying to load worship.
- compiled microsoft's icu v72, used clean windows 11 dat file. still got got "bgc is an invalid culture identifier.\r\n"
- removed bgc from the dat file, it fails on the next one bho, what's going on? Is it somehow stopping at 64 prefixes?
- removed all locales from the res_index besides en_US -> PASS. This isn't a real fix
- ran icupkg against the .dat and noticed it's missing some items. We might (as a workaround) be able to remove en-us-posix from our dat file, although there still would be an underlying issue that remains unknown
$ ./icupkg.exe FaithLife/icudtl.dat
Item lang/ca_ES_VALENCIA.res depends on missing item lang/ca_ES.res
Item region/en_150.res depends on missing item region/en_001.res
Item region/en_AG.res depends on missing item region/en_001.res
Item region/en_AI.res depends on missing item region/en_001.res
Item region/en_AU.res depends on missing item region/en_001.res
...
$ find ~/LogosBible10/data/wine64_bottle/ -iname 'icu*.d*'
/home/nate/LogosBible10/data/wine64_bottle/drive_c/windows/globalization/ICU/icudtl.dat
/home/nate/LogosBible10/data/wine64_bottle/drive_c/windows/syswow64/icu.dll
/home/nate/LogosBible10/data/wine64_bottle/drive_c/windows/syswow64/icuuc.dll
/home/nate/LogosBible10/data/wine64_bottle/drive_c/windows/syswow64/icuin.dll
/home/nate/LogosBible10/data/wine64_bottle/drive_c/windows/system32/icu.dll
/home/nate/LogosBible10/data/wine64_bottle/drive_c/windows/system32/icuuc.dll
/home/nate/LogosBible10/data/wine64_bottle/drive_c/windows/system32/icuin.dll
/home/nate/LogosBible10/data/wine64_bottle/drive_c/users/nate/AppData/Local/Logos/System/icuin52.dll
/home/nate/LogosBible10/data/wine64_bottle/drive_c/users/nate/AppData/Local/Logos/System/icudt52.dll
/home/nate/LogosBible10/data/wine64_bottle/drive_c/users/nate/AppData/Local/Logos/System/icuuc52.dll
/home/nate/LogosBible10/data/wine64_bottle/drive_c/users/nate/AppData/Local/Logos/System/icudtl.dat
It looks like Logos ships its own icudtl.dat file. So I assume that takes precedence over anything we put into C:\windows\{system32,syswow64}. But what does that mean in terms of the #22 "windows is too old" issue? Do our new ICU files allow that test to pass, but then aren't actually used by Logos? This seems weird to me. But maybe this info will give you some :bulb:?
$ ./icupkg.exe FaithLife/icudtl.dat Item lang/ca_ES_VALENCIA.res depends on missing item lang/ca_ES.res Item region/en_150.res depends on missing item region/en_001.res Item region/en_AG.res depends on missing item region/en_001.res Item region/en_AI.res depends on missing item region/en_001.res Item region/en_AU.res depends on missing item region/en_001.res ...
I'm sure I'm showing my ignorance here, but I'm still trying to understand all the relevant details.
- I see that
en_001is "English (World)" locale, which is different fromen_US_POSIX. Not sure how important that is, but something I learned (en_001, en_US_POSIX) - Could FaithLife's
icudtl.datfile simply rely on automatic fallback to the Windowsicudtl.datfile, which presumably does includeen_001.res? - I've tried running Logos with multiple DLL override variations, trying to isolate the problem more, but with no luck. If I force "built-in" DLLs for icu, icuin, and icuuc Logos won't start. Likewise if I block Logos' native icuin52 and icuuc52 DLLs.
I've tried running Logos with multiple DLL override variations, trying to isolate the problem more, but with no luck. If I force "built-in" DLLs for icu, icuin, and icuuc Logos won't start. Likewise if I block Logos' native icuin52 and icuuc52 DLLs.
Not surprising, looks like the app data versions are the same binaries that upstream normally ships, the icu.dll is actually a combination of icuin and icuuc implementations in one dll (then the icuin and icuuc dll that reside next to the icu.dll are just pointers to icu.dll)
It looks like Logos ships its own icudtl.dat file. So I assume that takes precedence over anything we put into C:\windows{system32,syswow64}. But what does that mean in terms of the https://github.com/FaithLife-Community/LogosLinuxInstaller/issues/22 "windows is too old" issue? Do our new ICU files allow that test to pass, but then aren't actually used by Logos? This seems weird to me.
Shot in the dark guess: Logos itself uses appdata, but some of logos' dependencies (perhaps the .net runtime) rely on the system icu.dll . I as was inspecting the calls I noticed getting locale information was rarely used across the logos application - when I suspect it would have been used more widely due to the number of languages shown at the same time. This is backed up by various observations when I mess with the respective dlls different components fail.
Tests Ran
- ran icupkg -l against the icudt.dll in appdata and found it does include the EN_US_POSIX locale
- moved the icudt72l.dat in appdata to icudt72l.dat_ - to figure out if it was used, and it was
- replaced the icudt72l.dat in appdata with one that only has the en_us locale and got some weird crashes
- replaced the icudt72l.dat in appdata with the one we ship into C:\Windows\Globalization\ICU/
- removed the appdata dlls
2024-10-20 22:15:33 INFO: 0110:err:module:import_dll Library icuin52.dll (which is needed by L"C:\\users\\user\\AppData\\Local\\Logos\\System\\Libronix.DigitalLibrary.Resources.Native.dll") not found - tried using logos v37 - same issue
Next steps
Further investigation toward root cause
Next step is probably to dive into the .NET runtime's code - maybe try to do a minimal reproduction with just a .NET application. The real bug might be there, but we may be able to work around it on our side with a custom ICU build.
Collaboration with wine folks?
It should be noted upstream wine may want to ship these dlls in future releases of wine (as windows now natively ships them)
Workaround for Logos (and possibly wine)
@tbleher / @thw26 would it be possible to compile a new .dat using the same build environment as https://github.com/FaithLife-Community/icu/releases/tag/75.1 with the en_US_POSIX locale filtered out?
Filtering can be done using the ICU Data Build Tool. Execution example inspired by docs
# Script assumes an in-source tree build, and runConfigureICU has already been ran. Feel free to adopt steps, meant as a demonstration, not prescriptive.
# Conditional statement to handle either microsoft's fork (icu source lives in a subdir) or upstream unicode (code resides at the root)
if [ -d icu ]; then cd icu; fi
cd icu4c/source/python
cat << EOF > filter.json
{
"localeFilter": {
"filterType": "regex",
"excludelist": [
"^en_US_POSIX$"
]
}
}
EOF
python3 -m icutools.databuilder --filter=filter.json --mode=gnumake --src_dir=../data > ../data/rules.mk
rm filter.json
cd ../data/
make clean all
cp out/icudt72l.dat icudtl.dat
Further investigation toward root cause
Next step is probably to dive into the .NET runtime's code - maybe try to do a minimal reproduction with just a .NET application. The real bug might be there, but we may be able to work around it on our side with a custom ICU build.
My next theory (maybe I'll think of a way to test it): Logos' own System.Globalization.dll is flawed (outdated?), b/c it fails to handle the valid (& properly-defined) en-us-posix culture.
- Logos ships it's own
.NETfiles (@thw26 can confirm, but I think this is partly to make things easier on us, not having to worry about installing.NETseparately into wine?). The wine crash report says that Logos is using.NET 8.0.7. - I assume
System.Globalization.dllis part of.NET, so maybe an update to Logos'.NET(or at least toSystem.Globalization.dll) will enable it to work properly. (8.0.10 binaries, 8.0.10 source code) - This isn't an issue in Windows because presumably the "default" culture is defined as something other than
en-us-posix.
Even if this theory is correct, though, we obviously can't rely on tweaking Logo's installed files. So maybe there's a way instead to set the "default" culture to something other than en-us-posix. Even better if we could figure out what Windows does and match that. I searched the registry for "lang", "en_", "en-", "posix", and I didn't find an explicit entry for "en-us-posix", and I don't think Windows uses env variables at all for language settings, so I don't know where and how this "default" culture gets defined or how to override it.
Your idea of paying closer attention to .NET code (or just the system calls) might help figure out how that culture designation gets fed to Logos, though, which might help us figure out how to override it.
Some info about how locale is determined in .NET:
- dotnet/runtime/98818
- .NET docs re: how culture is determined
- maybe some relevant code from the unicode-org source?
Not sure if this link has any helpful information.
https://serverfault.com/questions/559109/how-to-change-default-region-culture-settings-in-windows-2012
@ctrlaltf24, we are more than happy to ship our own ICU files. That's partially why we have our own repo, which is so that we can test for exactly these kinds of issues and distribute fixes as we need them.
I don't think it is necessarily for our benefit, but Logos does ship its own .NET files likely to have tighter control over the application's environment.
As regards Wine shipping ICU files, it may be the case that they would desire to ship them. More likely, we believe that the Winetricks folks would be the ones to ship these. We reasoned back during our work on supporting Logos 30+ that we should supply a PR to them as soon as we are able to support installing the ICU data files. Of course, if we find we need a custom version of them, that doesn't necessarily benefit us or simplify our application. Our intent was eventually to contribute ICU installation to Winetricks though.
Apologies if I missed something. I have tried to read as much as I can. Please let me know if I can answer anything else.
Good news, got it failing outside of Logos! And using that found the list of erroring locacles and removed them
Next step is probably to dive into the .NET runtime's code - maybe try to do a minimal reproduction with just a .NET application. The real bug might be there, but we may be able to work around it on our side with a custom ICU build.
Ran the following .NET application in the Logo's system dir (to use all of it's dependencies) /home/user/LogosBible10/data/bin/wine-devel_9.19-x86_64.AppImage /home/user/LogosBible10/data/wine64_bottle/drive_c/users/user/AppData/Local/Logos/System/dotnet-poc-en-us-posix-crash.exe
var cultures = System.Globalization.CultureInfo.GetCultures(System.Globalization.CultureTypes.AllCultures);
foreach (var culture in cultures)
{
Console.WriteLine(culture.Name + "\t" + culture.DisplayName);
}
Unhandled exception. System.Globalization.CultureNotFoundException: Culture is not supported. (Parameter 'name')
en-us-posix is an invalid culture identifier.
at System.Globalization.CultureInfo.GetCultureInfo(String name)
at System.Globalization.CultureData.IcuEnumCultures(CultureTypes types)
at Program.<Main>$(String[] args)
0024:err:eventlog:ReportEventW L"Application: dotnet-poc-en-us-posix-crash.exe\n"
0024:err:eventlog:ReportEventW L"CoreCLR Version: 8.0.724.31311\n"
0024:err:eventlog:ReportEventW L".NET Version: 8.0.7\n"
0024:err:eventlog:ReportEventW L"Description: The process was terminated due to an unhandled exception.\n"
0024:err:eventlog:ReportEventW L"Exception Info: System.Globalization.CultureNotFoundException: Culture is not supported. (Parameter 'name')\r\n"
0024:err:eventlog:ReportEventW L"en-us-posix is an invalid culture identifier.\r\n"
0024:err:eventlog:ReportEventW L" at System.Globalization.CultureInfo.GetCultureInfo(String name)\r\n"
0024:err:eventlog:ReportEventW L" at System.Globalization.CultureData.IcuEnumCultures(CultureTypes types)\r\n"
0024:err:eventlog:ReportEventW L" at Program.<Main>$(String[] args)\n"
Now we can reproduce in a lot more isolated environment!
Still no root cause, but was able to isolate and remove all the offending locales! Why these fail, I don't know. But with them removed, the problem goes away! Will work with devs on next steps
{
"localeFilter": {
"filterType": "regex",
"excludelist": [
"^bgc$",
"^bgc_IN$",
"^bho$",
"^bho_IN$",
"^cv$",
"^cv_RU$",
"^en_MV$",
"^en_US_POSIX$",
"^hi_Latn$",
"^hi_Latn_IN$",
"^ibb$",
"^ibb_NG$",
"^iu_CA$",
"^iu_Latn_CA$",
"^jv_Java$",
"^jv_Java_ID$",
"^oc_ES$",
"^pap_029$",
"^quc_GT$",
"^raj$",
"^raj_IN$",
"^tzm_Arab$",
"^tzm_Arab_MA$",
"^tzm_Tfng$",
"^tzm_Tfng_MA$"
]
}
}
Awesome! :detective:
So, to summarize, the working theory is that these locales, which are defined in system-wide ICU files, are not recognized as valid by Logos' included .NET System.Globalization code. Logos requires the system-wide ICU files in order to run, but then it apparently doesn't make use of the actual definitions therein.
So does it instead only accept locales defined in its own ICU files? Or is it more complicated than that?
Awesome! :detective:
So, to summarize, the working theory is that these locales, which are defined in system-wide ICU files, are not recognized as valid by Logos' included .NET
System.Globalizationcode. Logos requires the system-wide ICU files in order to run, but then it apparently doesn't make use of the actual definitions therein.
As far as I understand it yes. For some reason .NET doesn't see these locales as valid
So does it instead only accept locales defined in its own ICU files? Or is it more complicated than that?
More complicated probably, more likely windows doesn't see these locales as valid. The error happens with just the .NET runtime with Logo's shipped dlls. Probably not related to the ICU that it ships
Created a release on my fork with the environment and notes I used to build along with the binaries https://github.com/ctrlaltf24/microsoft-icu/releases/tag/v72.1.0.1-ctrlaltf24.1.
Feel free to test
How to test:
- Download https://github.com/ctrlaltf24/microsoft-icu/releases/download/v72.1.0.1-ctrlaltf24.1/icu-win.tar.gz
- (feel free to inspect the contents)
- Extract the contents of that tarball into the virtual C drive
- Navigate to your Logos install directory on Linux (for example on my system I would do:
cd ~/LogosBible10) - Navigate into the virtual C:\ drive (I would do
cd data/wine64_bottle/drive_c) - Now extract the tarball here (I would do
tar -xvf ~/Downloads/icu-win.tar.gz)
- Navigate to your Logos install directory on Linux (for example on my system I would do:
- Open Logos as you normally do
- Open a bible word study for "worship" and worship! It works!
- Continue using Logos normally, lmk if there are any more crashes related to
System.Globalization.CultureInfo
More complicated probably, more likely windows doesn't see these locales as valid. The error happens with just the .NET runtime with Logo's shipped dlls. Probably not related to the ICU that it ships
I'd be interested in hearing how your test program works if it's run outside of the Logos System directory. Does it have the same problems? Maybe it won't work at all unless .NET is installed system-wide via winetricks? Or, if you can share the exe with me, I'd be willing to experiment a bit with it
So I figured out how to build your simple C# code as a windows app, app.exe. I then used winetricks to install dotnet8 in my Logos wineprefix, then ran app.exe from C:\users\user\Desktop. And, guess what, it has the same error:
$ WINEDLLOVERRIDES="mshtml=d;" WINEPREFIX=$PWD ~/Téléchargements/wine64 ./drive_c/users/nate/Desktop/app.exe
Unhandled exception. System.Globalization.CultureNotFoundException: Culture is n
ot supported. (Parameter 'name')
en-us-posix is an invalid culture identifier.
at System.Globalization.CultureInfo.GetCultureInfo(String name)
at System.Globalization.CultureData.IcuEnumCultures(CultureTypes types)
at Program.<Main>$(String[] args) in /srv/rsync/other/app/Program.cs:line 4
0024:err:eventlog:ReportEventW L"Application: app.exe\n"
0024:err:eventlog:ReportEventW L"CoreCLR Version: 8.0.224.6711\n"
0024:err:eventlog:ReportEventW L".NET Version: 8.0.2\n"
0024:err:eventlog:ReportEventW L"Description: The process was terminated due to an unhandled exception.\n"
0024:err:eventlog:ReportEventW L"Exception Info: System.Globalization.CultureNotFoundException: Culture is not supported. (Parameter 'name')\r\n"
0024:err:eventlog:ReportEventW L"en-us-posix is an invalid culture identifier.\r\n"
0024:err:eventlog:ReportEventW L" at System.Globalization.CultureInfo.GetCultureInfo(String name)\r\n"
0024:err:eventlog:ReportEventW L" at System.Globalization.CultureData.IcuEnumCultures(CultureTypes types)\r\n"
0024:err:eventlog:ReportEventW L" at Program.<Main>$(String[] args) in /srv/rsync/other/app/Program.cs:line 4\n"
So the problem is not just with Logos' .NET 8 runtime, but also with MS' own .NET 8 runtime when run within wine. I know from testing in a VM that Windows 11 doesn't have this issue with Logos or the ICU culture identifiers. But somehow the combination of wine + .NET + ICU files has this problem.
Existing users: you may need to install ICU files again, it'll automatically update
How to apply the update:
TUI
Utilities -> Install ICU files
CLI
Use flag --install-icu
GUI
- App actions -> Install/update ICU files
- Run Action