CRASH running any app from cmd due to timezone delay-load query

[derekbruening]

On my win10 machine I just discovered that running any Dr. Memory build from a cmd shell, instead of a cygwin shell, immediately crashes at PC 0. Things work fine from a cygwin shell.

Investigating:

(1658.1d64): Access violation - code c0000005 (first chance)
First chance exceptions are reported before any exception handling.
This exception may be expected and handled.
eax=00000000 ebx=77cc449c ecx=00000001 edx=01508700 esi=00000000 edi=0000001f
eip=00000000 esp=0133be50 ebp=0133be7c iopl=0         nv up ei pl zr na pe nc
cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00010246
00000000 ??              ???
0:000> kn
 # ChildEBP RetAddr  
WARNING: Frame IP not in any known module. Following frames may be wrong.
00 0133be4c 015f41dc 0x0
01 0133be7c 015f40dd KERNELBASE_14e0000!LoadStringBaseExW+0xac
02 0133c0c0 015f34b3 KERNELBASE_14e0000!ConvertTimeZoneMuiString+0xe9
03 0133c2b0 015f332e KERNELBASE_14e0000!ConvertTimeZoneMuiStrings+0x16c
04 0133c374 77befe94 KERNELBASE_14e0000!GetTimeZoneInformation+0x6e
05 0133c398 77befbc9 ucrtbase!tzset_from_system_nolock+0x5d
06 0133c4c0 77bf0ce2 ucrtbase!tzset_nolock+0x78
07 0133c4f0 77bf0965 ucrtbase!__tzset+0x3a
08 0133c53c 77bf3738 ucrtbase!common_loctotime_t<__int64>+0x8b
09 0133c588 77bf3674 ucrtbase!convert_filetime_to_time_t<__int64>+0x68
0a 0133c5f0 77bf35b3 ucrtbase!common_stat_handle_file_opened<_stat64>+0xa3
0b 0133c650 77c3dda1 ucrtbase!common_stat<_stat64>+0x82
0c 0133c68c 77c46280 ucrtbase!__crt_state_management::wrapped_invoke<int (__cdecl*)(wchar_t const *,_stat64 *),wchar_t const *,_stat64 *,int>+0x24
+** ERROR: Symbol file could not be found.  Defaulted to export symbols for dbghelp.dll - 
0d 0133c698 53e83fc3 ucrtbase!_o__wstat64+0x10
0e 0133c720 53e850cd dbghelp!RangeMapWrite+0xa0ba3
0f 0133c77c 53e02fe0 dbghelp!RangeMapWrite+0xa1cad
10 0133c7b0 53e02e9c dbghelp!RangeMapWrite+0x1fbc0
11 0133c7d0 53dae54e dbghelp!RangeMapWrite+0x1fa7c
12 0133c808 53dae697 dbghelp+0x2e54e
13 0133c830 53db194f dbghelp+0x2e697
14 0133d4f4 53db258a dbghelp!RemoveInvalidModuleList+0x25cf
15 0133d968 53dd39a7 dbghelp!RemoveInvalidModuleList+0x320a
16 0133db98 53dd25f8 dbghelp!ImagehlpApiVersionEx+0x1aa7
17 0133e048 53dd2b2e dbghelp!ImagehlpApiVersionEx+0x6f8
18 0133e4bc 53dcc150 dbghelp!ImagehlpApiVersionEx+0xc2e
19 0133e50c 73aeb35a dbghelp!SymLoadModuleExW+0x30
1a 0133e788 73aeb52a drmemorylib!load_module+0x1ca [d:\drmemory_package\dynamorio\ext\drsyms\drsyms_windows.c @ 369] 
1b 0133e7a0 73aea9bd drmemorylib!lookup_or_load+0x7a [d:\drmemory_package\dynamorio\ext\drsyms\drsyms_windows.c @ 432] 
1c 0133e7b8 739c1607 drmemorylib!drsym_get_module_debug_kind+0x4d [d:\drmemory_package\dynamorio\ext\drsyms\drsyms_windows.c @ 1718] 
1d 0133e9c4 73813461 drmemorylib!report_init+0x547 [d:\drmemory_package\drmemory\report.c @ 1546] 
1e 0133ed90 53674544 drmemorylib!dr_init+0x2791 [d:\drmemory_package\drmemory\drmemory.c @ 1967] 
1f 0133edb4 534bcd37 dynamorio!instrument_init+0x184 [d:\drmemory_package\dynamorio\core\lib\instrument.c @ 735] 
20 0133f670 53716fd1 dynamorio!dynamorio_app_init+0x5c7 [d:\drmemory_package\dynamorio\core\dynamo.c @ 677] 
21 0133f6d8 53717938 dynamorio!auto_setup+0x21 [d:\drmemory_package\dynamorio\core\arch\x86_code.c @ 165] 
22 00000000 00000000 dynamorio!dynamo_auto_start+0x8 [D:\derek\drmemory\build_package\build_drmemory-debug-32\dynamorio\core\CMakeFiles\dynamorio.dir\arch\x86\x86.asm.obj.s @ 1572]

015f41d6 ff1500e26a01    call    dword ptr [KERNELBASE_14e0000!_imp__BasepNotifyLoadStringResource (016ae200)]

0:000> U 01602927 
KERNELBASE_14e0000!_imp_load__BasepNotifyLoadStringResource:
01602927 b800e26a01      mov     eax,offset KERNELBASE_14e0000!_imp__BasepNotifyLoadStringResource (016ae200)
0160292c e9dbffffff      jmp     KERNELBASE_14e0000!_tailMerge_ext_ms_win_kernel32_registry_l1_1_0_dll (0160290c)

0:000> U 0160290c
KERNELBASE_14e0000!_tailMerge_ext_ms_win_kernel32_registry_l1_1_0_dll:
0160290c 51              push    ecx
0160290d 52              push    edx
0160290e 50              push    eax
0160290f 68d82b6901      push    offset KERNELBASE_14e0000!_DELAY_IMPORT_DESCRIPTOR_ext_ms_win_kernel32_registry_l1_1_0_dll (01692bd8)
01602914 e8578dffff      call    KERNELBASE_14e0000!__delayLoadHelper2 (015fb670)

0:000> Uf 015fb670
KERNELBASE_14e0000!__delayLoadHelper2:
015fb670 8bff            mov     edi,edi
015fb672 55              push    ebp
015fb673 8bec            mov     ebp,esp
015fb675 ff35640f5701    push    dword ptr [KERNELBASE_14e0000!__DefaultResolveDelayLoadedAPIFlags (01570f64)]
015fb67b ff750c          push    dword ptr [ebp+0Ch]
015fb67e 68c0a16701      push    offset KERNELBASE_14e0000!DelayLoadFailureHook (0167a1c0)
015fb683 ff35640f5701    push    dword ptr [KERNELBASE_14e0000!__DefaultResolveDelayLoadedAPIFlags (01570f64)]
015fb689 ff7508          push    dword ptr [ebp+8]
015fb68c 6800004e01      push    offset KERNELBASE_14e0000!g_SbModuleTable_GetFileVersion <PERF> (KERNELBASE_14e0000+0x0) (014e0000)
015fb691 ff15dc866a01    call    dword ptr [KERNELBASE_14e0000!_imp__LdrResolveDelayLoadedAPI (016a86dc)]
015fb697 5d              pop     ebp
015fb698 c20800          ret     8
0:000> dd 016a86dc
016a86dc  77d2b000 77d9ce30 77d9cd90 77d5bab0

0:000> U 77d2b000 
ntdll!LdrResolveDelayLoadedAPI:
77d2b000 8bff            mov     edi,edi

Our private loader does not support delay-load (DRi#233). Same dbghelp.dll.

Stepping through that call from report_init from cygwin shell, it does reach here:

06 0133c4c0 77bf0ce2 ucrtbase!tzset_nolock+0x78

But it never calls into kernelbase_xxxx, b/c "ucrtbase!common_getenv_s" of "TZ" finds a value ("America/New_York").

It then skips the call to the system routine and instead calls environment:

77befbbf 803f00          cmp     byte ptr [edi],0
77befbc2 751d            jne     ucrtbase!tzset_nolock+0x90 (77befbe1)
77befbc4 e86e020000      call    ucrtbase!tzset_from_system_nolock (77befe37)

0:000> da edi
004fb718  "America/New_York"

77befbe1 8bcf            mov     ecx,edi
77befbe3 e8325e0a00      call    ucrtbase!tzset_from_environment_nolock (77c95a1a)

Indeed, %TZ% is unset in cmd, and setting it to "America/New_York" makes DrM work!

To solve: don't really want to try to set TZ as a workaround at init time: afraid some later drsyms load will still hit this path, and afraid of setting it for the app and messing it up.

Could try to implement LdrResolveDelayLoadedAPI (and how does it relate to the delay-load import feature https://github.com/DynamoRIO/dynamorio/issues/233?)

Or, we could implement redirect_GetTimeZoneInformation, which is already on the list of redirections we want: https://github.com/DynamoRIO/dynamorio/issues/1063.

[kilitary]

but, On my machine i run dr alban and no blue screens and other pro b l em s