Add support for native Arm64 arhitecture

[cristianadam]

Tested on Windows 11 with "premake5 vs2022".

The obligatory screenshot is below:

[chrisant996]

Last I checked, there is no IAT hooking library yet for ARM. That is a crucial dependency.

I do not have the bandwidth available to be able to tackle creating an IAT hooking library for ARM, and that is much better than just Clink.

[chrisant996]

This looks like a PR, but I don't see where it tackles the problem about ARM hooking.

Can you clarify?

[cristianadam]

This looks like a PR, but I don't see where it tackles the problem about ARM hooking.

Can you clarify?

I just compiled the project, and it worked out with the changes that I've uploaded. From what I've seen detours supports arm64.

I had a look with process explorer, the right dll is loaded. I don't know why it works 😅

[chrisant996]

Detours is only there as a hack for one user for whom it fooled their antivirus suite.

Detours does not achieve the proper hooking for Clink to work properly. It can sort of work, but if anything else hooks one of the APIs then all bets are off.

I plan to remove Detours.

Try it without Detours.

[cristianadam]

I've tried removing Detours completely, from build system and getting the code to compile, but in the end clink was no longer properly hooked and didn't work as expected.

Then I just did a minimal change that only removed the detours hooking:

diff --git a/clink/app/src/host/host_cmd.cpp b/clink/app/src/host/host_cmd.cpp
index ce3f66ea..aefbb6a8 100644
--- a/clink/app/src/host/host_cmd.cpp
+++ b/clink/app/src/host/host_cmd.cpp
@@ -69,7 +69,7 @@ static setting_str g_admin_title_prefix(
 //------------------------------------------------------------------------------
 static hook_type get_hook_type()
 {
-    static hook_type s_hook_type = app_context::get()->is_detours() ? detour : iat;
+    static hook_type s_hook_type = /*app_context::get()->is_detours() ? detour : */iat;
     return s_hook_type;
 }
 static const char* get_kernel_module()
diff --git a/clink/app/src/utils/hook_setter.cpp b/clink/app/src/utils/hook_setter.cpp
index 94e74f50..ba5eb0cd 100644
--- a/clink/app/src/utils/hook_setter.cpp
+++ b/clink/app/src/utils/hook_setter.cpp
@@ -225,8 +225,8 @@ bool hook_setter::attach_internal(hook_type type, const char* module, const char
 
     if (type == iat)
         return attach_iat(module, name, hook, original);
-    else if (type == detour)
-        return attach_detour(module, name, hook, original);
+//    else if (type == detour)
+//        return attach_detour(module, name, hook, original);
     else
         return false;
 }
@@ -244,8 +244,8 @@ bool hook_setter::detach_internal(hook_type type, const char* module, const char
 
     if (type == iat)
         return detach_iat(module, name, original, hook);
-    else if (type == detour)
-        return detach_detour(original, hook);
+//    else if (type == detour)
+//        return detach_detour(original, hook);
     else
         return false;
 }
@@ -253,6 +253,7 @@ bool hook_setter::detach_internal(hook_type type, const char* module, const char
 //------------------------------------------------------------------------------
 bool hook_setter::attach_detour(const char* module, const char* name, hookptr_t hook, hookptrptr_t original)
 {
+#if 0
     LOG("Attempting to detour %s in %s with %p.", name, module, hook);
     HMODULE hModule = GetModuleHandleA(module);
     if (!hModule)
@@ -298,7 +299,7 @@ bool hook_setter::attach_detour(const char* module, const char* name, hookptr_t
     // Return the trampoline in original.
     if (original)
         *original = hookptr_t(trampoline);
-
+#endif
     return true;
 }
 
diff --git a/clink/app/src/utils/hook_setter.h b/clink/app/src/utils/hook_setter.h
index e61ef887..631fa09d 100644
--- a/clink/app/src/utils/hook_setter.h
+++ b/clink/app/src/utils/hook_setter.h
@@ -4,7 +4,7 @@
 #pragma once
 
 //------------------------------------------------------------------------------
-enum hook_type { iat, detour };
+enum hook_type { iat/*, detour*/ };
 typedef void (__stdcall* hookptr_t)();
 typedef hookptr_t* hookptrptr_t;
 bool find_iat(void* base, const char* dll, const char* func_name, bool find_by_name, hookptrptr_t* import_out, hookptr_t* original_out);

And clink arm64 continued to work, here is the log:

2db4 start_logger              160 ---- 2022/10/17 10:06:33.975 -------------------------------------------------
2db4 start_logger              166 Host process is 'cmd.exe' (pid 11700)
2db4 start_logger              170 DLL path is 'C:\Projects\clink\repo\.build\vs2022\bin\final'
2db4 start_logger              185 Windows version 10.0.22621 (arm64)
2db4 start_logger              190 Clink version 1.3.47.4d4df6 (arm64)
2db4 hook_setter::hook_setter  159 >>> Started hook transaction.
2db4 hook_setter::attach_iat   317 Attempting to hook SetEnvironmentVariableW in IAT for module 00007FF7CCAC0000.
2db4 pe_info::iterate_imports  159 Found import in 'api-ms-win-core-processenvironment-l1-1-0.dll'
2db4 find_iat                  124 Found import at 00007FF7CCB253F0 (value is 00007FFD60116FF0).
2db4 hook_setter::attach_iat   317 Attempting to hook WriteConsoleW in IAT for module 00007FF7CCAC0000.
2db4 pe_info::iterate_imports  159 Found import in 'api-ms-win-core-console-l1-1-0.dll'
2db4 find_iat                  124 Found import at 00007FF7CCB25038 (value is 00007FFD600A2000).
2db4 hook_setter::attach_iat   317 Attempting to hook GetEnvironmentVariableW in IAT for module 00007FF7CCAC0000.
2db4 pe_info::iterate_imports  159 Found import in 'api-ms-win-core-processenvironment-l1-1-0.dll'
2db4 find_iat                  124 Found import at 00007FF7CCB253C8 (value is 00007FFD600C5080).
2db4 hook_setter::commit       207 <<< Hook transaction committed.
2db4 hook_setter::hook_setter  159 >>> Started hook transaction.
2db4 hook_setter::detach_iat   360 Attempting to unhook 00007FFCBD216CB8 from GetEnvironmentVariableW in IAT for module 00007FF7CCAC0000.
2db4 pe_info::iterate_imports  159 Found import in 'api-ms-win-core-processenvironment-l1-1-0.dll'
2db4 find_iat                  124 Found import at 00007FF7CCB253C8 (value is 00007FFCBD216CB8).
2db4 hook_setter::commit       207 <<< Hook transaction committed.
2db4 hook_setter::hook_setter  159 >>> Started hook transaction.
2db4 hook_setter::attach_iat   317 Attempting to hook ReadConsoleW in IAT for module 00007FF7CCAC0000.
2db4 pe_info::iterate_imports  159 Found import in 'api-ms-win-core-console-l1-1-0.dll'
2db4 find_iat                  124 Found import at 00007FF7CCB25020 (value is 00007FFD601B6D20).
2db4 hook_setter::commit       207 <<< Hook transaction committed.
2db4 hook_setter::hook_setter  159 >>> Started hook transaction.
2db4 hook_setter::attach_iat   317 Attempting to hook SetConsoleTitleW in IAT for module 00007FF7CCAC0000.
2db4 pe_info::iterate_imports  159 Found import in 'api-ms-win-core-console-l2-2-0.dll'
2db4 find_iat                  124 Found import at 00007FF7CCB25090 (value is 00007FFD601B4850).
2db4 hook_setter::commit       207 <<< Hook transaction committed.
2db4 win_screen_buffer::begin  228 Using native terminal support; found 'Windows build >= 15063, console V2'.
2db4 host_lua::load_scripts    158 Loaded 1 Lua scripts in 0 ms
2db4 =?                          0 Clink updater: autoupdate is disabled for local build directories.
2db4 reset_handle              144 resetting mismatched stdout handle
2db4 reset_handle              144 resetting mismatched stderr handle
2db4 extract_ctag              926 read 0 bytes
2db4 extract_ctag              926 read 0 bytes
2db4 history_db::initialise   1264 master bank ctag: |CTAG_1665993994_38167625_11700_0
2db4 history_db::compact      1526 History:  0 active, 0 deleted
2db4 reset_handle              144 resetting mismatched stdout handle
2db4 reset_handle              144 resetting mismatched stderr handle
2db4 history_db::compact      1526 History:  0 active, 0 deleted
2db4 reset_handle              144 resetting mismatched stdout handle
2db4 reset_handle              144 resetting mismatched stderr handle
2db4 history_db::compact      1526 History:  0 active, 0 deleted

[chrisant996]

Ok, I'll take a closer look this week. I don't have an ARM device, which is another complicating factor.

[cristianadam]

I have a Samsung Galaxy Book 5G, which can be obtained on Ebay.com for as low as 240-310$. I hacked mine to have a better display, since Samsung produced them really cheap.

I think a Raspberry PI 4 with 8GB could also be used with Windows 11 arm64, but I don't think the user experience is as good as an officially supported device.

[chrisant996]

Oh, of course -- this never clicked with me until now:

1. Originally Martin had used "jmp" hooking (the hooking approach used by Detours).
2. Over time Martin converted to mostly use "IAT" hooking, but ReadConsoleW was still using jmp hooking.
3. The usage of jmp hooking was why Clink couldn't work on ARM.
4. Eventually I experimented briefly with Detours in Clink; that led to problems, so:
5. In commit ba7111887f26b281a156ecc102ec2065fb33ebed I disabled Detours in Clink, and also finally converted ReadConsoleW to use IAT hooking instead of jmp hooking.

IAT hooking just replaces addresses and doesn't require parsing or constructing any assembly language code on the fly, which means the IAT hooking approach is already natively portable to ARM.

So, it should indeed be possible to support ARM in Clink.

The .zip package and .exe installer are a little more complicated to deal with, though.

I'll see what I can do.

[cristianadam]

Regarding the size of the Arm64 binaries.

Unpacked size:

• x86 and x64 - 4.59 MiB in size
• x86, x64 and arm64 - 6.38 MiB in size

That's a 38.99% size increase.

For zip files (LZMA compression with Ultra settings in 7Zip)

• x86 and x64 - 1.55 MiB
• x86, x64 and arm64 - 2.12 MiB

That is also a 38.70% size increase.

But given the fact that today the x86 architecture is becoming less and less used, it looks like the package already has an architecture the users don't use.

Packaging x64 and arm64 would be more future proof.

Sysinternals tools like DebugView or Process Explorer package all three architectures in one zip.

[chrisant996]

Yes. I just hate dealing with people angrily complaining that 1 MB of space of storage is being "wasted". I will likely just bundle in ARM, but I need to finish due diligence evaluation first.

Re: x86 -- 64 bit OS can and does run 32 bit programs, and any cmd spawned from a 32 bit program will use x86 even on a an x64 OS. Even I still use a few 32 bit programs that can launch cmd shells. But ARM can never be used on x86 or x64 machines, so it's not quite the same comparison.

But also, I currently cannot officially support ARM64 when I have no ARM devices and cannot test (or fix) ARM at all. At this time I'm not willing to spend my own money on an ARM device for this.

[MagicAndre1981]

But also, I currently cannot officially support ARM64 when I have no ARM devices and cannot test (or fix) ARM at all. At this time I'm not willing to spend my own money on an ARM device for this.

if you have a Raspberry Pi4 or old Lumia 950XL phone, you can install Windows 11 arm64 version and test ARM64 software, this is what I do for ARM64 testings

[chrisant996]

I don't have those. Also, I had to revert part of the ARM64 changes because it accidentally made it impossible to produce new Clink releases due to how the Lua language handles precompiled scripts.