python
Inconsistent results from platform.machine() on Windows ARM64
Bug report
Running x86 and AMD64 builds of Python 3.11.0 (official binary releases) in emulation on Windows 11 on an ARM64 machine gives me
Python 3.11.0 (main, Oct 24 2022, 18:26:48) [MSC v.1933 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> platform.machine()
'AMD64'
Python 3.11.0 (main, Oct 24 2022, 18:13:38) [MSC v.1933 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> platform.machine()
'ARM64'
This seem inconsistent. I would expect both to return 'ARM64', the actual underlying processor architecture, in the same way as running an x86 build on AMD64 returns 'AMD64'. But even if they were reporting the emulated architecture, it should be 'AMD64' and 'x86', not 'AMD64' and 'ARM64' (or possibly 'AMD64' and 'AMD64', if x86 binaries are also handled by the AMD64 emulator, I have no idea how that works).
This makes it hard to distinguish between an AMD64 build running on ARM64 and an AMD64 build running on AMD64. In fact, the only platform function that gives any indication at all is processor(), and parsing the desired information out of the free-form text returned by that seems fiddly. ('ARMv8 (64-bit) Family 8 Model 0 Revision 0, ' and 'Intel64 Family 6 Model 44 Stepping 2, GenuineIntel' here.)
Relatedly, how do I distinguish between an ARM64 build running on ARM64 and an AMD64 build running on ARM64? The inconsistency above makes them differ in platform.machine(), but what if that were fixed? This seems like a job for platform.architecture(), which is explicitly meant to give information about the running binary, but that just returns ('64bit', 'WindowsPE') in both cases. platform.python_compiler() happens to have that information ('MSC v.1933 64 bit (ARM64)' vs. 'MSC v.1933 64 bit (AMD64)'), but again that is fiddly free-form text and may not work for other compilers than MSC.
Your environment
- CPython versions tested on: 3.11.0
- Operating system and architecture: Windows 11 10.0.22000 ARM64, Windows 10 10.0.19043 AMD64
Out of curiosity, can you install the x86 and AMD64 versions of 3.12.0a1 and check platform.machine() in each? The new implementation in 3.12 prefers to get the CPU "Architecture" from WMI "Win32_Processor" instead of from the system environment variables "PROCESSOR_ARCHITEW6432" and "PROCESSOR_ARCHITECTURE".
Success! Thanks for the tip. In 3.12.0a1, all binaries return platform.machine() == 'ARM64' when run on ARM64 and platform.machine() == 'AMD64' when run on AMD64, which is what I would expect.
I notice however that, in a similar issue on macOS, #96993, @rossng seems to have a different expectation.
The problem remains that ARM64-on-ARM64 and AMD64-on-ARM64 are only distinguishable by platform.python_compiler(). Should I open a separate issue for that?
Detailed output
AMD64-on-ARM64
Python 3.12.0a1 (main, Oct 25 2022, 00:24:35) [MSC v.1933 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> for n in dir(platform):
... try:
... print("{:22s} {:s}".format(n, repr(getattr(platform, n)())))
... except:
... pass
...
_Processor <platform._Processor object at 0x0000021960314F50>
_get_machine_win32 'ARM64'
_mac_ver_xml None
_node 'M1Win11'
_sys_version ('CPython', '3.12.0a1', '', '', 'main', 'Oct 25 2022 00:24:35', 'MSC v.1933 64 bit (AMD64)')
_syscmd_ver ('Microsoft', 'Windows', '10.0.22000')
architecture ('64bit', 'WindowsPE')
java_ver ('', '', ('', '', ''), ('', '', ''))
libc_ver ('', '')
mac_ver ('', ('', '', ''), '')
machine 'ARM64'
node 'M1Win11'
platform 'Windows-11-10.0.22000-SP0'
processor 'ARMv8 (64-bit) Family 8 Model 0 Revision 0, '
python_branch ''
python_build ('main', 'Oct 25 2022 00:24:35')
python_compiler 'MSC v.1933 64 bit (AMD64)'
python_implementation 'CPython'
python_revision ''
python_version '3.12.0a1'
python_version_tuple ('3', '12', '0a1')
release '11'
system 'Windows'
uname uname_result(system='Windows', node='M1Win11', release='11', version='10.0.22000', machine='ARM64')
version '10.0.22000'
win32_edition 'Professional'
win32_is_iot False
win32_ver ('11', '10.0.22000', 'SP0', 'Multiprocessor Free')
>>> ^Z
x86-on-ARM64
Python 3.12.0a1 (main, Oct 25 2022, 00:13:39) [MSC v.1933 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> for n in dir(platform):
... try:
... print("{:22s} {:s}".format(n, repr(getattr(platform, n)())))
... except:
... pass
...
_Processor <platform._Processor object at 0x08BD6C78>
_get_machine_win32 'ARM64'
_mac_ver_xml None
_node 'M1Win11'
_sys_version ('CPython', '3.12.0a1', '', '', 'main', 'Oct 25 2022 00:13:39', 'MSC v.1933 32 bit (Intel)')
_syscmd_ver ('Microsoft', 'Windows', '10.0.22000')
architecture ('32bit', 'WindowsPE')
java_ver ('', '', ('', '', ''), ('', '', ''))
libc_ver ('', '')
mac_ver ('', ('', '', ''), '')
machine 'ARM64'
node 'M1Win11'
platform 'Windows-11-10.0.22000-SP0'
processor 'ARMv8 (64-bit) Family 8 Model 0 Revision 0, '
python_branch ''
python_build ('main', 'Oct 25 2022 00:13:39')
python_compiler 'MSC v.1933 32 bit (Intel)'
python_implementation 'CPython'
python_revision ''
python_version '3.12.0a1'
python_version_tuple ('3', '12', '0a1')
release '11'
system 'Windows'
uname uname_result(system='Windows', node='M1Win11', release='11', version='10.0.22000', machine='ARM64')
version '10.0.22000'
win32_edition 'Enterprise'
win32_is_iot False
win32_ver ('11', '10.0.22000', 'SP0', 'Multiprocessor Free')
>>> ^Z
ARM64-on-ARM64
Python 3.12.0a1 (main, Oct 25 2022, 00:13:29) [MSC v.1933 64 bit (ARM64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> for n in dir(platform):
... try:
... print("{:22s} {:s}".format(n, repr(getattr(platform, n)())))
... except:
... pass
...
_Processor <platform._Processor object at 0x000001AAC8D78FB0>
_get_machine_win32 'ARM64'
_mac_ver_xml None
_node 'M1Win11'
_sys_version ('CPython', '3.12.0a1', '', '', 'main', 'Oct 25 2022 00:13:29', 'MSC v.1933 64 bit (ARM64)')
_syscmd_ver ('Microsoft', 'Windows', '10.0.22000')
architecture ('64bit', 'WindowsPE')
java_ver ('', '', ('', '', ''), ('', '', ''))
libc_ver ('', '')
mac_ver ('', ('', '', ''), '')
machine 'ARM64'
node 'M1Win11'
platform 'Windows-11-10.0.22000-SP0'
processor 'ARMv8 (64-bit) Family 8 Model 0 Revision 0, '
python_branch ''
python_build ('main', 'Oct 25 2022 00:13:29')
python_compiler 'MSC v.1933 64 bit (ARM64)'
python_implementation 'CPython'
python_revision ''
python_version '3.12.0a1'
python_version_tuple ('3', '12', '0a1')
release '11'
system 'Windows'
uname uname_result(system='Windows', node='M1Win11', release='11', version='10.0.22000', machine='ARM64')
version '10.0.22000'
win32_edition 'Professional'
win32_is_iot False
win32_ver ('11', '10.0.22000', 'SP0', 'Multiprocessor Free')
>>> ^Z
AMD64-on-AMD64
Python 3.12.0a1 (main, Oct 25 2022, 00:24:35) [MSC v.1933 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> for n in dir(platform):
... try:
... print("{:22s} {:s}".format(n, repr(getattr(platform, n)())))
... except:
... pass
...
_Processor <platform._Processor object at 0x000002AF1B5FA630>
_get_machine_win32 'AMD64'
_mac_ver_xml None
_node 'DESKTOP-ILRVE2N'
_sys_version ('CPython', '3.12.0a1', '', '', 'main', 'Oct 25 2022 00:24:35', 'MSC v.1933 64 bit (AMD64)')
_syscmd_ver ('Microsoft', 'Windows', '10.0.19043')
architecture ('64bit', 'WindowsPE')
java_ver ('', '', ('', '', ''), ('', '', ''))
libc_ver ('', '')
mac_ver ('', ('', '', ''), '')
machine 'AMD64'
node 'DESKTOP-ILRVE2N'
platform 'Windows-10-10.0.19043-SP0'
processor 'Intel64 Family 6 Model 44 Stepping 2, GenuineIntel'
python_branch ''
python_build ('main', 'Oct 25 2022 00:24:35')
python_compiler 'MSC v.1933 64 bit (AMD64)'
python_implementation 'CPython'
python_revision ''
python_version '3.12.0a1'
python_version_tuple ('3', '12', '0a1')
release '10'
system 'Windows'
uname uname_result(system='Windows', node='DESKTOP-ILRVE2N', release='10', version='10.0.19043', machine='AMD64')
version '10.0.19043'
win32_edition 'Professional'
win32_is_iot False
win32_ver ('10', '10.0.19043', 'SP0', 'Multiprocessor Free')
>>> ^Z
x86-on-AMD64
Python 3.12.0a1 (main, Oct 25 2022, 00:13:39) [MSC v.1933 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import platform
>>> for n in dir(platform):
... try:
... print("{:22s} {:s}".format(n, repr(getattr(platform, n)())))
... except:
... pass
...
_Processor <platform._Processor object at 0x0163CE70>
_get_machine_win32 'AMD64'
_mac_ver_xml None
_node 'DESKTOP-ILRVE2N'
_sys_version ('CPython', '3.12.0a1', '', '', 'main', 'Oct 25 2022 00:13:39', 'MSC v.1933 32 bit (Intel)')
_syscmd_ver ('Microsoft', 'Windows', '10.0.19043')
architecture ('32bit', 'WindowsPE')
java_ver ('', '', ('', '', ''), ('', '', ''))
libc_ver ('', '')
mac_ver ('', ('', '', ''), '')
machine 'AMD64'
node 'DESKTOP-ILRVE2N'
platform 'Windows-10-10.0.19043-SP0'
processor 'Intel64 Family 6 Model 44 Stepping 2, GenuineIntel'
python_branch ''
python_build ('main', 'Oct 25 2022 00:13:39')
python_compiler 'MSC v.1933 32 bit (Intel)'
python_implementation 'CPython'
python_revision ''
python_version '3.12.0a1'
python_version_tuple ('3', '12', '0a1')
release '10'
system 'Windows'
uname uname_result(system='Windows', node='DESKTOP-ILRVE2N', release='10', version='10.0.19043', machine='AMD64')
version '10.0.19043'
win32_edition 'Enterprise'
win32_is_iot False
win32_ver ('10', '10.0.19043', 'SP0', 'Multiprocessor Free')
>>> ^Z
The problem remains that ARM64-on-ARM64 and AMD64-on-ARM64
Have you checked os.environ['PROCESSOR_ARCHITECTURE']? That should be the architecture of the current process.
Since you don't control the environment that was used to create the current process, it would be more reliable to call WinAPI GetSystemInfo() via PyWin32 or ctypes. This should return wProcessorArchitecture as PROCESSOR_ARCHITECTURE_INTEL (0) for x86, PROCESSOR_ARCHITECTURE_AMD64 (9) for AMD64 (x64), and PROCESSOR_ARCHITECTURE_ARM64 (12) for ARM64.
Confirmed, these methods both work on all combinations, on both 3.11.0 and 3.12.0a1. Thanks! That will do as a workaround, although of course it would be nicer to have it abstracted in the platform module, ideally in a way that works identically on other OSes (although at the moment Windows is all I care about).
(It seems weird that two processes started from the same shell would see different environments, but then I’m used to weirdness from Microsoft.)
Detailed output
Script
import os
print('PROCESSOR_ARCHITECTURE:', os.environ['PROCESSOR_ARCHITECTURE'])
import ctypes
import ctypes.wintypes
class SYSTEM_INFO(ctypes.Structure):
_fields_ = [('wProcessorArchitecture', ctypes.wintypes.WORD), ('dwPageSize', ctypes.wintypes.DWORD), ('lpMinimumApplicationAddress', ctypes.wintypes.LPVOID), ('lpMaximumApplicationAddress', ctypes.wintypes.LPVOID), ('dwActiveProcessorMask', ctypes.c_void_p), ('dwNumberOfProcessors', ctypes.wintypes.DWORD), ('dwProcessorType', ctypes.wintypes.DWORD), ('dwAllocationGranularity', ctypes.wintypes.DWORD), ('wProcessorLevel', ctypes.wintypes.WORD), ('wProcessorRevision', ctypes.wintypes.WORD)]
info = SYSTEM_INFO()
ctypes.windll.kernel32.GetSystemInfo(ctypes.byref(info))
print('GetSystemInfo: ', {f[0]: getattr(info, f[0]) for f in SYSTEM_INFO._fields_})
ctypes.windll.kernel32.GetNativeSystemInfo(ctypes.byref(info))
print('GetNativeSystemInfo:', {f[0]: getattr(info, f[0]) for f in SYSTEM_INFO._fields_})
AMD64-on-ARM64
PROCESSOR_ARCHITECTURE: AMD64
GetSystemInfo: {'wProcessorArchitecture': 9, 'dwPageSize': 4096, 'lpMinimumApplicationAddress': 65536, 'lpMaximumApplicationAddress': 140737488289791, 'dwActiveProcessorMask': 15, 'dwNumberOfProcessors': 4, 'dwProcessorType': 8664, 'dwAllocationGranularity': 65536, 'wProcessorLevel': 15, 'wProcessorRevision': 1034}
GetNativeSystemInfo: {'wProcessorArchitecture': 9, 'dwPageSize': 4096, 'lpMinimumApplicationAddress': 65536, 'lpMaximumApplicationAddress': 140737488289791, 'dwActiveProcessorMask': 15, 'dwNumberOfProcessors': 4, 'dwProcessorType': 8664, 'dwAllocationGranularity': 65536, 'wProcessorLevel': 15, 'wProcessorRevision': 1034}
x86-on-ARM64
PROCESSOR_ARCHITECTURE: x86
GetSystemInfo: {'wProcessorArchitecture': 0, 'dwPageSize': 4096, 'lpMinimumApplicationAddress': 65536, 'lpMaximumApplicationAddress': 2147418111, 'dwActiveProcessorMask': 15, 'dwNumberOfProcessors': 4, 'dwProcessorType': 586, 'dwAllocationGranularity': 65536, 'wProcessorLevel': 15, 'wProcessorRevision': 1034}
GetNativeSystemInfo: {'wProcessorArchitecture': 9, 'dwPageSize': 4096, 'lpMinimumApplicationAddress': 65536, 'lpMaximumApplicationAddress': 2147418111, 'dwActiveProcessorMask': 15, 'dwNumberOfProcessors': 4, 'dwProcessorType': 586, 'dwAllocationGranularity': 65536, 'wProcessorLevel': 15, 'wProcessorRevision': 1034}
ARM64-on-ARM64
PROCESSOR_ARCHITECTURE: ARM64
GetSystemInfo: {'wProcessorArchitecture': 12, 'dwPageSize': 4096, 'lpMinimumApplicationAddress': 65536, 'lpMaximumApplicationAddress': 140737488289791, 'dwActiveProcessorMask': 15, 'dwNumberOfProcessors': 4, 'dwProcessorType': 0, 'dwAllocationGranularity': 65536, 'wProcessorLevel': 0, 'wProcessorRevision': 0}
GetNativeSystemInfo: {'wProcessorArchitecture': 12, 'dwPageSize': 4096, 'lpMinimumApplicationAddress': 65536, 'lpMaximumApplicationAddress': 140737488289791, 'dwActiveProcessorMask': 15, 'dwNumberOfProcessors': 4, 'dwProcessorType': 0, 'dwAllocationGranularity': 65536, 'wProcessorLevel': 0, 'wProcessorRevision': 0}
AMD64-on-AMD64
PROCESSOR_ARCHITECTURE: AMD64
GetSystemInfo: {'wProcessorArchitecture': 9, 'dwPageSize': 4096, 'lpMinimumApplicationAddress': 65536, 'lpMaximumApplicationAddress': 140737488289791, 'dwActiveProcessorMask': 3, 'dwNumberOfProcessors': 2, 'dwProcessorType': 8664, 'dwAllocationGranularity': 65536, 'wProcessorLevel': 6, 'wProcessorRevision': 11266}
GetNativeSystemInfo: {'wProcessorArchitecture': 9, 'dwPageSize': 4096, 'lpMinimumApplicationAddress': 65536, 'lpMaximumApplicationAddress': 140737488289791, 'dwActiveProcessorMask': 3, 'dwNumberOfProcessors': 2, 'dwProcessorType': 8664, 'dwAllocationGranularity': 65536, 'wProcessorLevel': 6, 'wProcessorRevision': 11266}
x86-on-AMD64
PROCESSOR_ARCHITECTURE: x86
GetSystemInfo: {'wProcessorArchitecture': 0, 'dwPageSize': 4096, 'lpMinimumApplicationAddress': 65536, 'lpMaximumApplicationAddress': 2147418111, 'dwActiveProcessorMask': 3, 'dwNumberOfProcessors': 2, 'dwProcessorType': 586, 'dwAllocationGranularity': 65536, 'wProcessorLevel': 6, 'wProcessorRevision': 11266}
GetNativeSystemInfo: {'wProcessorArchitecture': 9, 'dwPageSize': 4096, 'lpMinimumApplicationAddress': 65536, 'lpMaximumApplicationAddress': 4294901759, 'dwActiveProcessorMask': 3, 'dwNumberOfProcessors': 2, 'dwProcessorType': 8664, 'dwAllocationGranularity': 65536, 'wProcessorLevel': 6, 'wProcessorRevision': 11266}
I just checked the source for QEMU user space emulation, which is used on Linux and BSD to emulate another CPU architecture in user space, relying on native system calls. When translating the uname() system call for an emulated process, it replaces the native machine with the emulated machine. Thus it seems that platform.machine() for an x64 Python that's running under QEMU emulation on ARM64 would report "x86_64" instead of "aarch64" (ARM64). In contrast, on Windows we return the native OS architecture. Should this be made consistent across platforms? If so, which choice is correct? If the choice to return the emulated architecture is correct, we could switch to calling GetNativeSystemInfo() on Windows, which will return the emulated architecture, instead of using a WMI query that returns the true machine architecture.
Off-topic comments on using ctypes:
- I would include the
wReservedfield inSYSTEM_INFOinstead of implicitly relying on alignment padding to set the offset ofdwPageSize. - It's better to use
kernel32 = ctypes.WinDLL('kernel32', use_last_error=True)instead ofctypes.windll.kernel32. Thectypes.windllloader cachesWinDLLinstances, which cache function prototypes. This can cause conflicts between packages. Also, a separateWinDLLinstance can setuse_last_error=Trueto enable the use of reliablectypes.get_last_error()instead of less reliablekernel32.GetLastError(). - ctypes opens scripts up to problems with data loss and crashes due to memory and stack corruption. To help avoid this, in general it's better to define function prototypes such as
kernel32.GetSystemInfo.argtypes = (ctypes.POINTER(SYSTEM_INFO),). ctypes will type check arguments ifargtypesis set, which helps to ensure that low-level C code is never passed bad arguments.
I’m not sure what is correct. Maybe before deciding about that one should discuss what is useful. I have use cases for wanting to know
- code of what architecture(s) the machine is capable of running, so that when I’m installing software, I can install the version compiled for that architecture. This is what to me
platform.machine()is documented to do, and does on Windows in 3.12.0a1: return something that depends only on the machine (including installed OS) and not on the running interpreter. However the documentation is not very clear, and apparently opinions differ on that (see #96993). An ideal function for this purpose would return a list of architectures in order of preference, maybe even marked by whether supported natively or in emulation. - the architecture of the running interpreter, so that in a setuptools setup.py I know what architecture we are building for (which wouldn’t necessarily need to be the same, but as far as I know there is no support for cross-compilation, so it is in practice, and I have not found any way of querying the latter directly) and can supply the corresponding libraries. This seems what
platform.architecture()was intended to do, but does not on Windows (because it bins all architectures into the two categories “32bit” and “64bit”) nor on macOS (because it looks at the binary, not the process, and may choose wrong when the binary is fat).
So far I do not have a use case for wanting to know the emulated architecture, but it may exist.
I would consider consistency across platforms very desirable (and, at the moment, lacking). What would also be useful in that regard is if one could agree on a known set of possible values for an “architecture”, in particular choose one of “x86_64” and “AMD64” and one of “aarch64” and “ARM64” and stick to it.
(Thanks for the ctypes tips! Omitting wReserved was an oversight and I didn’t notice that my code was just working by accident due to alignment. I wasn’t aware of the second point, and left out the third in a quick and minimal script.)