New version of cpuinfo is very slow (4.0.0 versus 5.0.0 or higher)...

[nicolargo]

trafficstars

Hi,

i use cpuinfo in Glances to display cpu info...

An issue as been openned by a user: see https://github.com/nicolargo/glances/issues/1700

It is also reproduced on my side:

>>> python test with py-cpuinfo version 4.0.0 (or lower)
Python 2.7.18rc1 (default, Apr  7 2020, 12:05:55) 
[GCC 9.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import time, cpuinfo
>>> s = time.time(); cpuinfo.get_cpu_info(); time.time()-s
{'count': 4, 'model': 69, 'extended_model': 0, 'l2_cache_line_size': 0, 'hz_advertised': '1.8000 GHz', 'family': 6, 'bits': 64, 'brand': 'Intel(R) Core(TM) i7-4500U CPU @ 1.80GHz', 'vendor_id': 'GenuineIntel', 'flags': ['abm', 'acpi', 'aes', 'aperfmperf', 'apic', 'arat', 'arch_perfmon', 'avx', 'avx2', 'bmi1', 'bmi2', 'bts', 'clflush', 'cmov', 'constant_tsc', 'cpuid', 'cpuid_fault', 'cx16', 'cx8', 'de', 'ds_cpl', 'dtes64', 'dtherm', 'dts', 'epb', 'ept', 'ept_ad', 'erms', 'est', 'f16c', 'flexpriority', 'flush_l1d', 'fma', 'fpu', 'fsgsbase', 'fxsr', 'ht', 'ibpb', 'ibrs', 'ida', 'invpcid', 'invpcid_single', 'lahf_lm', 'lm', 'mca', 'mce', 'md_clear', 'mmx', 'monitor', 'movbe', 'msr', 'mtrr', 'nonstop_tsc', 'nopl', 'nx', 'pae', 'pat', 'pbe', 'pcid', 'pclmulqdq', 'pdcm', 'pdpe1gb', 'pebs', 'pge', 'pln', 'pni', 'popcnt', 'pse', 'pse36', 'pti', 'pts', 'rdrand', 'rdtscp', 'rep_good', 'sdbg', 'sep', 'smep', 'ss', 'ssbd', 'sse', 'sse2', 'sse4_1', 'sse4_2', 'ssse3', 'stibp', 'syscall', 'tm', 'tm2', 'tpr_shadow', 'tsc', 'tsc_adjust', 'tsc_deadline_timer', 'vme', 'vmx', 'vnmi', 'vpid', 'xsave', 'xsaveopt', 'xtopology', 'xtpr'], 'extended_family': 0, 'hardware': '', 'raw_arch_string': 'x86_64', 'l2_cache_size': '4096 KB', 'l2_cache_associativity': 0, 'stepping': 1, 'hz_actual_raw': (1463646000, 0), 'hz_actual': '1.4636 GHz', 'arch': 'X86_64', 'hz_advertised_raw': (1800000000, 0), 'processor_type': 0}
0.021673202514648438

>>> python with py-cpuinfo version 5.0.0 or higher
Python 2.7.18rc1 (default, Apr  7 2020, 12:05:55) 
[GCC 9.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import time, cpuinfo
>>> s = time.time(); cpuinfo.get_cpu_info(); time.time()-s
{'python_version': '2.7.18.candidate.1 (64 bit)', 'family': 6, 'l1_instruction_cache_size': '64 KiB', 'stepping': 1, 'l1_data_cache_size': '64 KiB', 'brand_raw': 'Intel(R) Core(TM) i7-4500U CPU @ 1.80GHz', 'l3_cache_size': 4194304, 'arch_string_raw': 'x86_64', 'cpuinfo_version': [7, 0, 0], 'cpuinfo_version_string': '7.0.0', 'bits': 64, 'hz_advertised': [1800000000, 0], 'l2_cache_associativity': 6, 'vendor_id_raw': 'GenuineIntel', 'arch': 'X86_64', 'model': 69, 'count': 4, 'l2_cache_line_size': 256, 'l2_cache_size': '512 KiB', 'hz_advertised_friendly': '1.8000 GHz', 'hz_actual_friendly': '1.8004 GHz', 'flags': ['abm', 'acpi', 'aes', 'aperfmperf', 'apic', 'arat', 'arch_perfmon', 'avx', 'avx2', 'bmi1', 'bmi2', 'bts', 'clflush', 'cmov', 'constant_tsc', 'cpuid', 'cpuid_fault', 'cx16', 'cx8', 'de', 'ds_cpl', 'dtes64', 'dtherm', 'dts', 'epb', 'ept', 'ept_ad', 'erms', 'est', 'f16c', 'flexpriority', 'flush_l1d', 'fma', 'fpu', 'fsgsbase', 'fxsr', 'ht', 'ibpb', 'ibrs', 'ida', 'invpcid', 'invpcid_single', 'lahf_lm', 'lm', 'mca', 'mce', 'md_clear', 'mmx', 'monitor', 'movbe', 'msr', 'mtrr', 'nonstop_tsc', 'nopl', 'nx', 'osxsave', 'pae', 'pat', 'pbe', 'pcid', 'pclmulqdq', 'pdcm', 'pdpe1gb', 'pebs', 'pge', 'pln', 'pni', 'popcnt', 'pse', 'pse36', 'pti', 'pts', 'rdrand', 'rdrnd', 'rdtscp', 'rep_good', 'sdbg', 'sep', 'smep', 'ss', 'ssbd', 'sse', 'sse2', 'sse4_1', 'sse4_2', 'ssse3', 'stibp', 'syscall', 'tm', 'tm2', 'tpr_shadow', 'tsc', 'tsc_adjust', 'tsc_deadline_timer', 'tscdeadline', 'vme', 'vmx', 'vnmi', 'vpid', 'xsave', 'xsaveopt', 'xtopology', 'xtpr'], 'hz_actual': [1800422000, 0]}
1.118035078048706

0.02 second versus 1.11 second...

In your bug report please include:

• CPU architecture: Intel
• Operating System: Ubuntu
• Python version: 2.7
• Version of py-cpuinfo: 0.2.3 versus 7.0.0

[workhorsy]

In py-cpuinfo 5.0.0 and greater, the1 second delay is from when py-cpuinfo runs the CPUID instruction to measure the CPU Hz. It gets the CPU ticks, sleeps for 1 second, measures the ticks again, then returns the difference. So there isn't any way to get around wasting that 1 second. In py-cpuinfo 4.0.0 it is most likely failing to run CPUID for that OS configuration. Then it looks like it is running very quickly, when it is just failing.

Here is a short example of basically what we are doing:

# Works on py-cpuinfo  7.0.0
# Warning: Don't use this in the main python process. As it can crash
# entire Python runtime, if there is an error. This also only Works
# on x86 and x86_64 CPUs.
if __name__ == '__main__':
	from time import sleep
	from sys import stdout
	from cpuinfo import CPUID

	# Generate the CPUID machine code function
	cpuid = CPUID()
	ticks_fn = cpuid.get_ticks_func()

	try:
		while True:
			# Measure the Hz
			start = ticks_fn.func()
			sleep(1)
			end = ticks_fn.func()
			hz_measured = (end - start)

			# Print Hz
			print(hz_measured)
			stdout.flush()
	# Exit and cleanup on Ctrl+c
	except KeyboardInterrupt:
		pass
	finally:
		if ticks_fn: ticks_fn.free()

I will look into if there is a better way to do this.

[jlquinn]

Hi, if I'm reading the code correctly, you try every function and add fields as you go. So at least on linux, lscpu already provides this info and you don't need to run the CPUID timer.

Another way around this might be to provide a routine to get to specific fields faster. For example, I only need the flags, and the timer is not even a useful delay.

[nschloe]

Same issue here with tiptop. I only use py-cpuinfo to get the model name of the CPU, so all of the computation is actually wasted. I would support @jlquinn's suggestion to allow the user to only populate certain fields.

[jacobtomlinson]

I also just ran into this. I wanted to use this library to get some info out like CPU model name but taking more than 1s is too long for my application.

A few thoughts:

• Add flags to allow enabling/disabling different stats like CPUID?
• Allow choosing which fields you want and skip any gathering methods that do not collect them?
• Shorten the 1s sleep to 100ms and multiply the ticks by 10? (or even 10ms and multiply by 100?)