openvino.genai icon indicating copy to clipboard operation
openvino.genai copied to clipboard

Log model properties

Open rasapala opened this issue 7 months ago • 0 comments

Adding possibility to get and log compiled model properties. Very useful for benchmarking and comparing settings on different hardware.

Example output: Loaded model configuration: AFFINITY: CORE CPU_DENORMALS_OPTIMIZATION: NO CPU_SPARSE_WEIGHTS_DECOMPRESSION_RATE: 1 DYNAMIC_QUANTIZATION_GROUP_SIZE: 32 ENABLE_CPU_PINNING: YES ENABLE_HYPER_THREADING: NO EXECUTION_DEVICES: CPU EXECUTION_MODE_HINT: PERFORMANCE INFERENCE_NUM_THREADS: 24 INFERENCE_PRECISION_HINT: f32 KV_CACHE_PRECISION: f16 LOG_LEVEL: LOG_NONE MODEL_DISTRIBUTION_POLICY: NETWORK_NAME: Model0 NUM_STREAMS: 1 OPTIMAL_NUMBER_OF_INFER_REQUESTS: 1 PERFORMANCE_HINT: LATENCY PERFORMANCE_HINT_NUM_REQUESTS: 0 PERF_COUNT: YES SCHEDULING_CORE_TYPE: ANY_CORE

rasapala avatar Jul 24 '24 15:07 rasapala