neural-compressor
neural-compressor copied to clipboard
Published
20 hours ago •
intel
intel
io.UnsupportedOperation: fileno
Hi, I ran the resnet based quantization notebook. I am getting the following error. Please suggest me to solve this issue.
Used !pip install tensorflow==2.12.0 !pip install neural-compressor==2.3.2
2024-04-03 17:45:54 [WARNING] Output tensor names should not be empty.
2024-04-03 17:45:54 [WARNING] Input tensor names is empty.
2024-04-03 17:45:54 [INFO] Start auto tuning.
2024-04-03 17:45:54 [INFO] Execute the tuning process due to detect the evaluation function.
2024-04-03 17:45:54 [INFO] Adaptor has 5 recipes.
2024-04-03 17:45:54 [INFO] 0 recipes specified by user.
2024-04-03 17:45:54 [INFO] 3 recipes require future tuning.
2024-04-03 17:45:55 [INFO] *** Initialize auto tuning
2024-04-03 17:45:55 [INFO] {
2024-04-03 17:45:55 [INFO] 'PostTrainingQuantConfig': {
2024-04-03 17:45:55 [INFO] 'AccuracyCriterion': {
2024-04-03 17:45:55 [INFO] 'criterion': 'relative',
2024-04-03 17:45:55 [INFO] 'higher_is_better': True,
2024-04-03 17:45:55 [INFO] 'tolerable_loss': 0.01,
2024-04-03 17:45:55 [INFO] 'absolute': None,
2024-04-03 17:45:55 [INFO] 'keys': <bound method AccuracyCriterion.keys of <neural_compressor.config.AccuracyCriterion object at 0x7fe19a6a3940>>,
2024-04-03 17:45:55 [INFO] 'relative': 0.01
2024-04-03 17:45:55 [INFO] },
2024-04-03 17:45:55 [INFO] 'approach': 'post_training_static_quant',
2024-04-03 17:45:55 [INFO] 'backend': 'default',
2024-04-03 17:45:55 [INFO] 'calibration_sampling_size': [
2024-04-03 17:45:55 [INFO] 50,
2024-04-03 17:45:55 [INFO] 100
2024-04-03 17:45:55 [INFO] ],
2024-04-03 17:45:55 [INFO] 'device': 'cpu',
2024-04-03 17:45:55 [INFO] 'diagnosis': False,
2024-04-03 17:45:55 [INFO] 'domain': 'auto',
2024-04-03 17:45:55 [INFO] 'example_inputs': None,
2024-04-03 17:45:55 [INFO] 'excluded_precisions': [
2024-04-03 17:45:55 [INFO] 'bf16'
2024-04-03 17:45:55 [INFO] ],
2024-04-03 17:45:55 [INFO] 'framework': 'tensorflow',
2024-04-03 17:45:55 [INFO] 'inputs': [
2024-04-03 17:45:55 [INFO] ],
2024-04-03 17:45:55 [INFO] 'model_name': '',
2024-04-03 17:45:55 [INFO] 'ni_workload_name': 'quantization',
2024-04-03 17:45:55 [INFO] 'op_name_dict': None,
2024-04-03 17:45:55 [INFO] 'op_type_dict': None,
2024-04-03 17:45:55 [INFO] 'outputs': [
2024-04-03 17:45:55 [INFO] ],
2024-04-03 17:45:55 [INFO] 'quant_format': 'default',
2024-04-03 17:45:55 [INFO] 'quant_level': 'auto',
2024-04-03 17:45:55 [INFO] 'recipes': {
2024-04-03 17:45:55 [INFO] 'smooth_quant': False,
2024-04-03 17:45:55 [INFO] 'smooth_quant_args': {
2024-04-03 17:45:55 [INFO] },
2024-04-03 17:45:55 [INFO] 'layer_wise_quant': False,
2024-04-03 17:45:55 [INFO] 'layer_wise_quant_args': {
2024-04-03 17:45:55 [INFO] },
2024-04-03 17:45:55 [INFO] 'fast_bias_correction': False,
2024-04-03 17:45:55 [INFO] 'weight_correction': False,
2024-04-03 17:45:55 [INFO] 'gemm_to_matmul': True,
2024-04-03 17:45:55 [INFO] 'graph_optimization_level': None,
2024-04-03 17:45:55 [INFO] 'first_conv_or_matmul_quantization': True,
2024-04-03 17:45:55 [INFO] 'last_conv_or_matmul_quantization': True,
2024-04-03 17:45:55 [INFO] 'pre_post_process_quantization': True,
2024-04-03 17:45:55 [INFO] 'add_qdq_pair_to_weight': False,
2024-04-03 17:45:55 [INFO] 'optypes_to_exclude_output_quant': [
2024-04-03 17:45:55 [INFO] ],
2024-04-03 17:45:55 [INFO] 'dedicated_qdq_pair': False,
2024-04-03 17:45:55 [INFO] 'rtn_args': {
2024-04-03 17:45:55 [INFO] },
2024-04-03 17:45:55 [INFO] 'awq_args': {
2024-04-03 17:45:55 [INFO] },
2024-04-03 17:45:55 [INFO] 'gptq_args': {
2024-04-03 17:45:55 [INFO] },
2024-04-03 17:45:55 [INFO] 'teq_args': {
2024-04-03 17:45:55 [INFO] },
2024-04-03 17:45:55 [INFO] 'scale_propagation_max_pooling': True,
2024-04-03 17:45:55 [INFO] 'scale_propagation_concat': True
2024-04-03 17:45:55 [INFO] },
2024-04-03 17:45:55 [INFO] 'reduce_range': None,
2024-04-03 17:45:55 [INFO] 'TuningCriterion': {
2024-04-03 17:45:55 [INFO] 'max_trials': 100,
2024-04-03 17:45:55 [INFO] 'objective': [
2024-04-03 17:45:55 [INFO] 'performance'
2024-04-03 17:45:55 [INFO] ],
2024-04-03 17:45:55 [INFO] 'strategy': 'basic',
2024-04-03 17:45:55 [INFO] 'strategy_kwargs': None,
2024-04-03 17:45:55 [INFO] 'timeout': 0
2024-04-03 17:45:55 [INFO] },
2024-04-03 17:45:55 [INFO] 'use_bf16': False
2024-04-03 17:45:55 [INFO] }
2024-04-03 17:45:55 [INFO] }
2024-04-03 17:45:55 [WARNING] [Strategy] Please install `mpi4py` correctly if using distributed tuning; otherwise, ignore this warning.
2024-04-03 17:45:56 [WARNING] Found possible input node names: ['input'], output node names: ['predict'].
2024-04-03 17:45:57 [INFO] ConvertLayoutOptimizer elapsed time: 0.99 ms
2024-04-03 17:45:57 [INFO] Pass ConvertPlaceholderToConst elapsed time: 10.88 ms
2024-04-03 17:45:57 [INFO] Pass SwitchOptimizer elapsed time: 11.71 ms
2024-04-03 17:45:58 [INFO] Pass GrapplerOptimizer elapsed time: 1271.68 ms
WARNING:tensorflow:From /usr/local/lib/python3.10/dist-packages/neural_compressor/adaptor/tf_utils/util.py:384: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This API was designed for TensorFlow v1. See https://www.tensorflow.org/guide/migrate for instructions on how to migrate your code to TensorFlow v2.
2024-04-03 17:45:58 [INFO] Pass StripUnusedNodesOptimizer elapsed time: 27.96 ms
2024-04-03 17:45:58 [INFO] Pass RemoveTrainingNodesOptimizer elapsed time: 16.04 ms
2024-04-03 17:45:58 [INFO] Pass SplitSharedInputOptimizer elapsed time: 7.08 ms
2024-04-03 17:45:58 [INFO] Pass GraphFoldConstantOptimizer elapsed time: 3.81 ms
2024-04-03 17:45:58 [INFO] Pass FuseDecomposedBNOptimizer elapsed time: 12.27 ms
2024-04-03 17:45:58 [INFO] Pass FuseColumnWiseMulOptimizer elapsed time: 9.19 ms
2024-04-03 17:45:58 [INFO] Pass StripUnusedNodesOptimizer elapsed time: 20.38 ms
2024-04-03 17:45:58 [INFO] Pass GraphCseOptimizer elapsed time: 11.89 ms
2024-04-03 17:45:59 [INFO] Pass FoldBatchNormNodesOptimizer elapsed time: 624.99 ms
2024-04-03 17:45:59 [INFO] Pass RenameBatchNormOptimizer elapsed time: 9.45 ms
2024-04-03 17:45:59 [INFO] Pass ConvertLeakyReluOptimizer elapsed time: 4.42 ms
2024-04-03 17:45:59 [INFO] Pass ConvertAddToBiasAddOptimizer elapsed time: 7.14 ms
2024-04-03 17:45:59 [INFO] Pass FuseTransposeReshapeOptimizer elapsed time: 6.01 ms
2024-04-03 17:45:59 [INFO] Pass FuseConvWithMathOptimizer elapsed time: 6.57 ms
2024-04-03 17:45:59 [INFO] Pass ExpandDimsOptimizer elapsed time: 6.32 ms
2024-04-03 17:45:59 [INFO] Pass FetchWeightFromReshapeOptimizer elapsed time: 6.12 ms
2024-04-03 17:45:59 [INFO] Pass MoveSqueezeAfterReluOptimizer elapsed time: 5.86 ms
2024-04-03 17:45:59 [WARNING] Found possible input node names: ['input'], output node names: ['predict'].
2024-04-03 17:45:59 [INFO] Pass InjectDummyBiasAddOptimizer elapsed time: 10.9 ms
2024-04-03 17:45:59 [WARNING] All replaced equivalent node types are {}
2024-04-03 17:45:59 [INFO] Pass StripEquivalentNodesOptimizer elapsed time: 36.49 ms
2024-04-03 17:46:00 [INFO] Pass Pre Optimization elapsed time: 3384.37 ms
2024-04-03 17:46:01 [INFO] Get FP32 model baseline.
0%| | 0/32 [00:00<?, ?it/s]2024-04-03 17:46:01 [WARNING] Found possible input node names: ['input'], output node names: ['predict'].
94%|█████████▍| 30/32 [02:32<00:10, 5.08s/it]
2024-04-03 17:48:33 [INFO] Save tuning history to /content/nc_workspace/2024-04-03_17-45-25/./history.snapshot.
2024-04-03 17:48:33 [INFO] FP32 baseline is: [Accuracy: 0.7302, Duration (seconds): 152.3777]
2024-04-03 17:48:33 [INFO] Quantize the model with default config.
2024-04-03 17:48:33 [WARNING] Please note that calibration sampling size 50 isn't divisible exactly by batch size 32. So the real sampling size is 64.
2024-04-03 17:48:34 [WARNING] Found possible input node names: ['input'], output node names: ['predict'].
2024-04-03 17:48:35 [WARNING] Found possible input node names: ['input'], output node names: ['predict'].
2024-04-03 17:48:35 [WARNING] Found possible input node names: ['input'], output node names: ['predict'].
2024-04-03 17:48:37 [INFO] Pass Quantization elapsed time: 1409.96 ms
2024-04-03 17:48:39 [ERROR] Fail to quantize graph due to fileno.
2024-04-03 17:48:39 [ERROR] Unexpected exception UnsupportedOperation('fileno') happened during tuning.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/neural_compressor/quantization.py", line 223, in fit
strategy.traverse()
File "/usr/local/lib/python3.10/dist-packages/neural_compressor/strategy/auto.py", line 140, in traverse
super().traverse()
File "/usr/local/lib/python3.10/dist-packages/neural_compressor/strategy/strategy.py", line 504, in traverse
q_model = self.adaptor.quantize(copy.deepcopy(tune_cfg), self.model, self.calib_dataloader, self.q_func)
File "/usr/local/lib/python3.10/dist-packages/neural_compressor/utils/utility.py", line 301, in fi
res = func(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/neural_compressor/adaptor/tensorflow.py", line 700, in quantize
).convert()
File "/usr/local/lib/python3.10/dist-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 401, in convert
model = self.quantize()
File "/usr/local/lib/python3.10/dist-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 555, in quantize
with CaptureOutputToFile(tmp_dump_file):
File "/usr/local/lib/python3.10/dist-packages/neural_compressor/utils/utility.py", line 448, in __init__
self.orig_stream_fileno = stream.fileno()
io.UnsupportedOperation: fileno
2024-04-03 17:48:39 [ERROR] Specified timeout or max trials is reached! Not found any quantized model which meet accuracy goal. Exit.
Thanks in Advance.
Hi @jashokkumar83
It's very likely that the issue is caused by versions of related dependencies. Could you please try the following combination:
jupyter_client 7.4.2 jupyter-core 4.11.1 jupyter-server 1.21.0 jupyterlab-pygments 0.2.2 notebook 6.5.1 notebook_shim 0.2.0
