Hi Team, thanks for the wonderful tool. I am trying to quantize savedmodel with multiple inputs but facing below issue.
Code:
import tensorflow as tf
from neural_compressor.experimental import Quantization, common
quantizer = Quantization()
quantizer.model = '../model/'
dataset = quantizer.dataset('dummy', shape=[(1,256),(1,256)])
quantizer.calib_dataloader = common.DataLoader(dataset)
quantized_model = quantizer.fit()
output_path = '../model/'
quantized_model.save(output_path)
2022-07-15 10:10:03 [WARNING] Force convert framework model to neural_compressor model.
2022-07-15 10:10:03.488839: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-07-15 10:10:03.490012: I tensorflow/core/common_runtime/process_util.cc:146] Creating new thread pool with default inter op setting:
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:05 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:08 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:10 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:10 [WARNING] Output tensor names should not be empty.
2022-07-15 10:10:10 [WARNING] Input tensor names should not be empty.
2022-07-15 10:10:10 [INFO] Generate a fake evaluation function.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:13 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:13.470210: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >=
2022-07-15 10:10:13.470453: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session
2022-07-15 10:10:14.615203: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >=
2022-07-15 10:10:14.615386: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:18 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:20 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:20 [INFO] ConvertLayoutOptimizer elapsed time: 0.32 ms
2022-07-15 10:10:23.822868: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >=
2022-07-15 10:10:23.823242: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session
2022-07-15 10:10:24 [INFO] Pass GrapplerOptimizer elapsed time: 3781.35 ms
2022-07-15 10:10:24 [INFO] Pass SwitchOptimizer elapsed time: 277.86 ms
2022-07-15 10:10:24 [INFO] Pass RemoveTrainingNodesOptimizer elapsed time: 275.21 ms
2022-07-15 10:10:24 [INFO] Pass SplitSharedInputOptimizer elapsed time: 5.65 ms
2022-07-15 10:10:25 [INFO] Pass GraphFoldConstantOptimizer elapsed time: 276.08 ms
2022-07-15 10:10:25 [INFO] Pass FuseColumnWiseMulOptimizer elapsed time: 275.88 ms
WARNING:tensorflow:From /opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/util.py:322: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.compat.v1.graph_util.extract_sub_graph
2022-07-15 10:10:25 [WARNING] From /opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/util.py:322: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.compat.v1.graph_util.extract_sub_graph
2022-07-15 10:10:26 [INFO] Pass StripUnusedNodesOptimizer elapsed time: 1121.14 ms
2022-07-15 10:10:26 [INFO] Pass GraphCseOptimizer elapsed time: 279.52 ms
2022-07-15 10:10:27 [INFO] Pass FuseDecomposedBNOptimizer elapsed time: 539.8 ms
2022-07-15 10:10:27 [INFO] Pass FoldBatchNormNodesOptimizer elapsed time: 277.56 ms
2022-07-15 10:10:27 [INFO] Pass UpdateEnterOptimizer elapsed time: 277.27 ms
2022-07-15 10:10:28 [INFO] Pass ConvertLeakyReluOptimizer elapsed time: 279.11 ms
2022-07-15 10:10:28 [INFO] Pass ConvertAddToBiasAddOptimizer elapsed time: 275.88 ms
2022-07-15 10:10:28 [INFO] Pass FuseTransposeReshapeOptimizer elapsed time: 277.57 ms
2022-07-15 10:10:29 [INFO] Pass FuseConvWithMathOptimizer elapsed time: 276.92 ms
2022-07-15 10:10:29 [INFO] Pass ExpandDimsOptimizer elapsed time: 275.92 ms
2022-07-15 10:10:29 [INFO] Pass FetchWeightFromReshapeOptimizer elapsed time: 279.25 ms
2022-07-15 10:10:29 [INFO] Pass InjectDummyBiasAddOptimizer elapsed time: 282.46 ms
2022-07-15 10:10:30 [INFO] Pass MoveSqueezeAfterReluOptimizer elapsed time: 273.58 ms
2022-07-15 10:10:31 [INFO] Pass Pre Optimization elapsed time: 15803.25 ms
2022-07-15 10:10:32 [INFO] Get FP32 model baseline.
2022-07-15 10:10:32 [INFO] Save tuning history to /home/jupyter/model-compressor/conversion_notebooks/nc_workspace/2022-07-15_10-10-02/./history.snapshot.
2022-07-15 10:10:32 [INFO] FP32 baseline is: [Accuracy: 1.0000, Duration (seconds): 0.0000]
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:34 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:36 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:39 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:41 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:41 [WARNING] Found possible input node names: ['attention_mask', 'input_ids'], output node names: ['global_average_pooling1d'].
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:44 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:46 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:48 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:51 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:51 [WARNING] Found possible input node names: ['attention_mask', 'input_ids'], output node names: ['global_average_pooling1d'].
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:54 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:10:54.913870: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >=
2022-07-15 10:10:54.914104: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session
2022-07-15 10:10:56.035642: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >=
2022-07-15 10:10:56.035843: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session
2022-07-15 10:11:04 [INFO] Pass Quantization elapsed time: 5884.0 ms
2022-07-15 10:11:07 [INFO] Start sampling on calibration dataset.
2022-07-15 10:11:07.605083: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:354] MLIR V1 optimization pass is not enabled
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1377, in _do_call
return fn(*args)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1361, in _run_fn
target_list, run_metadata)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1455, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[0,0] = is not in [0, )
[[{{node text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup}}]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 573, in quantize
self._inference(self._sampling_model)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 181, in _inference
_ = model.sess.run(output_tensor, feed_dict) if model.iter_op==[]
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 968, in run
run_metadata_ptr)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1191, in _run
feed_dict_tensor, options, run_metadata)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1371, in _do_run
run_metadata)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1396, in _do_call
raise type(e)(node_def, op, message) # pylint: disable=no-value-for-parameter
tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph execution error:
Detected at node 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup' defined at (most recent call last):
File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py", line 17, in
app.launch_new_instance()
File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 976, in launch_instance
app.start()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 712, in start
self.io_loop.start()
File "/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 199, in start
self.asyncio_loop.run_forever()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
self._run_once()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
handle._run()
File "/opt/conda/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
await self.process_one()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 499, in process_one
await dispatch(*args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
await result
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 730, in execute_request
reply_content = await reply_content
File "/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py", line 387, in do_execute
cell_id=cell_id,
File "/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py", line 528, in run_cell
return super().run_cell(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2975, in run_cell
raw_cell, store_history, silent, shell_futures, cell_id
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3029, in _run_cell
return runner(coro)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py", line 78, in pseudo_sync_runner
coro.send(None)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3257, in run_cell_async
interactivity=interactivity, compiler=compiler, result=result)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3472, in run_ast_nodes
if (await self.run_code(code, result, async=asy)):
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3552, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_28346/3554570389.py", line 5, in
quantized_model = quantizer.fit()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 212, in call
return super(Quantization, self).call()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py", line 215, in call
results = self.execute()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 151, in execute
self.strategy.traverse()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py", line 393, in traverse
tune_cfg, self.model, self.calib_dataloader, self.q_func)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py", line 242, in fi
res = func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 536, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 268, in convert
model = self.quantize()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 569, in quantize
self._sampling_model.graph_def = sampling_graph_def
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 715, in graph_def
self._output_tensor_names)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 298, in graph_def_session
tf.import_graph_def(model, name='')
Node: 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup'
indices[0,0] = is not in [0, )
[[{{node text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup}}]]
Original stack trace for 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup':
File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py", line 17, in
app.launch_new_instance()
File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 976, in launch_instance
app.start()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 712, in start
self.io_loop.start()
File "/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 199, in start
self.asyncio_loop.run_forever()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
self._run_once()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
handle._run()
File "/opt/conda/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
await self.process_one()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 499, in process_one
await dispatch(*args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
await result
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 730, in execute_request
reply_content = await reply_content
File "/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py", line 387, in do_execute
cell_id=cell_id,
File "/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py", line 528, in run_cell
return super().run_cell(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2975, in run_cell
raw_cell, store_history, silent, shell_futures, cell_id
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3029, in _run_cell
return runner(coro)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py", line 78, in pseudo_sync_runner
coro.send(None)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3257, in run_cell_async
interactivity=interactivity, compiler=compiler, result=result)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3472, in run_ast_nodes
if (await self.run_code(code, result, async=asy)):
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3552, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_28346/3554570389.py", line 5, in
quantized_model = quantizer.fit()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 212, in call
return super(Quantization, self).call()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py", line 215, in call
results = self.execute()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 151, in execute
self.strategy.traverse()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py", line 393, in traverse
tune_cfg, self.model, self.calib_dataloader, self.q_func)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py", line 242, in fi
res = func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 536, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 268, in convert
model = self.quantize()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 569, in quantize
self._sampling_model.graph_def = sampling_graph_def
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 715, in graph_def
self._output_tensor_names)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 298, in graph_def_session
tf.import_graph_def(model, name='')
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 561, in new_func
return func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 408, in import_graph_def
producer_op_list=producer_op_list)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 516, in _import_graph_def_internal
_ProcessNewOps(graph)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 247, in _ProcessNewOps
for new_op in graph._add_new_tf_operations(compute_devices=False): # pylint: disable=protected-access
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3906, in _add_new_tf_operations
for c_op in c_api_util.new_tf_operations(self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3906, in
for c_op in c_api_util.new_tf_operations(self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3787, in _create_op_from_tf_operation
ret = Operation(c_op, self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 2133, in init
self._traceback = tf_stack.extract_stack_for_node(self._c_op)
2022-07-15 10:11:08 [ERROR] Fail to quantize graph due to Graph execution error:
Detected at node 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup' defined at (most recent call last):
File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py", line 17, in
app.launch_new_instance()
File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 976, in launch_instance
app.start()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 712, in start
self.io_loop.start()
File "/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 199, in start
self.asyncio_loop.run_forever()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
self._run_once()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
handle._run()
File "/opt/conda/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
await self.process_one()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 499, in process_one
await dispatch(*args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
await result
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 730, in execute_request
reply_content = await reply_content
File "/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py", line 387, in do_execute
cell_id=cell_id,
File "/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py", line 528, in run_cell
return super().run_cell(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2975, in run_cell
raw_cell, store_history, silent, shell_futures, cell_id
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3029, in _run_cell
return runner(coro)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py", line 78, in pseudo_sync_runner
coro.send(None)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3257, in run_cell_async
interactivity=interactivity, compiler=compiler, result=result)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3472, in run_ast_nodes
if (await self.run_code(code, result, async=asy)):
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3552, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_28346/3554570389.py", line 5, in
quantized_model = quantizer.fit()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 212, in call
return super(Quantization, self).call()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py", line 215, in call
results = self.execute()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 151, in execute
self.strategy.traverse()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py", line 393, in traverse
tune_cfg, self.model, self.calib_dataloader, self.q_func)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py", line 242, in fi
res = func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 536, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 268, in convert
model = self.quantize()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 569, in quantize
self._sampling_model.graph_def = sampling_graph_def
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 715, in graph_def
self._output_tensor_names)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 298, in graph_def_session
tf.import_graph_def(model, name='')
Node: 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup'
indices[0,0] = is not in [0, )
[[{{node text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup}}]]
Original stack trace for 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup':
File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py", line 17, in
app.launch_new_instance()
File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 976, in launch_instance
app.start()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 712, in start
self.io_loop.start()
File "/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 199, in start
self.asyncio_loop.run_forever()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
self._run_once()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
handle._run()
File "/opt/conda/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
await self.process_one()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 499, in process_one
await dispatch(*args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
await result
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 730, in execute_request
reply_content = await reply_content
File "/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py", line 387, in do_execute
cell_id=cell_id,
File "/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py", line 528, in run_cell
return super().run_cell(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2975, in run_cell
raw_cell, store_history, silent, shell_futures, cell_id
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3029, in _run_cell
return runner(coro)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py", line 78, in pseudo_sync_runner
coro.send(None)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3257, in run_cell_async
interactivity=interactivity, compiler=compiler, result=result)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3472, in run_ast_nodes
if (await self.run_code(code, result, async=asy)):
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3552, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_28346/3554570389.py", line 5, in
quantized_model = quantizer.fit()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 212, in call
return super(Quantization, self).call()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py", line 215, in call
results = self.execute()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 151, in execute
self.strategy.traverse()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py", line 393, in traverse
tune_cfg, self.model, self.calib_dataloader, self.q_func)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py", line 242, in fi
res = func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 536, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 268, in convert
model = self.quantize()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 569, in quantize
self._sampling_model.graph_def = sampling_graph_def
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 715, in graph_def
self._output_tensor_names)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 298, in graph_def_session
tf.import_graph_def(model, name='')
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 561, in new_func
return func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 408, in import_graph_def
producer_op_list=producer_op_list)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 516, in _import_graph_def_internal
_ProcessNewOps(graph)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 247, in _ProcessNewOps
for new_op in graph._add_new_tf_operations(compute_devices=False): # pylint: disable=protected-access
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3906, in _add_new_tf_operations
for c_op in c_api_util.new_tf_operations(self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3906, in
for c_op in c_api_util.new_tf_operations(self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3787, in _create_op_from_tf_operation
ret = Operation(c_op, self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 2133, in init
self._traceback = tf_stack.extract_stack_for_node(self._c_op)
.
2022-07-15 10:11:08 [WARNING] Fail to forward with batch size=1, set to 1 now.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:10 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:13 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:15 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:18 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:18 [WARNING] Found possible input node names: ['attention_mask', 'input_ids'], output node names: ['global_average_pooling1d'].
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:21 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:23 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:25 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:27 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:28 [WARNING] Found possible input node names: ['attention_mask', 'input_ids'], output node names: ['global_average_pooling1d'].
WARNING:tensorflow:No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:31 [WARNING] No training configuration found in save file, so the model was not compiled. Compile it manually.
2022-07-15 10:11:31.485104: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >=
2022-07-15 10:11:31.485309: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session
2022-07-15 10:11:32.564946: I tensorflow/core/grappler/devices.cc:75] Number of eligible GPUs (core count >= 8, compute capability >=
2022-07-15 10:11:32.565120: I tensorflow/core/grappler/clusters/single_machine.cc:358] Starting new session
2022-07-15 10:11:40 [INFO] Pass Quantization elapsed time: 5611.09 ms
2022-07-15 10:11:43 [INFO] Start sampling on calibration dataset.
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 536, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 275, in convert
post_cse_graph_def = PostCseOptimizer(model.graph_def).do_transformation()
AttributeError: 'NoneType' object has no attribute 'graph_def'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1377, in _do_call
return fn(*args)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1361, in _run_fn
target_list, run_metadata)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1455, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[0,0] = is not in [0, )
[[{{node text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup}}]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 573, in quantize
self._inference(self._sampling_model)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 181, in _inference
_ = model.sess.run(output_tensor, feed_dict) if model.iter_op==[]
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 968, in run
run_metadata_ptr)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1191, in _run
feed_dict_tensor, options, run_metadata)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1371, in _do_run
run_metadata)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1396, in _do_call
raise type(e)(node_def, op, message) # pylint: disable=no-value-for-parameter
tensorflow.python.framework.errors_impl.InvalidArgumentError: Graph execution error:
Detected at node 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup' defined at (most recent call last):
File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py", line 17, in
app.launch_new_instance()
File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 976, in launch_instance
app.start()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 712, in start
self.io_loop.start()
File "/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 199, in start
self.asyncio_loop.run_forever()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
self._run_once()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
handle._run()
File "/opt/conda/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
await self.process_one()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 499, in process_one
await dispatch(*args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
await result
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 730, in execute_request
reply_content = await reply_content
File "/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py", line 387, in do_execute
cell_id=cell_id,
File "/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py", line 528, in run_cell
return super().run_cell(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2975, in run_cell
raw_cell, store_history, silent, shell_futures, cell_id
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3029, in _run_cell
return runner(coro)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py", line 78, in pseudo_sync_runner
coro.send(None)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3257, in run_cell_async
interactivity=interactivity, compiler=compiler, result=result)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3472, in run_ast_nodes
if (await self.run_code(code, result, async=asy)):
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3552, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_28346/3554570389.py", line 5, in
quantized_model = quantizer.fit()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 212, in call
return super(Quantization, self).call()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py", line 215, in call
results = self.execute()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 151, in execute
self.strategy.traverse()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py", line 393, in traverse
tune_cfg, self.model, self.calib_dataloader, self.q_func)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py", line 242, in fi
res = func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 551, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 268, in convert
model = self.quantize()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 569, in quantize
self._sampling_model.graph_def = sampling_graph_def
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 715, in graph_def
self._output_tensor_names)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 298, in graph_def_session
tf.import_graph_def(model, name='')
Node: 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup'
indices[0,0] = is not in [0, )
[[{{node text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup}}]]
Original stack trace for 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup':
File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py", line 17, in
app.launch_new_instance()
File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 976, in launch_instance
app.start()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 712, in start
self.io_loop.start()
File "/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 199, in start
self.asyncio_loop.run_forever()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
self._run_once()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
handle._run()
File "/opt/conda/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
await self.process_one()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 499, in process_one
await dispatch(*args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
await result
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 730, in execute_request
reply_content = await reply_content
File "/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py", line 387, in do_execute
cell_id=cell_id,
File "/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py", line 528, in run_cell
return super().run_cell(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2975, in run_cell
raw_cell, store_history, silent, shell_futures, cell_id
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3029, in _run_cell
return runner(coro)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py", line 78, in pseudo_sync_runner
coro.send(None)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3257, in run_cell_async
interactivity=interactivity, compiler=compiler, result=result)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3472, in run_ast_nodes
if (await self.run_code(code, result, async=asy)):
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3552, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_28346/3554570389.py", line 5, in
quantized_model = quantizer.fit()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 212, in call
return super(Quantization, self).call()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py", line 215, in call
results = self.execute()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 151, in execute
self.strategy.traverse()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py", line 393, in traverse
tune_cfg, self.model, self.calib_dataloader, self.q_func)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py", line 242, in fi
res = func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 551, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 268, in convert
model = self.quantize()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 569, in quantize
self._sampling_model.graph_def = sampling_graph_def
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 715, in graph_def
self._output_tensor_names)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 298, in graph_def_session
tf.import_graph_def(model, name='')
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 561, in new_func
return func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 408, in import_graph_def
producer_op_list=producer_op_list)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 516, in _import_graph_def_internal
_ProcessNewOps(graph)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 247, in _ProcessNewOps
for new_op in graph._add_new_tf_operations(compute_devices=False): # pylint: disable=protected-access
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3906, in _add_new_tf_operations
for c_op in c_api_util.new_tf_operations(self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3906, in
for c_op in c_api_util.new_tf_operations(self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3787, in _create_op_from_tf_operation
ret = Operation(c_op, self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 2133, in init
self._traceback = tf_stack.extract_stack_for_node(self._c_op)
2022-07-15 10:11:43 [ERROR] Fail to quantize graph due to Graph execution error:
Detected at node 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup' defined at (most recent call last):
File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py", line 17, in
app.launch_new_instance()
File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 976, in launch_instance
app.start()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 712, in start
self.io_loop.start()
File "/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 199, in start
self.asyncio_loop.run_forever()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
self._run_once()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
handle._run()
File "/opt/conda/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
await self.process_one()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 499, in process_one
await dispatch(*args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
await result
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 730, in execute_request
reply_content = await reply_content
File "/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py", line 387, in do_execute
cell_id=cell_id,
File "/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py", line 528, in run_cell
return super().run_cell(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2975, in run_cell
raw_cell, store_history, silent, shell_futures, cell_id
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3029, in _run_cell
return runner(coro)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py", line 78, in pseudo_sync_runner
coro.send(None)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3257, in run_cell_async
interactivity=interactivity, compiler=compiler, result=result)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3472, in run_ast_nodes
if (await self.run_code(code, result, async=asy)):
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3552, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_28346/3554570389.py", line 5, in
quantized_model = quantizer.fit()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 212, in call
return super(Quantization, self).call()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py", line 215, in call
results = self.execute()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 151, in execute
self.strategy.traverse()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py", line 393, in traverse
tune_cfg, self.model, self.calib_dataloader, self.q_func)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py", line 242, in fi
res = func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 551, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 268, in convert
model = self.quantize()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 569, in quantize
self._sampling_model.graph_def = sampling_graph_def
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 715, in graph_def
self._output_tensor_names)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 298, in graph_def_session
tf.import_graph_def(model, name='')
Node: 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup'
indices[0,0] = is not in [0, )
[[{{node text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup}}]]
Original stack trace for 'text_encoder/token_and_position_embedding/StatefulPartitionedCall/StatefulPartitionedCall/embedding_4/embedding_lookup':
File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
exec(code, run_globals)
File "/opt/conda/lib/python3.7/site-packages/ipykernel_launcher.py", line 17, in
app.launch_new_instance()
File "/opt/conda/lib/python3.7/site-packages/traitlets/config/application.py", line 976, in launch_instance
app.start()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelapp.py", line 712, in start
self.io_loop.start()
File "/opt/conda/lib/python3.7/site-packages/tornado/platform/asyncio.py", line 199, in start
self.asyncio_loop.run_forever()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 541, in run_forever
self._run_once()
File "/opt/conda/lib/python3.7/asyncio/base_events.py", line 1786, in _run_once
handle._run()
File "/opt/conda/lib/python3.7/asyncio/events.py", line 88, in _run
self._context.run(self._callback, *self._args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue
await self.process_one()
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 499, in process_one
await dispatch(*args)
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell
await result
File "/opt/conda/lib/python3.7/site-packages/ipykernel/kernelbase.py", line 730, in execute_request
reply_content = await reply_content
File "/opt/conda/lib/python3.7/site-packages/ipykernel/ipkernel.py", line 387, in do_execute
cell_id=cell_id,
File "/opt/conda/lib/python3.7/site-packages/ipykernel/zmqshell.py", line 528, in run_cell
return super().run_cell(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2975, in run_cell
raw_cell, store_history, silent, shell_futures, cell_id
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3029, in _run_cell
return runner(coro)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/async_helpers.py", line 78, in pseudo_sync_runner
coro.send(None)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3257, in run_cell_async
interactivity=interactivity, compiler=compiler, result=result)
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3472, in run_ast_nodes
if (await self.run_code(code, result, async=asy)):
File "/opt/conda/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3552, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "/tmp/ipykernel_28346/3554570389.py", line 5, in
quantized_model = quantizer.fit()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 212, in call
return super(Quantization, self).call()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/component.py", line 215, in call
results = self.execute()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 151, in execute
self.strategy.traverse()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py", line 393, in traverse
tune_cfg, self.model, self.calib_dataloader, self.q_func)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py", line 242, in fi
res = func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 551, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 268, in convert
model = self.quantize()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 569, in quantize
self._sampling_model.graph_def = sampling_graph_def
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 715, in graph_def
self._output_tensor_names)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/model/model.py", line 298, in graph_def_session
tf.import_graph_def(model, name='')
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 561, in new_func
return func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 408, in import_graph_def
producer_op_list=producer_op_list)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 516, in _import_graph_def_internal
_ProcessNewOps(graph)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/importer.py", line 247, in _ProcessNewOps
for new_op in graph._add_new_tf_operations(compute_devices=False): # pylint: disable=protected-access
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3906, in _add_new_tf_operations
for c_op in c_api_util.new_tf_operations(self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3906, in
for c_op in c_api_util.new_tf_operations(self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3787, in _create_op_from_tf_operation
ret = Operation(c_op, self)
File "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 2133, in init
self._traceback = tf_stack.extract_stack_for_node(self._c_op)
.
2022-07-15 10:11:43 [ERROR] Unexpected exception AttributeError("'NoneType' object has no attribute 'graph_def'") happened during tuning.
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 536, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 275, in convert
post_cse_graph_def = PostCseOptimizer(model.graph_def).do_transformation()
AttributeError: 'NoneType' object has no attribute 'graph_def'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/experimental/quantization.py", line 151, in execute
self.strategy.traverse()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/strategy/strategy.py", line 393, in traverse
tune_cfg, self.model, self.calib_dataloader, self.q_func)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/utils/utility.py", line 242, in fi
res = func(*args, **kwargs)
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tensorflow.py", line 551, in quantize
data_loader=data_loader).convert()
File "/opt/conda/lib/python3.7/site-packages/neural_compressor/adaptor/tf_utils/graph_converter.py", line 275, in convert
post_cse_graph_def = PostCseOptimizer(model.graph_def).do_transformation()
AttributeError: 'NoneType' object has no attribute 'graph_def'
2022-07-15 10:11:43 [ERROR] Specified timeout or max trials is reached! Not found any quantized model which meet accuracy goal. Exit.
AttributeError Traceback (most recent call last)
/tmp/ipykernel_28346/3554570389.py in
5 quantized_model = quantizer.fit()
6 output_path = '../models/gev8_image_encoder_quant/'
----> 7 quantized_model.save(output_path)
AttributeError: 'NoneType' object has no attribute 'save'
Any suggestion?
neural-compressor copied to clipboard
intel
