modify_model_interface removing input float tensor is not completed, memory allocator occupied this dummy tensor

[jimch3n]

1. System information

TFLM Tensorflow r2.5

2. Code

code in tensorflow\tensorflow\lite\tools\optimize\modify_model_interface.cc

Function: RemoveInputTensor code: c // Removes the input tensor and the related operator. for (auto tot : inputs) { SubGraphT* subgraph = model->subgraphs.at(tot.subgraph_index).get(); TFLITE_DCHECK(tot.input_index < subgraph->tensors.size()); TFLITE_DCHECK(tot.op_index < subgraph->operators.size()); if (tot.input_index >= original_number_tensors) { subgraph->tensors.erase(subgraph->tensors.begin() + tot.input_index); // DID NOT REMOVED } subgraph->operators.erase(subgraph->operators.begin() + tot.op_index); subgraph->inputs[tot.model_index] = tot.output_index; }

1. We have float input/output from quantization aware model to, reduce tensor arena size, using tool "modify_model_interface" utility convert to int8 - int8.
2. Convert result can run successfully, but memory allocator show unknown scratch buffer used. To check it print memory plan enable TF_LITE_SHOW_MEMORY_USE in micro_allocator.cc, that show original float input tensor still exist. 0 (id=0): size=24576, offset=0, first_used=-1 last_used=-1

I think removing shift all other tensor index in the graph, that could not a easy to remove. To workaround this I was resizing the dummy float input tensor to smaller one, and keep as it.

Thanks Jim

[abattery]

Thanks for filing the feature request! Would you mind uploading an affected TFLite model file or sharing the reproducible steps?

[jimch3n]

• feedback_google.zip

Here are the models before an after convert using modify_model_interface

• Reproduce command is

bazel run -c opt //tensorflow/lite/tools/optimize:modify_model_interface_main -- check_point_epoch0029_qua.tflite v28_check_point_epoch0029_main_v.tflite int8 int8

• below figure is the screen shot to loading model been quantized 'v28_check_point_epoch0029_main_v.tflite'