About to use "Quantize weights" method

[dalvlv]

I notice that the size of model.pb can be smaller if use "Quantize weights" method proposed by tensorflow-team. But I don't know how to use it. Can you think about this way？

[amir-abdi]

@dalvlv I added a new feature in the new commit to master branch. It exposes the quantize feature of tensorflow. I tested some small networks, and the size of the saved model was not different. Can you test your model and share the results in terms of:

• file size,
• time to run the network ?

To use the feature, set the -quantize flag to True

[dalvlv]

@amir-abdi I use the new "keras_to_tensorflow2.py", but it called a mistake named "No module named 'tensorflow.tools.graph_transforms'". My tensorflow version is 1.3.0 and I don't know why it doesn't work.

[amir-abdi]

@dalvlv This is a new feature of tensorflow and you need to a newer version. I have tested with 1.5, so 1.5 or higher works. https://www.tensorflow.org/performance/quantization

[dalvlv]

@amir-abdi sorry for that I spend a two-week vacation so that I can't reply you in time. I use tf 1.5 but the size of the saved model was not different, either. I thihk if the reason is the order of "freeze" and "quantize". "Freeze" should be the first , and then "quantize".

[dalvlv]

Hi, @amir-abdi, I first use “freeze” and then "quantize", then the model size becomes about 1/4 of the 'frozen' model. But when I put the "quantize" model into Android, it made a mistake, When I don’t use “quantize”， there's no mistake. The mistake in Android is "Not a valid TensorFlow Graph serialization: Node 'conv1d_21/bias/read' expects to be colocated with unknown node 'conv1d_21/bias", do you know why? Thank you .

[amir-abdi]

@dalvlv I experienced the same "model size not changing" issue. I assumed maybe my models are not big enough for quantization to take effect.

Based on your input, freezing first helps; but I wonder if the smaller network is losing some necessary components... If ran any further tests with Android and had more insight into this, let me know to update.

[dalvlv]

Hi, @amir-abdi I don't know which step should be first? "freezed"or"quantized"? Is the order constant? As for my model, smaller network results in the F1-score decreased by 6%, and I think it makes too much loss. Do you know what standard error is from "freezed" to "quantized "?

[srivatsankrishnan]

@amir-abdi I don't see a reduction in the model size. How do I know that the quantization feature works?

[Cozmo25]

The graph must be frozen before the transforms can be applied, so running convert_variables_to_constants() before TransformGraph() works.

[AmrMahmoud12]

The graph must be frozen before the transforms can be applied, so running convert_variables_to_constants() before TransformGraph() works.

how did you make this? if you reverse order , it will release error that you run before assignment . can you tell me what did you do exactly, i am stucked from 2 weeks in this point as I cant reduce the size

[AmrMahmoud12]

I am trying to convert style transfer model to tensorflow but failure with all I used tensorflow 1.5.0 , python=3.6.9 but didnt work for quantization I used this command python keras_to_tensorflow.py --input_model="model" --output_model="model.pb" --quantize="True" the result is the same file as the h5 file with no change can you till me what is the problem?

[Cozmo25]

The graph must be frozen before the transforms can be applied, so running convert_variables_to_constants() before TransformGraph() works.

how did you make this? if you reverse order , it will release error that you run before assignment . can you tell me what did you do exactly, i am stucked from 2 weeks in this point as I cant reduce the size

Add a flag to pass in the desired transforms:

flags.DEFINE_list('transforms', None,
		    'List of transforms to be applied to the frozen model if quantize is set to true. Should '
		     'be comma separated list.')

and change the FLAGS.quantize arg handler as follows:

if FLAGS.quantize:
        constant_graph = graph_util.convert_variables_to_constants(
	    sess,
	    sess.graph.as_graph_def(),
            converted_output_node_names)

        from tensorflow.tools.graph_transforms import TransformGraph
        transforms = FLAGS.transforms
        transformed_graph_def = TransformGraph(constant_graph, [],
                                               converted_output_node_names,
                                               transforms)
        constant_graph = transformed_graph_def