Convert Returnn model to TFLite for on-device deployment

[manish-kumar-garg]

trafficstars

I want to convert the returnn full attention setup model to TFLite for on-device deployment. Should I reimplement the model in TF2.0, load weights and convert to TFLite? Or any other way is there? What steps should I follow?

[albertz]

@curufinwe might have some experience on this. Do you really need TFLite? Normal TF is not an option? Why? Things like quantization, sparsification, etc, all should work just as well with normal TF.

[curufinwe]

We (at i6) have not yet tried to convert our models to TFLite. My suggestion would be to freeze the graph and then use the tools that come with TFLite to do the conversion.