Interpreter close doesn't free memory.

[tenkeyless]

Memory doesn't free after interpreter.close();.

Even I put close code on button memory doesn't free at all.

It become grow up when I enter the screen.

This is full code.

import 'dart:io';

import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:tflite_flutter/tflite_flutter.dart';

class LoadTFLiteScreen extends StatefulWidget {
  const LoadTFLiteScreen({super.key});

  @override
  State<LoadTFLiteScreen> createState() => _LoadTFLiteScreenState();
}

class _LoadTFLiteScreenState extends State<LoadTFLiteScreen> {
  final _modelPath =
      'assets/mobilenet_v2_classification/mobilenet_v2_model_1_f16.tflite';
  final _labelsPath = 'assets/mobilenet_v2_classification/imagenet_label.txt';

  late final Interpreter interpreter;
  late final List<String> labels;

  Future<Interpreter> getModel(String modelPath, {int? cpuThreadNum}) async {
    final options = InterpreterOptions();

    if (cpuThreadNum != null) {
      options.threads = cpuThreadNum;
    }

    if (Platform.isAndroid) {
      options.addDelegate(XNNPackDelegate());
    }

    if (Platform.isIOS) {
      options.addDelegate(GpuDelegate());
    }

    return await Interpreter.fromAsset(modelPath, options: options);
  }

  Future<void> loadModel() async {
    interpreter = await getModel(_modelPath);
    setState(() {});
  }

  Future<void> loadLabels() async {
    final labelTxt = await rootBundle.loadString(_labelsPath);
    labels = labelTxt.split('\n');
    labels.removeLast();
  }

  @override
  void initState() {
    super.initState();

    loadModel();
    loadLabels();
  }

  @override
  void dispose() {
    interpreter.close();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: const Text("Load TFLite..")),
    );
  }
}

And this is memory status in XCode.

The point at which the memory graph goes up is when you leave and re-enter the screen. Each time re-enter the screen, the memory goes up continuously.

Until fix this problem, it would be good put your Intrepreter outside of class.

Here is sample code.

import 'dart:io';

import 'package:flutter/material.dart';
import 'package:flutter/services.dart';
import 'package:tflite_flutter/tflite_flutter.dart';

class LoadTFLiteScreen extends StatefulWidget {
  const LoadTFLiteScreen({super.key});

  @override
  State<LoadTFLiteScreen> createState() => _LoadTFLiteScreenState();
}

Interpreter? interpreter;

class _LoadTFLiteScreenState extends State<LoadTFLiteScreen> {
  final _modelPath =
      'assets/mobilenet_v2_classification/mobilenet_v2_model_1_f16.tflite';
  final _labelsPath = 'assets/mobilenet_v2_classification/imagenet_label.txt';

  // late final Interpreter interpreter;
  late final List<String> labels;

  Future<Interpreter> getModel(String modelPath, {int? cpuThreadNum}) async {
    final options = InterpreterOptions();

    if (cpuThreadNum != null) {
      options.threads = cpuThreadNum;
    }

    if (Platform.isAndroid) {
      options.addDelegate(XNNPackDelegate());
    }

    if (Platform.isIOS) {
      options.addDelegate(GpuDelegate());
    }

    return await Interpreter.fromAsset(modelPath, options: options);
  }

  Future<void> loadModel() async {
    interpreter ??= await getModel(_modelPath);
    setState(() {});
  }

  Future<void> loadLabels() async {
    final labelTxt = await rootBundle.loadString(_labelsPath);
    labels = labelTxt.split('\n');
    labels.removeLast();
  }

  @override
  void initState() {
    super.initState();

    loadModel();
    loadLabels();
  }

  @override
  void dispose() {
    interpreter?.close();
    super.dispose();
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(title: const Text("Load TFLite..")),
    );
  }
}

It takes up space when it first loads, but memory stays the constant size.

[PaulTR]

Hey @tenkeyless, can you post your entire app on GitHub so I can close it to test? I'm not seeing the memory issue on my end, so I want to verify it with your known case.

Thanks!

[PaulTR]

Actually I was able to replicate this with the image classification PR that's out there #109. It looks like it's an iOS only bug since I can't replicate it at all on Android and consistently sit around 160mb - 200mb:

but on iOS I see the number climb really quickly as I switch back and forth between tabs.

I'll reach out to someone to see if this is reproducible on pure iOS since my guess is it isn't an issue from this plugin.

[PaulTR]

Alright waiting for a reply from eng. Looks like it'd be in however we're working with iOS in TfLiteInterpreterDelete that we're accessing through c_api.h.

[PaulTR]

Alright so I have no experience with iOS :) Would it be possible for you to try running the interpreter with native iOS (https://www.tensorflow.org/lite/guide/ios)? I want to see if we can replicate the leak locally, or if the issue is around https://github.com/tensorflow/flutter-tflite/blob/64d9842ae7a7410fced4a9ac8dcb3ec6f1dfd2d6/lib/src/bindings/bindings.dart#L28

[tenkeyless]

I made sample app for this issue. (https://github.com/tenkeyless/realtime_classification)

This is a screen with memory status for this app.

And I will try tensorflow lite with native iOS soon.