UI freezes while using Web worker

[OlegShNayax]

Hi guys,

I'm using Drift with Flutter Web. I have a really big list of objects that I need to insert in database(IndexedDB) without UI freeze. I configured my project according to flutter_web_worker_example (add support of web worker), but when I inserting list in database via web worker, UI freezes. (Please see attached video file)

I created a example with described issue at drfit_webworker_poc

Database and example of insert action

@DriftDatabase(tables: [MachineTable])
class AppDatabase extends _$AppDatabase {
  AppDatabase(QueryExecutor executor) : super(executor);
  AppDatabase.connect(DatabaseConnection c) : super.connect(c);

  @override
  int get schemaVersion => 1;

  Future<void> insertMachines(List<Machine> machines) {
    return batch((batch) {
      batch.insertAll(
          machineTable,
          machines,
          mode: InsertMode.insertOrReplace);
    });
  }
}

Configuration of web workers in database

class PlatformInterface {
  static QueryExecutor createDatabaseConnection(String databaseName) {
    return LazyDatabase(() async {
      return _connectToWorker(databaseName).executor;
    });
  }

  static DatabaseConnection _connectToWorker(String databaseName) {
    final worker = SharedWorker(kReleaseMode ? 'worker.dart.min.js' : 'worker.dart.js', databaseName);
    return remote(worker.port!.channel());
  }
}

Example of performance test that executes in application

driftDbPerformanceTest() async {
  print("Start drift db performance test");
  final database = AppDatabase(Platform.createDatabaseConnection('sample'));

  await Future.delayed(const Duration(seconds: 3));
  final stopwatch = Stopwatch()..start();

  final json = jsonDecode(await rootBundle.loadString('jsons/basic_old_response.json'));
  final machines = (json as List<dynamic>).map((e) => Machine.fromJson(e)).toList();
  print("finish machines serialization json(length: ${machines.length}) in milliseconds ${stopwatch.elapsed.inMilliseconds}");

  stopwatch.reset();
  await database.insertMachines(machines);
  print("insert machines using drift(length: ${machines.length}) in milliseconds ${stopwatch.elapsed.inMilliseconds}");
}

https://user-images.githubusercontent.com/103174674/197016244-62eeca7c-c684-47a2-8827-fd6d1838809c.mp4

[simolus3]

Thanks for the detailed issue. I didn't check this yet, but I suppose this is the cost of creating and serializing all the SQL statements to send to the worker (this still happens on the main thread).

I have some pending investigations I need to do on performance (https://github.com/simolus3/drift/discussions/2064 is also in my backlog). If it's possible, you could probably reduce the lag if it's possible to run the entire insert operation on a background worker (so that the worker loads the data and inserts it)?

[OlegShNayax]

@simolus3 Thanks for replying back!

Can you please provide an example of running the insert operation on a background worker ?

[simolus3]

It's not that easy unfortunately.

In general, you want something like Flutter's compute but with web workers. I'm not sure if a package exists to make this easier already. Otherwise, you're already creating the database here. So the same file could create an instance of your AppDatabase class and insert some data into it.

If you want to load data from an URL, you'd have to find a way to inform the worker about that URL. You could use onConnect and messagePorts to establish a communication channel between the worker and the main tab. This is also what drift is doing internally to relay database operations through the worker. You could use this communication channel to send commands like "download this URL, then insert the resulting rows into the database" to the worker. The worker would then execute that chunk of work.

[OlegShNayax]

@simolus3 Thanks! I'll try to implement background worker in example

[mahdinazmi]

@simolus3 Thanks! I'll try to implement background worker in example

I have same problem. If you can implement it, let me know, thanks.