Issue in Auto code generation

[kabilana2002]

Auto Code generation in drift is very slow on my device

We have more than 200 tables in a single .dart file(I have more than 200 tables so I can't able to create each file for every table individually, so I've segregated the 200 tables into four major parts-> 50 tables in each file). Drift takes 15-25 minutes to generate code. Modifying a table takes the same time as generating table code. Sometimes the device itself crashes because of this process.

Manually copying the modified table-generated code isn't a permanent solution. Any alternative solutions will be appreciated. Fixing the issue might help our developers to focus more on the code logic.

[simolus3]

Thanks for the report. I see you're already using modular builds 👍 Can you run an (incremental, after changing a table) build with performance tracking enabled and share those results?

[kabilana2002]

This is my report. Can you explain briefly what to do?

[simolus3]

Thanks! So it's not an analyzer problem but a slowdown that actually comes from drift. Do you see what the colorful stuff in the second line is?

This is probably something that I need to take a look at first hand. Do you have a way to share the database code with me or is that so entangled with the rest of the app that this would be difficult? I can also try to reproduce this by just randomly generating 200 dummy tables somewhere and running a build on that, but the more I know about this setup, the more I can do to analyze the problem.

[kabilana2002]

@DriftDatabase(
  tables: [
    ...configTables,
    ...masterTables,
    ...otherTables,
    ...reportTables,
    ...tagTables,
  ],
  daos: [
    ConfigDao,
    MasterDao,
    OthersDao,
    ReportsDao,
    TagDao,
    MdsrDao,
  ],
  // include: {'tables.drift'},
)
class AppDatabase extends $AppDatabase {
  AppDatabase._() : super(_openConnection());
  static final AppDatabase instance = AppDatabase._();
  factory AppDatabase() => instance;

  @override
  int get schemaVersion => 1  ;
}

// the LazyDatabase util lets us find the right location for the file async.
LazyDatabase _openConnection() => LazyDatabase(() async {
      // Also work around limitations on old Android versions
      if (AppPlatform.isAndroid) {
        await applyWorkaroundToOpenSqlite3OnOldAndroidVersions();
      }

      // Make sqlite3 pick a more suitable location for temporary files - the
      // one from the system may be inaccessible due to sandboxing.
      final cachebase = (await getTemporaryDirectory()).path;
      // We can't access /tmp on Android, which sqlite3 would try by default.
      // Explicitly tell it about the correct temporary directory.
      sqlite3.tempDirectory = cachebase;

      final file = await _getDatabaseFile();
      debugPrint('_openConnection: ${file.path}');

      // return NativeDatabase.createInBackground(file, logStatements: true);
      return NativeDatabase.createInBackground(file);
    });

// put the database file, called db.sqlite here, into the documents folder
// for your app.
Future<File> _getDatabaseFile() async {
  final dbFolder = await getApplicationDocumentsDirectory();
  return File(p.join(dbFolder.path, 'db.sqlite'));
}

Future<void> deleteDatabaseFile() async {
  final file = await _getDatabaseFile();
  if (await file.exists()) {
    await file.delete();
  }
}


////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
   ...configTables,
    ...masterTables,
    ...otherTables,
    ...reportTables,
    ...tagTables,

in the above 5 .dart files on average I have 50 tables in each file, which probably contains 5-12 columns.

while autogenerating all, takes around 1 hour(In Mac Air M3). so let's say I have a new table to add in configTables, I will auto-generate it for this single file alone( it takes 11-14 minutes), I ain't changed the existing tables in it, I just added a new table to it, so It doesn't need to auto-generate for the entire file, rather than there must be a way to auto-generate the single table in the particular file.

If you don't have an idea about what I'm trying to say, we could end up in a small quick call.

[petrnymsa]

Observing same behavior.

On quite large project after upgrading Flutter SDK and Drift we are getting minutes to Drift finish. (Macbook PRO M4 or quite powerful Windows machine).

I can happily any investigation info if needed

[petrnymsa]

In our project long build_runner time is definitely related to Drift.

When I disable drift builder in build.yaml, project regenerates under 30s. With Drift enabled it is around 10 minutes

[simolus3]

Thanks for the report! Can you try running the build with performance tracking enabled and perhaps share those graphs?

A quick workaround may be to restrict the sources on which drift runs, you can do that with a build config (build.yaml):

targets:
  $default:
    builders:
      drift_dev:
        generate_for:
          include:
            - lib/src/path/to/your/database.dart

But of course, drift shouldn't slow your build down so this is still something worth taking a look at.

[petrnymsa]

We already have restricted build in build.yaml.

I tried to run performance tracking, but it could finish before the out-of-memory exception. Increasing Heap space for Dart VM did not help.

However, when I switched to modular drift generation, we got build_runner run around 1-2 minutes, which is kinda ok.

Without Drif,t however,r it still runs around 20s - we are using json_serilalizable, freezed and auto_mappr.

[simolus3]

Thanks for the follow-up! Unfortunately it will be kind of hard to track this down without a way to reproduce this. Can you share parts of the database structure with me (e.g. how many tables / columns do you have, are these split across different files, how many accessors are you generating, ...)?