Custom row classes

[simolus3]

This tracks enhancements to the custom row classes feature in moor 4.3. Custom row classes let users specify an existing class that moor will use as a data class:

@UseRowClass(MyRowClass)
class MyTable extends Table {
  IntColumn get id => integer().autoIncrement()();
}

class MyRowClass {
  final int id;

  MyRowClass(this.id);
}

Goals

• [x] Basic support for custom row classes
• [x] Proper type checking
• [x] Support custom row classes for tables declared in moor files
• [x] Support named constructors
• [x] Aid in generating toColumns if desired (not urgent, we can delegate that method to the generated companion)
• [ ] Support something similar for generated queries?

[rubenferreira97]

I have been using this awesome enhancement since the new moor version and I would like to suggest/discuss if the following features are possible to implement. If this is not the right place to take this discussion, please say so, I will delete this comment.

1. Allow @UseRowClass(MyRowClass) with abstract classes:

@UseRowClass(Category)
abstract class CategoriesTable extends Table {
  IntColumn get id => integer()();
  IntColumn get parentId => integer().nullable()();
  TextColumn get title => text()();

  @override
  String get tableName;

  @override
  List<String> get customConstraints =>
      ['FOREIGN KEY(parentId) REFERENCES ' + tableName + '(id)'];

  @override
  Set<Column> get primaryKey => {id};
}

//Don't need annotation @UseRowClass(Category)
class OccurrenceCategoriesTable extends CategoriesTable {
  @override
  String get tableName => 'occurrence_categories';
}

//Don't need annotation @UseRowClass(Category)
class SocietyCategoriesTable extends CategoriesTable {
  @override
  String get tableName => 'societies_categories';
}

Child classes could inherit @UseRowClass(Category). Overwrite the annotation if needed. The current implementation generates dataclasses for both classes (OccurrenceCategories and SocietyCategories).

2. Generate custom row classes companions The previous code (with @UseRowClass(Category) on both classes) generates OccurrenceCategoriesTableCompanion extends UpdateCompanion<Category> and SocietyCategoriesTableCompanion extends UpdateCompanion<Category>. In my opinion these generated classes are code bloat and break abstraction, since they could be easily replaced with a single class CategoryCompanion extends UpdateCompanion<Category>. Maybe add a generateCompanion parameter to @UseRowClass(Category).

[simolus3]

I think these are some interesting suggestions, thanks!

Allow @UseRowClass(MyRowClass) with abstract classes

I think annotations are kind of weird to use with inheritance. For instance, if an @UseRowClass annotation also applies to subclasses, how would they opt-out if they want to use a moor-generated row class instead? Since annotations aren't inherited in Dart by default, I also worry that this behavior may be non-intuitive to some.

However, note that you can store annotations in a constant field and then apply that field as an annotation. So if you're worried about code duplication in annotations, maybe that helps.

const withCategory = @UseRowClass(Category);

@withCategory
class OccurrenceCategoriesTable extends CategoriesTable {
@withCategory
class SocietyCategoriesTable extends CategoriesTable {

Generate custom row classes companions

I agree that there should be a way to disable an automatic generation of companions, and I think the UseRowClass annotation could be a good fit for that. I think there needs to be a separate mechanism to generate companions for abstract table classes then, since otherwise you'd end up with no companion at all. I'll think about it some more!

[rubenferreira97]

Since annotations aren't inherited in Dart by default, I also worry that this behavior may be non-intuitive to some.

Seems fare enough, I wasn't aware that annotations weren't inherited in Dart. Java for example have the @Inherited annotation to make this possible. Maybe this should a feature request in dart SDK (if is not one already). Even if this not applies to moor I think it would be great to have a way to make inheritance possible in annotations. It seems overkill and unreasonable to add a custom annotation in moor just to solve this. This idea was just a tiny detail, your solution seems a nice workaround, even if it don't solve the same purpose of inheritance.

I think the UseRowClass annotation could be a good fit for that

Yes, I really think the default should generate only the custom row class companion, but wasn't sure if this was moor design, since it may restrict flexibility (which IMO is not a problem since when you use UseRowClass you are admitting a specific dataclass). I would generate a single MyRowClassCompanion extends UpdateCompanion<MyRowClass> companion to fit moor design, keeping in mind that multiple UseRowClass with the same class can be written, which may result in duplicate classes on code generation.

I think there needs to be a separate mechanism to generate companions for abstract table classes then, since otherwise you'd end up with no companion at all.

If you follow the previous approach I think you don't need to care if they are abstract classes or not. By catching all UseRowClass classes like in a set isn't possible to create all companions?

Really appreciate the fast input 😃. Just trying to make moor even more awesome, I hope you take these suggestions as a open discussion and not nitpicking.

[mateuy-dev]

From the first post I understand that a UseRowClass can be defined when usind a .moor file. What needs to be added at the create table sentence to specify the class?

[simolus3]

@mateuyabar You have to import the Dart file and then use WITH at the end of the table declaration - see this section in the documentation for a bit more on this.

[simolus3]

@rubenferreira97 I think adding an option to generate companions could solve most of the problems (just turn it off for the actual table classes and enable it for the abstract table). However, moor currently generates classes based on the tables mentioned in @UseMoor. Since you probably don't include the common abstract table definition there, moor would ignore the table and not give you the shared companion. There have been some other requests to change this generation model to allow a more modular generation, that might help here as well. Or maybe we could have an option to share companions between specific tables and give them a custom name, which essentially boils down to the same thing here. But note that using multiple tables with the same schema is kind of a niche usage, so we can't change the API too much just for this :D

[rubenferreira97]

Since you probably don't include the common abstract table definition there, moor would ignore the table and not give you the shared companion.

I don't include the common abstract table here but I think that's not a problem, since I would include the concrete classes, for example:

@withCategory
class OccurrenceCategoriesTable extends CategoriesTable {...}
@withCategory
class SocietyCategoriesTable extends CategoriesTable {...}

@UserMoor(Tables: [OccurrenceCategoriesTable, SocietyCategoriesTable]
class Database extends MoorDb {...}

Moor would need to generate only one CategoryCompanion since both define @UseRowClass(Category). Moor shouldn't generate more than one companion for the same class as well as generate companions for @UseRowClass (moor currently generates classes even if detonated with @UseRowClass).

[westito]

Thanks for this feature, that's i had waited for. I always used separated model classes with a lot of boilerplate "toDb", "fromDb" code.

[westito]

mutable_classes builder option and immutability is not applicable here. It's up to you to use final modifier or not in models.

[westito]

Feature request: Generated row class mapper. When I use Insertable interface I need declare all columns in toColumns(). That's a boilerplate what I want to avoid with @UseRowClass. Can you make a class mapper like: CustomerCompanion.mapRowClass(customer). In this case not need to use Insertable interface or I can simply write return CustomerComanion.mapRowClass(this); in toColumns() override.

[simolus3]

Yeah I would recommend creating a companion in toColumns(), then you can call toColumns() on that companion. It's a bit of boilerplate, but you don't have to manually apply type converters or anything.

It's hard to generate the mapRowClass factory you're suggesting because row classes are designed to be as general as possible. As long as you provide some constructor with arguments compatible to the table, moor will use that. You don't even have to add every column to that class, or you can use a custom wrapper class for your values. This makes it hard (it not impossible) for moor to generate an automatic mapping from your row class to a companion.

[westito]

Code generation not my cup of tea, but I see this in generated code:

@override
Product map(Map<String, dynamic> data, {String? tablePrefix}) {
    final effectivePrefix = tablePrefix != null ? '$tablePrefix.' : '';
    return Product(
      id: const IntType().mapFromDatabaseResponse(data['${effectivePrefix}id']),
      code: const StringType()
          .mapFromDatabaseResponse(data['${effectivePrefix}code'])!,
    );
}

Can't be generate like:

@override
Map<String, Expression> toRowClassColumns(Product rowClass, bool nullToAbsent) {
    final map = <String, Expression>{};
    if (!nullToAbsent && rowClass.id != null) {
      map['id'] = Variable<int?>(rowClass.id);
    }
    map['code'] = Variable<String>(rowClass.code);
    return map;
}

?

I understand that tables and classes can have different fields, but this mapper could use intersection of table and class columns. Or maybe I can annotate row class fields that included in this mapper.

[westito]

I made a PR for this mapper feature https://github.com/simolus3/moor/pull/1239

[westito]

Next level: Like hibernate, merge table definition and row classes with annotations. I made a very basic proof-of-concept code: https://github.com/westito/moor/pull/1

[simolus3]

Nice! I should mention that in general I don't want moor to turn into a full ORM like hibernate. Design-wise I see moor as a (arguably fairly heavy) convenience layer for sqlite3 in Dart that still embraces a relational style instead of hiding it. If you want to, you're welcome to build an ORM on top of moor of course.

[westito]

moORM initial alpha version :))) https://github.com/westito/moORM/tree/develop

Sample implementation: https://github.com/westito/moORM/blob/develop/moor/example/example.dart

Backward compatible with moor

[kuhnroyal]

I am looking to use a generic row class from a moor file but it seems that WITH Foo<Bar> is not possible, expects a semicolon after Foo. @simolus3 Do you think that is possible to implement?

[simolus3]

I think this might be challenging to do in general - we'd essentially have to parse every valid Dart type, including things like Foo<Bar Function(String? x, {required bool y})>. The analyzer does not expose a public API that could help us parse a subset of Dart.

With general type aliases, I wonder if you could declare typedef FooBar = Foo<Bar> somewhere and then use that as a row class in moor? The generator might still trip over that, but this is easier to support and IMO is reasonably ergonomic to use. I'll try it out later and fix the generator if it doesn't like that.

[kuhnroyal]

Tried it and getting Existing Dart class FooBar was not found, are you missing an import?. And the import is there :)

[simolus3]

Generic classes referenced through a typedef should now work on develop .

[kuhnroyal]

Great, I will test that tomorrow, thanks for the quick action!

[kuhnroyal]

This gets me a lot further, generation succeeds, table classes are correctly generated. Something still needs to be workout for the companion classes.

class DataEntity<T> {
  final T id;
  final bool deleted;

  DataEntity({
    required this.id,
    required this.deleted,
  });
}

typedef FooDataEntity = DataEntity<FooEnum>;
typedef BarDataEntity = DataEntity<BarEnum>;

This generates 2x DataEntityCompanion, one for each enum. And I am actually not sure what the correct output would be here. If I think about it, I would want one generic DataEntityCompanion matching the row class.

[simolus3]

I assume this is because of use_data_class_name_for_companions being enabled, right? So at least we don't run into this problem by default :D

Pushing generics into companions is possible, but it will be quite some work in the generator (we don't currently support sharing companions between tables) and hard to get right (we'd have to copy generic bounds and other things that I might be forgetting). Given that develop hasn't been released yet, I think the best solution may be to generate one FooDataEntityCompanion and one BarDataEntityCompanion, so we'd just use the typedef as a base name in this case.

[kuhnroyal]

True, I have use_data_class_name_for_companions enabled. Separate companion names based on the typedef would be a good start. Having generic companions would be nice to have at some point ( I have 30+ tables :D).

[rubenferreira97]

I know it might be harder to achieve, but I agree with @kuhnroyal, a generic DataEntityCompanion would be preferable. Two classes (or more) refering to the same properties and behaviour adds code bloat and it does not make sense in a language that allows inheritance. High order classes should define a behavior, in this case DataEntityCompanion. If users want different classes I think they should declare it (FooDataEntity and BarDataEntity).

IMO this still applies:

Generate custom row classes companions The previous code (with @UseRowClass(Category) on both classes) generates OccurrenceCategoriesTableCompanion extends UpdateCompanion<Category> and SocietyCategoriesTableCompanion extends UpdateCompanion<Category>. In my opinion these generated classes are code bloat and break abstraction, since they could be easily replaced with a single class CategoryCompanion extends UpdateCompanion<Category>. Maybe add a generateCompanion parameter to @UseRowClass(Category).

Moor shouldn't generate more than one companion for the same class as well as generate companions for @UseRowClass (moor currently generates classes even if detonated with @UseRowClass).

Moor should have a way to implement generic data classes.

[westito]

I think "Aid in generating toColumns if desired" in the list is done :) What do you mean by "generated queries"?

[simolus3]

@westito Thanks for your work on generating a mapper to insertables! :tada:

I wonder if we should support a similar feature for generated queries (declared in a drift file), so e.g. you could do

class MyResult {
  final String foo, bar;
  MyResult(this.foo, this.bar);
}

query WITH MyResult: SELECT foo, bar FROM myTable;

Supporting nested result sets here could be a bit challenging though.

[westito]

I think the virtual views I made can cover this.

[JCKodel]

This isn't supported on views? =(

CREATE VIEW BaseItem_EssentialOils (
  SELECT id, name AS title, botanicalName as subtitle
  FROM EssentialOils;
) WITH BaseItem;

[WARNING] drift_dev on lib/data/database.dart:
Could not parse file: line 22, column 1: Error: Expected the file to end here.
   ╷
22 │ ) WITH BaseItem;
   │ ^
   ╵
[SEVERE] drift_dev on lib/data/database.dart:

type 'Null' is not a subtype of type 'ParsedDriftFile' in type cast

[tRuNKator]

This isn't supported on views? =(

CREATE VIEW BaseItem_EssentialOils (
  SELECT id, name AS title, botanicalName as subtitle
  FROM EssentialOils;
) WITH BaseItem;

[WARNING] drift_dev on lib/data/database.dart:
Could not parse file: line 22, column 1: Error: Expected the file to end here.
   ╷
22 │ ) WITH BaseItem;
   │ ^
   ╵
[SEVERE] drift_dev on lib/data/database.dart:

type 'Null' is not a subtype of type 'ParsedDriftFile' in type cast

Try remove semicolon in FROM EssentialOils