Better support for hiding generated files

[Wdestroier]

The documentation has this section:

However, the user must always be able to choose if they want to generate the files next to source-code or in the .dart_tool directory.

[davidmorgan]

Do you mean output like the .g.dart files output by json_serializable?

Currently these have to go next to the source code: if they are written somewhere else, other tools won't find them and they won't work.

I do want to support writing them somewhere: it will be a nice usability improvement. But it's not possible today.

Or did you mean for some other type of output / generator?

[rrousselGit]

IMO the main issue is parts needing a path.
It's one thing to put generated code in .dart_tool.

But then your part will end-up as:

part '../../../../.dart_tool/build/generated/lib/src/directory/file.g.dart';

[Wdestroier]

Do you mean output like the .g.dart files output by json_serializable?

Yes, to avoid mixing the source-code with generated code.

IMO the main issue is parts needing a path.

True. In other languages (Java, C#, Python) the part file doesn't need to be written by hand.

[davidmorgan]

Right, we definitely need a better solution here.

I'll rename the issue "Better support for hiding generated files", please say if that does not properly capture the issue :)

I think the correct answer here is one or both of:

• multi-root packages: .dart_tool/build/generated can be a second root
• source archives, so all the generated source can live in a single file

[Wdestroier]

I'll rename the issue "Better support for hiding generated files", please say if that does not properly capture the issue :)

The title captures the issue very well, thank you!

I think the correct answer here is one or both of: multi-root packages, source archives.

Would be nice if each package could have a .dart_tool/build/generated directory (or .dart_tool/build-generated, easier to navigate in the IDE), unless the user wants to keep the generated code of some generators next to source-code.

I thought each package could perhaps configure each builder in pubspec.yaml to generate next to source-code or not (in its own build/generated directory).

Currently these have to go next to the source code: if they are written somewhere else, other tools won't find them and they won't work.

Could mirroring the folder structure work? For example: the [1] file, generated for [2], could live in [3].

[1] api\lib\persistence\app_database.g.dart [2] api\lib\persistence\app_database.dart [3] api\.dart_tool\build-generated\lib\persistence\app_database.g.dart

That way, part 'app_database.g.dart' and part of 'app_database.dart' could – hopefully – magically work and migrating these tools would be relatively simple, if necessary.

[rrousselGit]

Fwiw I raised an issue not too long ago about improving part: https://github.com/dart-lang/language/issues/4358

IMO that's the bottleneck here. If we improve part to loosen the path syntax, build_runner could more easily generate files in a hidden folder without degrading usabillity.

[davidmorgan]

Fun fact: the Dart SDK already supports packages with multiple root folders: it's part of the bazel support, needed because bazel does output its generated files to a different folder.

That means everything already works end to end, e.g. analyzer for IDE integration, all the compilers, plugins. It's just only used for bazel <--> in google3.

I think what's needed is to also wire it up for pub packages, pub workspaces, etc. Not a small task, but not too scary :)

[davidmorgan]

One note is that when enhanced parts arrive, parts can have imports.

That means they can no longer be merged so easily: it's simplest to have one part per builder.

But that starts to be quite a bad experience if you use more than one builder.

So we'll definitely want a way to hide all the (enhanced) parts.

[rrousselGit]

Parts can also define parts, right?

If so, I'm not worried. You could have .g.dart define parts that point to .dart_tool

[davidmorgan]

Parts can also define parts, right?

Yes, enhanced parts can have imports and parts.

If so, I'm not worried. You could have .g.dart define parts that point to .dart_tool

Does that actually work today? I'd be surprised if the whole tool chain copes well with relative imports to .dart_tool.

Anyway, I hope we can get to a better (or at least, officially blessed and supported) solution before enhanced parts + augmentations :)

[hpoul]

Not sure if this is really a good idea, didn't the flutter localization tool just go into the other direction, from hidden, synthetic package to generating code into the source path? IMHO it's nice to have all generated code visible.. it could be an IDE feature to group associated files or hide them? Imo it's always good to have them in VCS and distributed with libraries.. 🤔

[lucavenir]

Are we ready to hide generated code, tho?

Imho, no. Other fixes must be merged first.

For example, would I be able to understand why/how the generated code is erroring at analysis time, whereas the annotated code looks correct?

To date, this can quite happen.

Often times, I've seen folks hide *.g.dart and *.freezed.dart files in analysis_options.dart. That's bad, because once the builder generates erroring code, you won't notice until you spin up a test or run an application.

We could solve this in a similar way macros did: let the error highlight in the annotated code, and then one could open up the in-memory references and read the code.

class A with _$A {}
          // ^^^ errors, because reasons, even tho using _$A doesn't error on its own

There's also these to keep in consideration, since one should be able to "go-to-definition" of the annotated code effortlessly, but because of the above we should also be able to "go-to-generated-implementation" as an escape hatch.

[rrousselGit]

Are we ready to hide generated code, tho?

It's not about fully hiding it. It's just about not having it alongside the sources. You'd still see it in your IDE

[lrhn]

Just to pedant:

part '../../../../.dart_tool/build/generated/lib/src/directory/file.g.dart';

That will not work if your library is inside lib/ and has a package: URI, because then the relative URI reference is resolved relative to the package: URI, which cannot refer to something which doesn't have a package: URI too, and cannot use .. to go above the root.

If files in .dart_tool/build/generated/lib/... can also be referenced by package: URIs (if the package has multiple package URI roots in some way), then the relative URI should probably be

part '../../../src/directory/file.g.dart';

If we want multi-rooted package: namespaces, we need to update the package_config.json format. Probably something like adding an extra optional list of secondary roots.

    {
      "name": "myHelperPackage",
      "rootUri": "../../myHelperPackage/",
      "packageUri": "lib/",
      "altPackageUris: ["../generated/lib/"],
      "languageVersion": "3.9"
    },

I'd increment the format version to 3 at that point. A package_config.json v2 capable tool would not be able to correctly resolve all package: URIs of this format.

Or just make "packageUri" be allowed to be a string or a list of strings. Or only a list of strings (with a format version increment, it doesn't have to be backwards compatible, and it might be better that it's not. Just to "fail fast".)

A problem with that is that you can't resolve a package URI to one path. It's multiple possible paths, and then you need to try to read the file before you can know if a path works. (Or use exists if your file source supports that, which HTTP doesn't. The package configuration tries to be agnostic about the URIs it works with.)

Maybe it would be better to have "mounts", with certain sub-directories of package:foo/ being mapped to different roots, and each path having exactly one resolution. The current packageUri is the / mount.

[galacticgibbon]

In VSCode/Cursor, I use this setting

    "files.exclude": {
        "**/*.freezed.dart": true,
        "**/*.g.dart": true
    },

Works great.