Dartpad support

[dcharkes]

The Flutter API docs use dartdoc tools to render DartPad on the website:

• https://github.com/flutter/flutter/blob/fe0bacbacfab6b67473c0e4c174a047f7da21682/dartdoc_options.yaml#L13-L15
• https://github.com/flutter/flutter/blob/20dd20932dc87d14be4bc04f12a1a1f755b73e8a/packages/flutter/lib/src/material/app_bar.dart#L127-L132
• https://github.com/flutter/flutter/tree/master/dev/snippets

However, dartdoc tools are not rendered on pub.dev (and in the IDE). This makes it impossible to dartdoc tools for packages.

We should consider adding first-class support for dartpad in dartdoc.

(I'm pretty sure we've discussed this before, but I can't find a tracking issue.)

[sigurdm]

The biggest obstacle I think is that dartpad (last I checked) still works with a fixed set of packages - so it would only work for documenting those packages.

[srawlins]

https://github.com/dart-lang/dartdoc/issues/3503 "Implement Flutter's @tool snippet natively" can be considered a parent issue for this issue.

You are right, @sigurdm that supporting dartpad generally does support only a limited number of customers, but I still think it's an important feature; I have long wanted to implement it but it never quite made it above the prioritization bar. The main benefits I can think of are:

1. Simplifying generating docs for the Flutter SDK and the Dart SDK, by avoiding the complicated @tool logic.
2. Improving the performance of generating docs for Flutter in particular, which makes much use of @tool snippet (or @tool sample? I forget). Not shelling out to a different dart process hundreds of times.
3. Many flutter favorite packages can make use at pub.dev, like maybe provider, riverpod, some top plugins, and some core packages like path, collection, etc. Whoever is available at dartpad.
4. Any package developer who publishes their docs on their own server can conceivably make use of it, if they also host their own dartpad instance, and we make the directive configurable. This is maybe a long shot, but there could be great ROI if we do have some important package authors doing this, or enterprise package authors who host their own pub service and dartpad.

[sigurdm]

Sure - don't get me wrong - I think embedding dartpad everywhere would be super-cool and we absolutely should do it.

And I don't think even think it is unlikely we can make dartpad resolve most packages.

[srawlins]

I believe that Flutter's {@tool dartpad} feature amounts to the following:

1. Given some dartdoc like this:

   /// {@tool dartpad}
   /// This sample shows produces an enabled and disabled [CupertinoButton] and
   /// [CupertinoButton.filled].
   ///
   /// ** See code in examples/api/lib/cupertino/button/cupertino_button.0.dart **
   /// {@end-tool}
   

   (found in packages/flutter/lib/src/cupertino/button.dart)

2. The tool extracts the summary text.

3. The tool extracts the text examples/api/lib/cupertino/button/cupertino_button.0.dart.

4. The tool reads that file and writes it (maybe with some transformations) to a snippets/ location in doc/ (where dartdoc is also writing generated HTML). I believe this new location is what DartPad sources.

5. The tool reads the dartpad-sample.html template.
6. The tool interpolates values into several mustache-looking variables: 1. {{id}} - This is derived from examples/api/lib/cupertino/button/cupertino_button.0.dart, into something like cupertino.CupertinoButton.1, though I don't know the transformation. 2. {{serial}} - I don't know how this is derived, but it appears to be a counter unique on each page, starting at 1, used to make unique HTML IDs. 3. {{channel}} - This refers to the channel on DartPad (stable, beta, or main), which aligns with the Flutter channel for which docs are being generated.
7. The tool then spits out the following HTML:
   1. A simple link to serve as an anchor.
   2. A div with a unique ID wrapping a few other things:
      1. The summary text.
      2. A copy-to-clipboard button.
      3. The text, "To create a local project with this code sample, run:"
      4. A flutter tool command with the sample ID.
      5. An iframe that loads up dartpad.dev, with the given sample ID and channel.

This should not be difficult to extract into a general {@dartpad} doc directive. And I think with some dartdoc_options.yaml options, the Flutter framework could still get the secret sauce of specifying a DartPad channel, and writing out the "To create a local project..." text.

Proposed design

A new doc directive, {@dartpad} can be recognized by both the analyzer and dartdoc.

• The file path for where the sample lives is found as a positional argument for {@dartpad}, like {@dartpad examples/api/lib/cupertino/button/cupertino_button.0.dart}. There is already support in analyzer for doc directives, and for a positional argument like this. The path given is normalized for the file system (i.e. separators), and is relative to the containing Pub package.
  • Note that the Cupertino example is not currently found in the containing Pub package, but is instead found in the containing Pub workspace. (It's probably just relative to some arbitrary place that the Flutter DartPad tool chooses.) When Cupertino widgets move to their own Pub package, the samples should move with them, so that they can be picked up by the pub.dev tooling.
• The analyzer can validate that the path exists, and provide click-through.
• Dartdoc has more work to do. 😁
• Dartdoc would also recognize the new directive and read its arguments, but not with its own parsing. It should be able to just get the info from the analyzer ASTs. (I believe it does this today.)
• Dartdoc would also read the file referenced in the positional argument, and copy it (maybe with transformations?) to doc/snippets/.
• Dartdoc would also replace the {@dartdoc} directive with the anchor, summary text, copy-to-clipboard button, and the DartPad iframe.
• Optionally (maybe based on a dartdoc_options.yaml option), Dartdoc would also add the "To create a local project..." text.
• Optionally (maybe based on a dartdoc_options.yaml option), Dartdoc would also set the DartPad channel URL parameter.
• There is maybe some trickiness in the copy-to-clipboard button. But this all works today for api.flutter.dev, and we could probably crib that logic.
• I think DartPad could always fetch the sample code from the given "sample_id" path, relative to... something. Maybe with another parameter, "package_name", it would fetch from https://pub.dev/documentation/{{package}}/snippets/{{sample_id}}.

I'd love any feedback, @piinks @szakarias @sigurdm @johnpryan @gspencergoog @bwilkerson

[gspencergoog]

The tool interpolates values into several mustache-looking variables: 1. {{id}} - This is derived from examples/api/lib/cupertino/button/cupertino_button.0.dart, into something like cupertino.CupertinoButton.1, though I don't know the transformation. 2. {{serial}} - I don't know how this is derived, but it appears to be a counter unique on each page, starting at 1, used to make unique HTML IDs.

The id is the result of parsing the Dart code and finding the symbol that the dartdoc is attached to, and the serial is the index of the example in that dartdoc comment.

The code for this is in the snippets tool: https://github.com/flutter/flutter/blob/main/dev/snippets/bin/snippets.dart#L198

The dartdoc code generates a number of environment variables that the snippet tool looks for and uses to construct the names from. They are designed to be unique and understandable. The code where it does this is in the documentation_comment handler: https://github.com/dart-lang/dartdoc/blob/main/lib/src/model/documentation_comment.dart#L277

Also, these generated source files aren't just used for the dartpad support: the flutter create command has a --sample argument that loads the sample code into the main of a local project so that you can play around with it.

For example: flutter create --sample=widgets.SingleChildScrollView.1 mysample

[sigurdm]

I'd love any feedback, @Piinks @szakarias @sigurdm @johnpryan @gspencergoog @bwilkerson

Generally I think it looks solid.

Should we make it possible to also have inline dartpad snippets? Not sure exactly how it would look - but it seems more ergonomic for smaller samples...

Optionally (maybe based on a dartdoc_options.yaml option), Dartdoc would also set the DartPad channel URL parameter.

I think we should leave that out of the first edition. While I can see reasons you want another channel, I think it is very unwieldy in practice to configure that from inside the package (channels develop independently from the (immutable) package). Maybe we can infer the channel from pana somehow, when we choose the sdk to analyze with.

[srawlins]

Optionally (maybe based on a dartdoc_options.yaml option), Dartdoc would also set the DartPad channel URL parameter.

I think we should leave that out of the first edition.

I'm only trying to provide an equivalent feature for the Flutter framework. For the Flutter framework, it is important that the DartPad instances shown in the beta docs use the beta channel and the stable docs use the stable channel. It leads to user confusion if the DartPads don't reflect that.

Maybe we can provide another way to set the channel URL parameter.

[srawlins]

(Although, I don't think this will affect the analyzer side of the implementation, so I don't have any skin in this decision 😅 ; I'll leave it for @piinks to raise any concerns.)

[gspencergoog]

Should we make it possible to also have inline dartpad snippets? Not sure exactly how it would look - but it seems more ergonomic for smaller samples...

Yes, this seems like a good idea when you first think about it, but...

On the Flutter team, we started with this idea too, and it rapidly became clear that:

1. People don't like editing code in Dartpad comments.
2. Automated testing of the samples to make sure they compile and run correctly gets a lot harder when you need to extract them for the test.
3. It's harder to do things like analysis of the sample code.

Once we switched to external files for the samples, everything got easier, and we haven't looked back.

[dcharkes]

Once we switched to external files for the samples, everything got easier, and we haven't looked back.

+1

In my projects I've also switched to external files. Not only do you want those snippets to be analyzed, you also might want to hide part of the snippet (using tags in the snippet file to mark start and end), and run the snippets to test them.

I'm using a "updater script" to copy the external files into the code comments. Hopefully Dart-pad support will remove the need for such script.