[testdata] This package does not have ... in the `dependencies` or `dev_dependencies` section of `pubspec.yaml`

[jonasfj]

Can't publish if test/testdata/foo.dart imports package:example/.

This fails due to: validator/strict_dependencies.dart

We need a way to include arbitrary test data, perhaps we could:

• Have a mechanism to opt-out of certain publishing checks.
• Perhaps respect exclude patterns from analysis_options.yaml).
• A convention that testdata/ folders are never analyzed.

  • Needs to be landed package layout conventions

  • Needs coordination and buy-in from wider Dart team.

  • Needs support from analyzer, we'd need **/testdata/** folders to be excluded by default in analysis_options.yaml (possibly with an option to include such a folder again).

  • It aligns with golang:

    The go tool will ignore a directory named "testdata", making it available to hold ancillary data needed by the tests.

    See: https://pkg.go.dev/cmd/go#hdr-Test_packages

Example attempt to publish dartdoc_test.

Code that caused this is available here: https://github.com/google/dart-neats/tree/dartdoc_test-v0.1.0/dartdoc_test

Publishing dartdoc_test 0.1.0 to [https://pub.dev:](https://pub.dev/)
├── CHANGELOG.md (<1 KB)
├── LICENSE (11 KB)
├── README.md (2 KB)
├── analysis_options.yaml (<1 KB)
├── bin
│   └── dartdoc_test.dart (<1 KB)
├── example
│   ├── CHANGELOG.md (<1 KB)
│   ├── README.md (<1 KB)
│   ├── analysis_options.yaml (1 KB)
│   ├── lib
│   │   ├── error_example.dart (1 KB)
│   │   └── example.dart (2 KB)
│   ├── pubspec.yaml (<1 KB)
│   └── test
│       └── dartdoc_test.dart (<1 KB)
├── lib
│   ├── dartdoc_test.dart (2 KB)
│   └── src
│       ├── analyzer.dart (2 KB)
│       ├── command
│       │   ├── add.dart (2 KB)
│       │   ├── analyze.dart (2 KB)
│       │   └── command_runner.dart (2 KB)
│       ├── dartdoc_test.dart (2 KB)
│       ├── extractor.dart (6 KB)
│       ├── logger.dart (3 KB)
│       ├── model.dart (3 KB)
│       ├── reporter.dart (6 KB)
│       └── resource.dart (4 KB)
├── mono_pkg.yaml (<1 KB)
├── pubspec.yaml (<1 KB)
└── test
    ├── analyzer_test.dart (2 KB)
    ├── dartdoc_test.dart (<1 KB)
    ├── extractor_test.dart (3 KB)
    ├── integration_test.dart (2 KB)
    └── testdata
        ├── code_sample
        │   ├── error_example_0.dart (<1 KB)
        │   ├── error_example_1.dart (<1 KB)
        │   ├── error_example_2.dart (<1 KB)
        │   ├── error_example_3.dart (<1 KB)
        │   ├── example_0.dart (<1 KB)
        │   ├── example_1.dart (<1 KB)
        │   ├── example_2.dart (<1 KB)
        │   ├── example_3.dart (<1 KB)
        │   ├── example_5.dart (<1 KB)
        │   └── example_6.dart (<1 KB)
        ├── dartdoc_test.txt (2 KB)
        ├── run analyze command by default with verbose flag.txt (2 KB)
        ├── run analyze command by default.txt (1 KB)
        ├── run analyze command with exclude option.txt (<1 KB)
        ├── run analyze command with verbose flag.txt (2 KB)
        ├── run analyze command.txt (1 KB)
        └── show help.txt (<1 KB)

Total compressed archive size: [19](https://github.com/google/dart-neats/actions/runs/10595651623/job/29361988970#step:5:20) KB.
Validating package...
Package validation found the following potential issues:
* line 1, column 1 of test/testdata/code_sample/example_5.dart: This package does not have example in the `dependencies` or `dev_dependencies` section of `pubspec.yaml`.
    ╷
  1 │ import 'package:example/example.dart';
    │ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ╵
* line 1, column 1 of test/testdata/code_sample/error_example_3.dart: This package does not have example in the `dependencies` or `dev_dependencies` section of `pubspec.yaml`.
    ╷
  1 │ import 'package:example/error_example.dart';
    │ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ╵
* line 1, column 1 of test/testdata/code_sample/example_6.dart: This package does not have example in the `dependencies` or `dev_dependencies` section of `pubspec.yaml`.
    ╷
  1 │ import 'package:example/example.dart';
    │ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ╵
* line 1, column 1 of test/testdata/code_sample/error_example_2.dart: This package does not have example in the `dependencies` or `dev_dependencies` section of `pubspec.yaml`.
    ╷
  1 │ import 'package:example/error_example.dart';
    │ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ╵
* line 1, column 1 of test/testdata/code_sample/example_1.dart: This package does not have example in the `dependencies` or `dev_dependencies` section of `pubspec.yaml`.
    ╷
  1 │ import 'package:example/example.dart';
    │ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ╵
* line 1, column 1 of test/testdata/code_sample/example_3.dart: This package does not have example in the `dependencies` or `dev_dependencies` section of `pubspec.yaml`.
    ╷
  1 │ import 'package:example/example.dart';
    │ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ╵
* line 1, column 1 of test/testdata/code_sample/error_example_1.dart: This package does not have example in the `dependencies` or `dev_dependencies` section of `pubspec.yaml`.
    ╷
  1 │ import 'package:example/error_example.dart';
    │ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ╵
* line 1, column 1 of test/testdata/code_sample/example_2.dart: This package does not have example in the `dependencies` or `dev_dependencies` section of `pubspec.yaml`.
    ╷
  1 │ import 'package:example/example.dart';
    │ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ╵
* line 1, column 1 of test/testdata/code_sample/example_0.dart: This package does not have example in the `dependencies` or `dev_dependencies` section of `pubspec.yaml`.
    ╷
  1 │ import 'package:example/example.dart';
    │ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ╵
* line 1, column 1 of test/testdata/code_sample/error_example_0.dart: This package does not have example in the `dependencies` or `dev_dependencies` section of `pubspec.yaml`.
    ╷
  1 │ import 'package:example/error_example.dart';
    │ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ╵

Package has 10 warnings.
The server may enforce additional checks.
Error: Process completed with exit code [65](https://github.com/google/dart-neats/actions/runs/10595651623/job/29361988970#step:5:66).

[sigurdm]

I think we can fix this one by not analyzing test at all.

I think it is reasonable to only look at lib/ and bin/ for publication validation.

But we should probably also look into some kind of opt-out mechanism for these validations (we have --skip-validation but that is very coarse grained).