`externalSchematics` should install packages locally when they are no present in the `node_modules`

[d-koppenhagen]

trafficstars

🚀 Feature request

Command (mark with an x)

• [ ] new
• [ ] build
• [ ] serve
• [ ] test
• [ ] e2e
• [ ] generate
• [ ] add
• [ ] update
• [ ] lint
• [ ] extract-i18n
• [ ] run
• [ ] config
• [ ] help
• [ ] version
• [ ] doc
• [x] other

Description

when creating own schematics, they can make use of other schematics by calling the externalSchematic function provided by the @angular-devkit/schematics collection.

To call an external schematic, the function can be used like this:

options = {};
externalSchematic('my-schematic-package', 'my-schematic-name', options);

But to execute an external schematic package it has to be part of the node_modules. Internally the CLI uses the function for example for calling the PWA schematic. This won't be problematic as is already installed as part of the @angular/schematics package and in fact it's part of the node_modules.

Once you'll try to install an external schematic which isn't already available in the node_modules the schematic fails.

A workaround is to manually integrate an installation procedure first and then call the function like this:

export const installNpmPackage = (
  context: SchematicContext,
  packageName: string,
): Promise<void> => {
  return new Promise<void>((resolve) => {
    context.logger.info(
      `📦 Installing package '${packageName}' for external schematic setup...`,
    );
    const spawnOptions: SpawnOptions = { stdio: 'inherit' };
    spawn('npm', ['install', packageName], spawnOptions).on(
      'close',
      (code: number) => {
        if (code === 0) {
          context.logger.info(
            `✅ '${packageName}' package installed successfully`,
          );
          resolve();
        } else {
          const errorMessage = `❌ installation of '${packageName}' package failed`;
          context.logger.error(errorMessage);
          throw new Error();
        }
      },
    );
  });
};

const packageName = '@briebug/cypress-schematic';
await installNpmPackage(context, packageName);
return externalSchematic(packageName, 'ng-add', {
  removeProtractor: true,
  addCypressTestScripts: true,
});

Describe the solution you'd like

It would be great to integrate the installation process for a package that's not present in the node_modules in the externalSchematic function.

Describe alternatives you've considered

[alan-agius4]

Hi @d-koppenhagen,

Thanks for opening this feature request.

I don’t think that externalSchematics should install the package if missing. There are multiple required options that are not available for this method.

• Should the dependency be save as a devDependency, no-save or direct dependency?
• Which package manager to use?
• Which version/tag of package that contains the schematic should be used?
• Possible alternate private npm registry url.

In most cases when having a schematic that depends on an external schematic, the author should add the package that contains the external schematic as a direct dependency. This also ensures that the correct version of the external schematic is installed as otherwise you might end up with a broken behaviour when the API of the dependent schematic is changed.

If having it as a direct dependency is not possible, running a separate install task prior of running externalSchematics is a more reasonable and cleaner approach IMHO.

[d-koppenhagen]

Hey @alan-agius4 , I unserstand. But why no let users pass those options as an extra object. If the object is present, install the package using the definition in the object like this:

const options = {};
const installOptions = {
   type: NodeDependencyType.Dev, // enum with all dependency types, if not defined, it won't be added as dependency to the package.json file
   version: '~2.27.0', // use 'latest' by default when not defined
   overwrite: true, // use 'false' and log an error by default
   packageManager: PackageManager.Npm // or yarn or pnpn. If not defined: take the default that's defined in `angular.json`
   registry: 'https://...' // use "normal" NPM registry by default (or assume that it's already define in .npmrc file (locally or globally)
};
externalSchematic('my-schematic-package', 'my-schematic-name', options, installOptions);
// only install when installOptions object is present

[alan-agius4]

Mind sharing the use-case of not having the external dependency as a direct dependency of the schematic package?

I still think that the installation of the package should be a separate task and not part of externalSchematic method at least with the current version of schematics.

[d-koppenhagen]

it might just be relevant when I want to combine external schematics and want to make use of them only once. For example when I will create a schematics collection for an enterprise that will setup an angular workspace with companies defaults. If the company for example decided to use always cypress instead of some protractor (to use the example from the description in this issue), it might just make sense to execute the external schematic from @briebug/cypress-schematic initially. This will add all the stuff needed and adds Cypress etc. to the workspace. But this would be an initial one-time-job, so no need to keep @briebug/cypress-schematic as dependency as it just sets up something and can be dropped afterwards, can't it?

[pfeileon]

@d-koppenhagen Your workaround isn't feasible, as well, unless you are fine with creating a node_modules folder and package-lock.json file in your working directory and either leaving or deleting these in a cleanup step. It's basically the same outcome as using NodePackageInstallTask (like in this stackoverflow issue).

Since Angular got rid of a built-in linting configuration, I'm trying to ng add @angular-eslint/schematics from a custom ng-new schematic and can't overcome this issue. If externalSchematics would at least accept a path to node_modules, said workaround could be used without side-effects by creating the project folder by hand and installing the packages there.

[angular-robot[bot]]

Just a heads up that we kicked off a community voting process for your feature request. There are 20 days until the voting process ends.

Find more details about Angular's feature request process in our documentation.

[angular-robot[bot]]

Thank you for submitting your feature request! Looks like during the polling process it didn't collect a sufficient number of votes to move to the next stage.

We want to keep Angular rich and ergonomic and at the same time be mindful about its scope and learning journey. If you think your request could live outside Angular's scope, we'd encourage you to collaborate with the community on publishing it as an open source package.

You can find more details about the feature request process in our documentation.

[pfeileon]

It's kind of sad that the bot closes the voting process after 20 days while the documentation says:

To include the community in the feature request process, we open voting for 60 days.

Anyway, my interpretation is that Schematics is basically dead atm. (Which is really disappointing since it is the one thing which really would've separated Angular from every other frontend framework/library.)

[dgilsonAfelio]

So sad to see this feature dead

[Hyperxq]

it might just be relevant when I want to combine external schematics and want to make use of them only once. For example when I will create a schematics collection for an enterprise that will setup an angular workspace with companies defaults. If the company for example decided to use always cypress instead of some protractor (to use the example from the description in this issue), it might just make sense to execute the external schematic from @briebug/cypress-schematic initially. This will add all the stuff needed and adds Cypress etc. to the workspace. But this would be an initial one-time-job, so no need to keep @briebug/cypress-schematic as dependency as it just sets up something and can be dropped afterwards, can't it?

I think it's a real problem. I will describe your scenario and an additional one:

1. I want to execute a schematic only one time for this reason I don't want to install it permanently.
2. I am calling external schematic but they are not already installed.

For your scenario I think who needs to install these schematics temporally is the cli. What happen if we can define all these options in the schema.json where you can define all the description of the dependency. Additionally, you can define if this is a temporal dependency.

The problem if the installation occur inside the schematic is that is not a responsibility of the schematic and externalSchematic only call it.

[Hyperxq]

I propose a solution for this problem:

I think we can add an additional field to the schema.json with the name: dependencies. Where you can specify all the information of the schematics to download before the schematic execution.

{
  "$schema": "http://json-schema.org/draft-07/schema",
  "id": "MySchematic",
  "title": "My Schematic",
  "type": "object",
  "properties": {
    "name": {
      "type": "string",
      "description": "The name of the schematic"
    },
    "path": {
      "type": "string",
      "description": "The path to create the schematic"
    }
  },
  "required": ["name", "path"],
  "dependenciesConfig": {
    "@nrwl/workspace": {
      "temporal": false,
      "type": "dev",
      "version": "^12.0.0",
      "overwrite": false,
      "registry": "https://registry.npmjs.org/"
    }
  }
}

The packageManager needs to be specify with the cli commands.

@d-koppenhagen do you think that can be a useful feature?

[d-koppenhagen]

From my perspective it seems to be a good solution

[Hyperxq]

From my perspective it seems to be a good solution

@d-koppenhagen I will test this solution with Project Builder that is extra layer for angular schematics. If works I will present as a solution for this repo