[FEATURE REQUEST] flex-plugin-scripts should work in isolation

[clink-aaron]

Feature Request

With v4 of flex-plugin-scripts, it's no longer possible to run scripts like flex-plugin start in isolation. If you try to emulate what the cli does with flex-plugin start plugin --name plugin-example --port 8080 it still reaches out to ~/.twilio-cli/flex/plugins.json.

It's potentially problematic to have a development process dependent on anything outside a project directory. if working on entirely different projects or say different versions of the same project, it could lead to side effects.

• testing via dockerized builds leads to issues with incorrect absolute paths between local/containerized
• workarounds for dockerized environments don't work for native development

We want to allow developers to pull down a plugin, run npm start and start developing without having to be concerned about external tools, and keeping those tools current. Version 3 of flex-plugin-scripts used this philosophy, and it was ideal.

CI deployment via twilio-clli is essentially unsupported because of this.

Description of Solution

flex-plugin-scripts should be entirely isolated from any dependency on configuration files outside its project directory

• flex-plugin-scripts could be configured to accept the additional parameters via cli arguments
• @twilio-labs/plugin-flex could be configured to provide the required parameters

Additional Context

For now we can continue to use v3 of flex-plugin-scripts but that's hardly an ideal solution as we'd prefer to keep our deployment process up to date and current.

It's apparent there's quite a push for twilio-cli, despite the documentation indicating the twilio-cli way of deployment "beta", the flex UI considers v3 deployments "legacy", implying it could eventually be unsupported and deprecated.

[rnairtwilio]

@clink-aaron If you run start command in the plugin directory without using --name, it should run the plugin locally and bypass the plugins.json file settings. That could be a potential workaround.

[clink-aaron]

@rnairtwilio that has not been my experience. To better demonstrate the problem I've created a git repo here where you can reproduce the problem.

The output gives the following error:

Error: ENOENT: no such file or directory, open '/root/.twilio-cli/flex/plugins.json'
    at Object.openSync (fs.js:458:3)
    at Object.readFileSync (fs.js:360:35)
    at readFileSync (/app/node_modules/flex-dev-utils/dist/fs.js:217:25)
    at Object.readJsonFile (/app/node_modules/flex-dev-utils/dist/fs.js:230:44)
    at Object.readPluginsJson (/app/node_modules/flex-dev-utils/dist/fs.js:254:20)
    at parseUserInputPlugins (/app/node_modules/flex-plugin-scripts/dist/utils/parser.js:22:23)
    at /app/node_modules/flex-plugin-scripts/dist/scripts/start.js:250:71
    at step (/app/node_modules/flex-plugin-scripts/dist/scripts/start.js:52:23)
    at Object.next (/app/node_modules/flex-plugin-scripts/dist/scripts/start.js:33:53)
    at fulfilled (/app/node_modules/flex-plugin-scripts/dist/scripts/start.js:24:58) {
  errno: -2,
  syscall: 'open',
  code: 'ENOENT',
  path: '/root/.twilio-cli/flex/plugins.json'
}

[ktalebian]

@clink-aaron flex-plugin * commands are no longer supported. However, we do in fact run each plugin in isolation. When you run twilio flex:plugins:start --name plugin1 --name plugin2 we run a separate process per plugin, see code https://github.com/twilio/flex-plugin-builder/blob/main/packages/plugin-flex/src/commands/flex/plugins/start.ts#L119 and the few lines after that sets the environment and then spawns a separate isolated child process per plugin.

You can try running the start command and then running ps aux | grep flex on a separate terminal to see how each plugin is running on a separate pid.

What is the actual problem you are running into though?

[clink-aaron]

@ktalebian To summarize the issues I mentioned in my initial ticket:

• deployment infrastructure now requires dependency on twilio-cli and associated plugins, as well as maintenance of those environments
• development is no longer stand-alone, "clone, npm install, and go"
• dockerized development would require an access to home directory. This is bad practice.

One of the main benefits of dockerized development is complete isolation of a project. Sure you could add a volume, but then you have a hidden but vital piece of your app, and nuking deps and re-installing wouldn't even fix.

To clarify your other points I should have been more descriptive. by "in isolation" I'm meaning "decouple from twilio-cli".

requiring the twilio-cli also then requires a dependency of @twilio-labs/plugin-flex. For some reason, twilio decided to implement their own plugin management system rather than using npm. They're storing plugins in a hardcoded, unconfigurable directory (~/.twilio-cli).

workarounds

There are some ways to work around this limitation, but it's frustrating to need to do so in the first place, and doesn't work well for workspaces. Example: https://github.com/clink-aaron/flex-plugin-poc/blob/cli-demo/package.json

this would work, but then child apps are going to treat the working directory as home as well. Not ideal.

resolution

This is kinda a perfect storm of unfortunate design decisions. On the bright side, it means there's more than one suitable way to address this:

• Restore support for running flex-plugin scripts directly, remove any dependency on ~/.twilio-cli config directory, and instead rely on passed params (my initial proposal).

OR

• allow twilio-cli config directory to be configurable via cli flag (also would require updating cli plugins to consume new param).

OR

• refactor twilio-cli to first check the project root for a .twilio-cli directory. This would be a great solution, easy to implement, and would work across workspaces and dockerized environments.
• would be consistent with existing conventions (eg: .npmrc, .yarnrc)

[ktalebian]

@clink-aaron the entire Twilio org is moving behind using the Twilio CLI as the main/only interface for their product. This is a topic that requires discussion with the greater organization and cannot be made directly by our team only.

I will bring this up in our next meeting and will report back here.

[dgradwell-ams]

@ktalebian any updates on this?

[anjha91]

Hi,

This is a long open item more than a year now, hence closing it. Feel free to reopen it if still required.