Automatic default webpack.config.js and externals

[alexkli]

Problem

There serverless-webpack plugin requires some manual work for things that could be automatic for an NodeJS runtime environment (using OpenWhisk myself, but I guess that's true for all providers offering NodeJS):

• must provide and configure webpack.config.js - now you have at least 3 required files for build/deployment with package.json &serverless.yml, taking away from the simplicity of serverless just being one script
• must manually configure webpack externals for the node modules preinstalled on the openwhisk nodejs6 etc. runtimes to optimize the action size (webpack's tree shaking helps, but if your action limit is 1 MB that's easy to hit with huge libraries like aws-sdk that can be on the container)

Proposal

• Generate the webpack.config.js automatically iff serverless-webpack plugin is present (enabled in serverless.yml) and no custom webpack.config.js already exists
• Allow to pass webpack's mode as command line option to serverless deploy or package, which is either production (minimized & obfuscated) or development (keep full sources for debugging).
• Automatically exclude preinstalled node modules present on the docker image via webpack's externals. This can happen by
  • including such list files in the plugin itself for standard openwhisk containers, determined by runtime, fully automatic, or
  • referencing custom files that list these for customized containers but from a shared place, based on runtime and image.
• How these exclusion files are managed and referenced is still an open question.
• Same for comparing the versions - if an older version is provided on the container, then it might fail if it gets excluded.
• Any future additional config options (if needed) can be added inside serverless.yml and/or as cli options and passed through to the automatically generated webpack.config.js

State

I have a working prototype and would be ready to release this as separate plugin, but I think it would work nice as part of a standard plugin. The prototype is in this gist: https://gist.github.com/alexkli/1906091ed9229bf5e39b0d8557e6dbd6

Initially wanted to add this to serverless-openwhisk, but they thought it's better for the webpack plugin itself: https://github.com/serverless/serverless-openwhisk/issues/118

[HyperBrain]

Hi @alexkli , thanks for the detailed explanation and the ideas.

In general, I agree that for some use cases it is helpful to have some automatic configuration possibilities like the ones you describe. However, I think that especially the webpack configuration is very target service specific and would create the need that the webpack plugin has to know details about the services (e.g. which packages to exclude by default etc.). The approach of the plugin is to be completely provider agnostic, which includes the standard packages installed in the provider environment. Additionally it is heavily dependent on the webpack version used (3.x and 4.x are allowed and there are different options in the configuration that have to be set). The plugin must also be agnostic to the webpack version.

I had a look at your Gist and in my opinion the correct way to add a provider specific configuration generation is via a separate plugin. You even could integrate the plugin with the serverless-openwhisk plugin and the serverless-webpack plugin. The webpack plugin exposes hooks for all actions it takes, so that you even can hook into the packaging and adjust the output for OpenWhisk.

[alexkli]

@HyperBrain Yes, thanks, for now I went with a separate plugin, leveraging the various events, and it works great! Not opensource yet, working on that when it's more stable.

Generally, I think it's best if the more complex cases don't make the simple cases unnecessary complicated. I understand that you can have complicated webpack setups that require your own webpack.config.js, but should that force a new developer with a simple case (get my action js with lots of ootb dependencies small enough to be accepted by OpenWhisk) to learn all the pecularities of webpack as well? My experience so far is that many people coming to serverless have zero background on webpack, so to them it's just a means to an end.

Couldn't the serverless-webpack plugin provide default setups for the main providers (AWS, openwhisk, etc.)? In the end it is about using webpack for serverless and making your life easier there 😄

[HyperBrain]

I agree that a complete zero-config for different providers will get the onboarding smoother. Maybe an approch could be that you add serverless-webpack to your package.json and then add something like serverless-webpack-zconfig-openwhisk which will let serverless-webpack use a reasonable zero-config for OpenWhisk. To make that happen, I'd just have to extend the plugin's hook interface and let the zeroconfig module hook into it. The big advantage with that approach is, that the plugin itself stays agnostic, but can be easily configured to do the right things without any configuration settings for any provider. Also adding new providers would be just to add a new config module.

[alexkli]

That's more or less what my current solution in a separate plugin is.

Currently I am using the standard before:package:initialize event to create the webpack.config.js if it does not exist yet, and setup an process.on('exit', ...) hook to make sure it gets removed at the end (success or failure). This works fine - no new events needed.

The downside is that you just end up configuring all these plugins in package.json and serverless.yml as well, which seems like a lot.

But maybe that's more of an issue with the serverless framework itself... and it's focus on being provider neutral. At least in our use case we only use OpenWhisk, so that abstraction isn't super relevant. I can see how that's different for others that have the need to mix and match, though.

[AaronBuxbaum]

For what it's worth, the documentation on Serverless and the actual functionality of this package do not seem to match, unless I'm missing a piece of the puzzle:

Configuration possibilities range from zero-config to fully customizable

One or the other should change to keep things correct