Unable to bundle individual packages (node_modules missing)

[clementAC]

Hi, I have some problems to be able to produce working individual packages. I don't have node_modules in the .zip files

Here is an extract of my serverless conf serverless.yml

plugins:
  - serverless-esbuild
  - serverless-offline-local-authorizers-plugin
  - serverless-offline
  - serverless-stack-output
  - serverless-prune-plugin

custom:
  esbuild:
    bundle: true
    minify: false
    sourcemap: true
    packager: npm
    installExtraArgs: ['--production']
    exclude: ['aws-sdk', 'pg-native']
    plugins: plugins.js
    keepOutputDirectory: true

package:
  patterns:
    - '!node_modules/.prisma/client/query-engine-*'
    - '!node_modules/prisma/query-engine-*'
    - '!node_modules/@prisma/engines/**'
    - 'node_modules/.prisma/client/query-engine-rhel-*'
  individually: true

plugin.js

const { nodeExternalsPlugin } = require('esbuild-node-externals');

module.exports = [nodeExternalsPlugin()];

In the .esbuild/.build/ I have the node_modules with all the dependencies and I can see in the output of serverless deploy the Packing external modules: completed also with all my dependencies present in the package.json When I go to .esbuild/.serverless in one of my individual .zip I can see that I only have the dep indicated in the pattern (i.e: .prisma/client..) and a .js with a js.map files with my function code. If I remove the patterns conf, there only the function code.

When I trigger the functions, I have error complaining about missing modules. The functions works when there is individual: false

Versions (please complete the following information):

• OS: Mac os M1
• serverless: 3.19.0
• esbuild: 0.14.43
• serverless-esbuild: 1.30.0
• esbuild-node-externals: 1.4.1

[samchungy]

We do an analysis of your individual zips and try to only pack those dependencies. Is there a reason you're marking pg-native with exclude?

[clementAC]

no particular reasons but with or without it changes nothing on the behavior of individually: true

[tinchoz49]

@clementAC are you working from a monorepo?

[samchungy]

I don't think I can quite help you here without a sample repo? Can you produce something minimal for me to look at?

[tinchoz49]

I have the same issue but because I'm in a monorepo. The problem is how serverless-esbuild resolve the esbuild-node-externals dependency. https://github.com/floydspace/serverless-esbuild/blob/master/src/pack-externals.ts#L211

In a monorepo with PNPM the module cannot be found in that relative path.

Can be solved by doing:

function hasEsbuildNodeExternals () {
  try {
    require.resolve('esbuild-node-externals')
    return true
  } catch () { 
    return false;
  }
}

or another way is to leave the user to configure the externals in a js config file: https://github.com/floydspace/serverless-esbuild/pull/323

[webdeveric]

I'm having the same issue with node_modules missing in the zip artifact.

In my case, I'm using format: 'esm' and platform: 'node'. I've debugged this and found the issue (for me) is in getDepsFromBundle().

I think this function should take into account the output format, not the platform.

https://github.com/floydspace/serverless-esbuild/blob/a233a1a5294ddf0875341312590f0078f468ae26/src/helper.ts#L125-L132

[samchungy]

Yeah that makes sense - when I did this I didn't realise that esbuild pivots on more based on the platform. Also, at the time esm was still very experimental

[clementAC]

Sorry for not answering. I eventually not use individually:true for the past months. I'm now trying to make it work again. On node 14 everything seems to work as expected but as soon as I update to node 16, the behavior is not the same. Patterns seems broken =/

I have this in every function where I want prisma integrated:

 package:
   patterns:
      - '!node_modules/.prisma/**'
      - '!node_modules/prisma/libquery_engine-*'
      - '!node_modules/@prisma/engines/**'
      - 'node_modules/.prisma/client/libquery_engine-rhel-*'
      - 'node_modules/.prisma/client/*.js'

The directives are respected concerning the .prisma and prisma directory but for the @prisma directory nothing is taken into account and I have artifact that bloat my final packages.

The @prisma directory is directly linked to the package whereas the two others are generated maybe it's a kind of a race condition?

As it seems not fully related, I made another issue