evanw
Provide support for AWS Inspector compatability
Issue
When using esbuild (with --bundle and --minify options) to package projects for use in a Lambda function, the function can not be scanned by Amazon Inspectors SBOM generator tool.
Ideal Behaviour
It would be nice to be able to pass an option that could retain a 'skeleton' node_modules directory that just contains a package.json for each dependency, like so:
- node_modules
- dependencyA
- package.json
- dependencyB
- package.json
- dependencyA
This would allow the SBOM generator tool to find the required dependency metadata to successfully scan for vulnerabilities.
Steps to reproduce:
esbuild src/index.js --bundle --minify --platform=node --target=node18 --outfile=build/index.js
Other possible fixes
It's possible to add a postbuild step which achieves the same thing. For a single project this is an easy fix but for a large organisation with many teams/AWS accounts it is not a great solution and a native esbuild option would be preferable.
I'm not sure what esbuild needs to do here. Does any other bundler do something similar? If you want to gather bundled dependencies' names, you can enable metafile.
Hi and thanks for getting back to me on this :)
AWS Inspector uses a tool called 'SBOM generator'. In the case of Lambda functions this creates a .json file for each function containing a list of all dependencies and a list of known vulnerabilities that are connected with those dependencies.
What we see with projects that are bundled is that the dependency list is always empty because the directory structure that the SBOM generator tool is expecting to find doesn't exist anymore. The tool is attempting to look through the node_modules directory and checks the name and version in the package.json per dependency.
What we'd like is to be able re-create the node_modules directory with only package.json files per dependency, and only for those dependencies that have been bundled. In an example build directory this would look like this (where node_modules is in the above structure and index.js is the bundled project):
- build
- node_modules
- index.js
This will allow the SBOM generator tool to scan the project for vulnerabilities while also keeping the build size as small as possible.
Here's a bash command that uses the output of the metafile to do this. It would be great to be able to pass an option that could achieve the same thing.
jq -r '.inputs | keys[] | select(startswith("node_modules/")) | split("/")[1]' build/meta.json | xargs -I{} find node_modules/{}/package.json | cpio -pd ./build
Thanks again for the response :)
James
The metafile is the way to do this with esbuild. I'm not going to built support into esbuild itself for each individual build metadata task that people want to do. Instead, people should write tools that use esbuild's standard metadata for their individual build metadata tasks. Closing as I do not intend to do this.
Hi Evan,
Thanks a lot for getting back to me on this. I completely understand your reservations on this and to be honest I would have been surprised if you’d have said yes but we’re in a difficult place with this and trying all options!
We’ve also been in conversations with AWS about this and we’ve made some progress there – they now support various package manager lock files as a method of discovering dependencies. It is possible that we could bring them round to supporting the esbuild metafile natively as CDK uses esbuild so it’s important for them to find a neat solution to this.
The blocker here is that the metafile (unless you can tell me otherwise and I’d be very happy for you to!) does not include version numbers for the included packages. This is essential to be able to judge whether a package has known vulnerabilities.
Would there be any scope to adding version numbers for included packages in the metafile?
Thanks a lot and I really appreciate you taking the time to read this!
James
From: Evan Wallace @.> Date: Friday, 20 December 2024 at 02:01 To: evanw/esbuild @.> Cc: James Smith - Prod Eng - Delivery Engineering @.>, Author @.> Subject: Re: [evanw/esbuild] Provide support for AWS Inspector compatability (Issue #3871)
External: Think before clicking
The metafile is the way to do this with esbuild. I'm not going to built support into esbuild itself for each individual build metadata task that people want to do. Instead, people should write tools that use esbuild's standard metadata for their individual build metadata tasks. Closing as I do not intend to do this.
— Reply to this email directly, view it on GitHubhttps://github.com/evanw/esbuild/issues/3871#issuecomment-2556112347, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BIZZ4NMYUPQYCBPXWA2XWY32GN3ADAVCNFSM6AAAAABMGRHIHCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNJWGEYTEMZUG4. You are receiving this because you authored the thread.Message ID: @.***>
One more thought – I’d be up for implementing version numbers for packages in the metafile (as described in my last message) if you’d be open to a PR for it 😊
Thanks,
James
From: Evan Wallace @.> Date: Friday, 20 December 2024 at 02:01 To: evanw/esbuild @.> Cc: James Smith - Prod Eng - Delivery Engineering @.>, Author @.> Subject: Re: [evanw/esbuild] Provide support for AWS Inspector compatability (Issue #3871)
External: Think before clicking
The metafile is the way to do this with esbuild. I'm not going to built support into esbuild itself for each individual build metadata task that people want to do. Instead, people should write tools that use esbuild's standard metadata for their individual build metadata tasks. Closing as I do not intend to do this.
— Reply to this email directly, view it on GitHubhttps://github.com/evanw/esbuild/issues/3871#issuecomment-2556112347, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BIZZ4NMYUPQYCBPXWA2XWY32GN3ADAVCNFSM6AAAAABMGRHIHCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNJWGEYTEMZUG4. You are receiving this because you authored the thread.Message ID: @.***>
The blocker here is that the metafile (unless you can tell me otherwise and I’d be very happy for you to!) does not include version numbers for the included packages. This is essential to be able to judge whether a package has known vulnerabilities.
The information in esbuild's metafile can tell your tool which files are in the bundle. From there, your tool can use that information to look up the relevant information in package.json on the file system.