iov-core
iov-core copied to clipboard
iov-one
Evaluate and trim down dependencies
The recent event-stream exploit leads to renewed concern about security of dependencies.
The easiest steps to take are to limit the number of dependencies, and maximize the quality of the dependencies. (Before starting to audit them all). By quality, we can usually look at activity and reputation of maintainers.
To start with, it would be good to determine what dependencies we actually install when we import the library, rather than those used to compile and test (exclude devDependencies). From that list, we should see if anything stands out as dangerous to be eliminated, and if there are any easy ones to trim (only used by one place in the code, easy to pull out, etc).
I consider this issue a first pass maintenance chore, to get current state mainly, not a deep audit.
Current result of yarn install --prod yields 157 packages in node_modules. (Maybe more, I think we need --flat as well to really show them all).
A quick glance shows some likely redudancy if nothing else:
safe-buffer
safer-buffer
As well as what look like devDependencies:
source-map
source-map-support
This is not high priority, but more just some repo dusting for a rainy day when the mind is not ready for hard coding...
nB: I have discovered many npm repos to be in effect one index.js file with less than 10 lines of code. Since this is usually simple logic not going to change, we can copy/reimplement (depending on license) and place in a util package.
Example: https://github.com/brummelte/sleep-promise to allow await sleep. Overkill, huh? (v8.0.1 on github???)
Reducing dependencies is a continuously ongoing operation with the following steps done most recently:
- Convert Ganache CLI installation from npm to docker (removed 1/3 of all deps https://github.com/iov-one/iov-core/pull/625)
- Convert @iov/bns from runtime to dev dependency in @iov/core (https://github.com/iov-one/iov-core/pull/632)
- Implement Ethereum support with only 1 external dependency (https://github.com/iov-one/iov-core/blob/v0.10.2/packages/iov-ethereum/package.json#L33)
- Remove Ripemd160 support asap (https://github.com/iov-one/iov-core/issues/589)
The problem with the packages mentioned above is: we don't import them directly. Those are transient dependencies introduced by other maintainers and as long there is nothing seriously wrong with those packages it will be hard to convince them to remove those packages.
Great article by the way. Time for reproducible build culture in JS world.
Our monorepo yarn.lock is not published to npm. So caller projects (no matter if they use npm or yarn) do not get our fixed versions. I think this is what NPM shrinkwrap is for. Is there a yarn equivalent?
Good point... I think we need to publish the root yarn.lock to npm for every package. It may be a superset of what is needed, but will only constrain what is actually installed in the package.json dependencies.
If you are using an npm-shrinkwrap.json file right now, be aware that you may end up with a
different set of dependencies. Yarn does not support npm shrinkwrap files as they don’t have
enough information in them to power Yarn’s more deterministic algorithm. If you are using a
shrinkwrap file it may be easier to convert everyone working on the project to use Yarn at the
same time. Simply remove your existing npm-shrinkwrap.json file and check in the newly
created yarn.lock file.
https://yarnpkg.com/lang/en/docs/migrating-from-npm/
Convert Ganache CLI installation from npm to docker (removed 1/3 of all deps #625) Implement Ethereum support with only 1 external dependency (https://github.com/iov-one/iov-core/blob/v0.10.2/packages/iov-ethereum/package.json#L33)
:+1:
The problem with the packages mentioned above is: we don't import them directly. Those are transient dependencies introduced by other maintainers and as long there is nothing seriously wrong with those packages it will be hard to convince them to remove those packages.
I know.. the issue is if we import a package to do one little thing, and it imports three more and... it opens up a number of attack vectors.
If we use a package to do some complex work and it imports three more helpers, that is fine... Not sure if there is a tool to visualize where all dependencies are pulled in from, and how many places we import/use a library in our code. Not about removing essential packages, just about seeing if we can trim out a few edges and drop a lot of weight for minimal cost.
Right now we get a security vulnerability warning for jquery, which is pulled in as a dependency of the typedoc theme. So removing typedoc from the repo's core dependencies list would be nice.
Sounds like a good plan. Only needed for dev dependencies. But maybe outsourced to a script/docker image