Can't build after v2.11.2 - yarn build process gets killed
Describe the Bug
I can't seem to update to a version after 2.11.2. yarn build gets stuck and doesn't finish somehow, server kills the process because it takes too long. Takes around 10 minutes before it gets killed
I'm not sure if it's something to do with umami, yarn or nextjs.
As I have umami on a "central" server every other site goes down when building, which is very worrying. Is nodejs/nextjs always this resource heavy? So unoptimized it needs 8GB RAM and so on?
Database
MySQL
Relevant log output
root@central:/var/www/html/umami# git checkout v2.12.0
Previous HEAD position was a38baa50 Merge pull request #2684 from umami-software/hotfix2
HEAD is now at 87a03785 Merge pull request #2794 from umami-software/dev
root@central:/var/www/html/umami# yarn install && yarn build
yarn install v1.22.22
[1/4] Resolving packages...
[2/4] Fetching packages...
warning [email protected]: The engine "pnpm" appears to be invalid.
[3/4] Linking dependencies...
warning " > @umami/[email protected]" has incorrect peer dependency "@prisma/client@^4.8.0".
warning " > [email protected]" has incorrect peer dependency "next@^13.4.0".
warning " > [email protected]" has incorrect peer dependency "react@^16.8.0 || 17.x".
warning " > [email protected]" has incorrect peer dependency "react-dom@^16.8.0 || 17.x".
warning " > [email protected]" has unmet peer dependency "prop-types@^15.7.2".
warning " > [email protected]" has incorrect peer dependency "rollup@^2.0.0".
warning "stylelint-config-css-modules > [email protected]" has incorrect peer dependency "stylelint@^16.0.2".
warning " > [email protected]" has incorrect peer dependency "stylelint@>= 11.x < 15".
warning " > [email protected]" has incorrect peer dependency "stylelint@^16.0.0".
warning Workspaces can only be enabled in private projects.
[4/4] Building fresh packages...
$ node -e "if (process.env.NODE_ENV !== 'production'){process.exit(1)} " || husky install
husky - Git hooks installed
Done in 93.22s.
yarn run v1.22.22
$ npm-run-all check-env build-db check-db build-tracker build-geo build-app
$ node scripts/check-env.js
$ npm-run-all copy-db-files build-db-client
$ node scripts/copy-db-files.js
Database type detected: mysql
Copied /var/www/html/umami/db/mysql to /var/www/html/umami/prisma
$ prisma generate
Environment variables loaded from .env
Prisma schema loaded from prisma/schema.prisma
✔ Generated Prisma Client (v5.14.0) to ./node_modules/@prisma/client in 535ms
Start using Prisma Client in Node.js (See: https://pris.ly/d/client)
import { PrismaClient } from '@prisma/client'
const prisma = new PrismaClient()
or start using Prisma Client at the edge (See: https://pris.ly/d/accelerate)
import { PrismaClient } from '@prisma/client/edge'
const prisma = new PrismaClient()
See other ways of importing Prisma Client: http://pris.ly/d/importing-client
┌─────────────────────────────────────────────────────────────┐
│ Deploying your app to serverless or edge functions? │
│ Try Prisma Accelerate for connection pooling and caching. │
│ https://pris.ly/cli/--accelerate │
└─────────────────────────────────────────────────────────────┘
$ node scripts/check-db.js
✓ DATABASE_URL is defined.
✓ Database connection successful.
✓ Database version check successful.
┌─────────────────────────────────────────────────────────┐
│ Update available 5.14.0 -> 5.15.1 │
│ Run the following to update │
│ yarn add --dev prisma@latest │
│ yarn add @prisma/client@latest │
└─────────────────────────────────────────────────────────┘
Environment variables loaded from .env
Prisma schema loaded from prisma/schema.prisma
Datasource "db": MySQL database "umami" at "localhost:3306"
6 migrations found in prisma/migrations
No pending migrations to apply.
✓ Database is up to date.
$ rollup -c rollup.tracker.config.mjs
src/tracker/index.js → public/script.js...
created public/script.js in 524ms
$ node scripts/build-geo.js
Saved geo database: /var/www/html/umami/geo/GeoLite2-City.mmdb
$ next build
▲ Next.js 14.2.3
- Environments: .env
Creating an optimized production build ...
Browserslist: caniuse-lite is outdated. Please run:
npx update-browserslist-db@latest
Why you should do it regularly: https://github.com/browserslist/update-db#readme
Killed
error Command failed with exit code 137.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
ERROR: "build-app" exited with 137.
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Which Umami version are you using? (if relevant)
2.11.2 => 2.12.x
Which browser are you using? (if relevant)
No response
How are you deploying your application? (if relevant)
On a small VPS, by doing git checkout <new version> && yarn install && yarn build
I think the default memory limit for Node is 8GB for a 64bit process. You can lower that to avoid being killed by the Linux OOM-Reaper - https://stackoverflow.com/questions/12779724/limiting-node-jss-memory-usage
It's just the way GC'ed programs work, they avoid GC'ing if they think they have headroom. If the process is killed with OOM, that suggests headroom doesn't actually exist. While programs can optimize for lower allocation throughput, modern GC will still try and not GC until it needs to.
@Silvenga Thanks, gonna try that.
I assume it's only adding the parameter to this line?
https://github.com/umami-software/umami/blob/d7141e626e8ff7fa5973b468a8671c071f0c489c/package.json#L20
My VPS only has 2GB ram, not a lot of processes happen on there but when it's building, the vps goes dead. I don't care if it takes long, i just don't want the build process to take down the whole server, it's so stupid
I found that nextjs also has memory problems and they have this page to optimize stuff
If someone encounters the "killed exit code 137" issue, which is related to low RAM, simply increasing the swap space should resolve the problem.
I also had similar problem with RAM. A 1GB DigitalOcean server cannot create the build, but a 2GB server is fine.
The VPS i initially ran umami on doesn't have any swap and it was 2GB of RAM
I checked my server. 2GB RAM with a 1GB swap file worked fine.
Ok, so swap matters. Imma try creating a swap file too