deno
deno copied to clipboard
denoland
deno server memory is larger than nodejs
My team has been using Deno for more than a year. But I found that the memory consumption in docker container is relatively large, which is more than twice that of nodejs with the same function.
This is frustrating, but strangely, if I use Deno inside the program Deno.memoryUsage() is even better than nodejs' process which used process.memoryUsage(). The value is smaller, but it is another case in docker stats.
Here is the test sample of docker I used, I use the origin nodejs http and origin Deno http test, and also test koa and oak.
The console logs:
deno_1 | heapTotal 3.53MB，heapUsed 3.07MB，rss 3.70MB，external:0.00MB
oakjs_1 | heapTotal 5.42MB，heapUsed 3.95MB，rss 4.74MB，external:0.00MB
oak_1 | heapTotal 6.78MB，heapUsed 5.23MB，rss 6.27MB，external:0.00MB
koa_1 | heapTotal 6.93MB，heapUsed 5.50MB，rss 24.12MB，external:0.78MB
node_1 | heapTotal 4.92MB，heapUsed 4.22MB，rss 18.33MB，external:0.52MB
But docker stats：
CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
deno c55528ee5171 0.00% 9.355 MiB / 3.858 GiB 0.24% 2.27 kB / 656 B 0 B / 0 B 5
oakjs a1074c23261b 0.00% 18.3 MiB / 3.858 GiB 0.46% 1.88 kB / 656 B 0 B / 0 B 5
oak efafc26ffd90 0.00% 24.75 MiB / 3.858 GiB 0.63% 1.88 kB / 656 B 0 B / 0 B 5
koa 78b030ee5801 0.00% 9.227 MiB / 3.858 GiB 0.23% 1.97 kB / 656 B 0 B / 0 B 7
node ae1d64819c6f 0.00% 6.715 MiB / 3.858 GiB 0.17% 3.1 kB / 656 B 0 B / 0 B 7
The oak bundled js seems to be better than oak which started on typescript.
The following are my two actual Deno projects, which is also the reason why I did this test. The console logs:
project1: heapTotal 50.32MB，heapUsed 47.62MB，rss 49.13MB，external:22.68MB
project2: heapTotal 9.14MB， heapUsed 8.04MB， rss 8.63MB， external:1.16MB
docker stats:
CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
project1 43144c7cdcb4 0.03% 272.2 MiB / 3.858 GiB 6.89% 51.1 kB / 21.9 kB 0 B / 0 B 5
project2 64700e6c8c7b 0.03% 97.79 MiB / 3.858 GiB 2.48% 516 B / 516 B 0 B / 0 B 6
The above tests were conducted in the docker environment of Linux, nodejs image is node:16-alpine and deno image is denoland/deno:alpine-1.23.2：
Server Version: 1.13.1
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 3
CPUs: 4
It's very comfortable to develop with Deno. I personally like it better than nodejs, but in the actual operation process, the memory occupation is embarrassing. I hope to solve this problem. I believe that the bottom layer is the deno of Rust, In this regard, it should be the advantage, not the opposite.
I have similar experience with deno. The memory consumption is usually twice as large as that of node when running the same application.
One of the big differences of Deno versus Node.js is that when using TypeScript and type checking, the TypeScript type checker (tsc) is loaded into memory. We don't unload this once we are bootstrapped IIRC. It is something we should look into.
Have you tried comparing a workload generated by deno compile against Node.js? The TypeScript type checker is never loaded in those cases.
One of the big differences of Deno versus Node.js is that when using TypeScript and type checking, the TypeScript type checker (tsc) is loaded into memory. We don't unload this once we are bootstrapped IIRC. It is something we should look into.
Have you tried comparing a workload generated by
deno compileagainst Node.js? The TypeScript type checker is never loaded in those cases.
Thank you for your reply.
I just tested it with deno compile, and only made the following modifications. It is indeed half of the memory. It dropped from 21.11M to 12.78M.
I changed the dockerfile of oak like this:
FROM denoland/deno:alpine-1.23.2
EXPOSE 8000
WORKDIR /app
# Prefer not to run as root.
RUN chown -R deno /app
RUN chmod 755 /app
ADD . .
# CMD deno run --allow-net mod.ts
RUN deno compile --allow-net -o oak mod.ts \
&& chmod +x ./oak
CMD ./oak
However, this leads to additional problems.
First, using the above method to build will cause the docker image volume to increase. Of course, I can try docker's as builder, but I don't have a suitable Linux machine to test for the time being.
Another problem is that I found before that I use decorators in my project, but it does not use its information when using deno compile to build. Here is one of my test cases:
import { Reflect } from "https://deno.land/x/[email protected]/mod.ts";
function Prop(target: any, propertyKey: string) {
const type = Reflect.getMetadata("design:type", target, propertyKey);
console.log("type", type); // type [Function: String]
}
class Person {
@Prop
name!: string;
}
Normally, it should print [function: string], when my tsconfig.json has the following configuration enabled:
{
"compilerOptions": {
"emitDecoratorMetadata": true,
"experimentalDecorators": true,
}
}
But when I use deno compile --config tsconfig.json aa.ts and when executing the generated file, it print type undefined.
Personally, I hope to make less changes when using docker, such as adding a configuration to ignore the TS check at runtime. Of course, don't forget to read tsconfig.json first so that my decorator can still take effect.
Of course, I can try docker's as builder, but I don't have a suitable Linux machine to test for the time being.
Do you mean a multi-stage Docker build? If so, you should be able to do that without a Linux machine.
Of course, I can try docker's as builder, but I don't have a suitable Linux machine to test for the time being.
Do you mean a multi-stage Docker build? If so, you should be able to do that without a Linux machine.
Yes, my Linux server can't upgrade docker temporarily, so I can't use this function.
I tried with MBP. It may be the optimization relationship of M1 pro, and the statistical memory occupation is more exaggerated. This makes me more confused.
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
1f4c93753884 deno_node_docker-oak_compile-1 0.04% 118MiB / 7.667GiB 1.50% 1.02kB / 0B 0B / 0B 7
f7fbd7ce25e1 deno_node_docker-node-1 0.00% 7.562MiB / 7.667GiB 0.10% 1.09kB / 0B 0B / 0B 7
17e462a92932 deno_node_docker-oakjs-1 0.02% 125.5MiB / 7.667GiB 1.60% 1.02kB / 0B 0B / 0B 7
86ae2b7448d9 deno_node_docker-koa-1 0.00% 10.43MiB / 7.667GiB 0.13% 1.02kB / 0B 0B / 0B 7
5f27d980543c deno_node_docker-deno-1 0.02% 112MiB / 7.667GiB 1.43% 1.02kB / 0B 0B / 0B 7
fdcfc9d30df5 deno_node_docker-oak-1 0.01% 143.9MiB / 7.667GiB 1.83% 1.02kB / 0B 0B / 0B 7
Here I write the dockerfile file in this way:
FROM denoland/deno:alpine-1.23.2 as builder
WORKDIR /app
COPY . .
RUN deno compile --allow-net -o oak mod.ts \
&& chmod +x oak
FROM denoland/deno:alpine-1.23.2
WORKDIR /app
COPY --from=builder /app/oak /app
CMD ./oak
If the from in line 10 is modified to alpine:latest, an error is reported when running:
qemu-x86_64: Could not open '/lib64/ld-linux-x86-64.so.2': No such file or directory
I may need to find a minimum image that contains this file.
@jiawei397 This multi-stage build Dockerfile is working (deployment tested with flyio). You have to use the same alpine variant (frolvlad/alpine-glibc:alpine-3.17 ) as deno base image. The image size is 148 MB. The memory usage displayed in flyio is 58 MB.
ARG DENO_VERSION=1.36.3
FROM denoland/deno:alpine-${DENO_VERSION} AS builder
WORKDIR /app
# Cache the dependencies as a layer
COPY deno.json .
# These steps will be re-run upon each file change in your working directory:
ADD . .
# Compile the main app
RUN deno compile --allow-net --allow-read --allow-env main.ts -o main
# Use a Docker multi-stage build to create a lean production image.
# Use the same alpine image as deno one because of glibc requirement.
FROM frolvlad/alpine-glibc:alpine-3.17 as runner
WORKDIR /app
# Prefer not to run as root.
RUN addgroup --gid 1000 deno \
&& adduser --uid 1000 --disabled-password deno --ingroup deno \
&& chown deno:deno /app/
USER deno
COPY --from=builder /app/main .
# The port that your application listens to.
EXPOSE 8000
CMD ["./main"]
Good job! I've written a simple Oak example, and it works perfectly. It's fantastic However, my project uses a lot of decorators, and it doesn't run after compilation. The bugs in Deno's bundle and compile have not been fixed, which is quite unfortunate.
That's a pity about the issue with the decorators. Hopefully, it will be resolved in the next Deno release 🤞.
Update by 2023-09-16, Deno 1.36.2.
This time, I added the Docker comipile builder.
deno_node_docker-deno-1 | heapTotal 8.52MB，heapUsed 6.64MB，rss 30.17MB，external:0.07MB
deno_node_docker-oak_compile_builder-1 | heapTotal 9.96MB，heapUsed 7.85MB，rss 32.70MB，external:0.07MB
deno_node_docker-koa-1 | heapTotal 7.51MB，heapUsed 5.90MB，rss 24.87MB，external:0.81MB
deno_node_docker-node-1 | heapTotal 5.00MB，heapUsed 4.51MB，rss 18.88MB，external:0.55MB
deno_node_docker-oak-1 | heapTotal 9.96MB，heapUsed 7.84MB，rss 47.28MB，external:0.07MB
deno_node_docker-oakjs-1 | heapTotal 9.41MB，heapUsed 7.94MB，rss 36.78MB，external:0.07MB
Below is docker stats:
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
0c68c6a5bed1 deno_node_docker-deno-1 0.00% 11.09MiB / 15.67GiB 0.07% 7.79kB / 2kB 0B / 305kB 9
6d9fa0470f41 deno_node_docker-oak-1 0.00% 25.21MiB / 15.67GiB 0.16% 835kB / 31.8kB 0B / 13.5MB 9
3de09fa86a55 deno_node_docker-oak_compile_builder-1 0.00% 13.97MiB / 15.67GiB 0.09% 126B / 0B 0B / 0B 8
016485ff328c deno_node_docker-koa-1 0.00% 9.414MiB / 15.67GiB 0.06% 126B / 0B 0B / 0B 7
29a4d092116d deno_node_docker-node-1 0.00% 7.18MiB / 15.67GiB 0.04% 126B / 0B 0B / 0B 7
e321aeecc851 deno_node_docker-oakjs-1 0.00% 16.09MiB / 15.67GiB 0.10% 7.79kB / 1.93kB 0B / 594kB 9
It seems that the effect after compiling is better than the JS version, and even better than the TS version.