bench icon indicating copy to clipboard operation
bench copied to clipboard

Bun is 2x faster than Deno

Open aquapi opened this issue 2 years ago • 18 comments

I have benchmarked Deno a few times on my machine and many benchmarks Deno is 2x slower than Bun Maybe you can try use another benchmarking tool like wrk or bombardier to see the result.

Funding

  • You can sponsor this specific effort via a Polar.sh pledge below
  • We receive the pledge once the issue is completed & verified
Fund with Polar

aquapi avatar Aug 05 '23 06:08 aquapi

You can see these benchmarks

  • bunsvr/benchmark
  • SaltyAom/bun-http-framework-benchmark

aquapi avatar Aug 05 '23 07:08 aquapi

There are quite a lot of benchmarks around and not a lot seem to agree with eachother, let alone that bun is 2x faster than deno. For your bunsvr/benchmark, where can I see that bun is 2x faster? I can only see different frameworks not runtimes in the results.

Looking at SaltyAom/bun-http-framework-benchmark there is an open issue that shows windows could impact the results. The Linux results in that thread also wildly vary, one where node + uws wins (uws is also what bun uses). Also the results from this user shows that deno with oak is within ~11%* of bun (edit: missed that JSON results were bugged, thanks @aquapi). Even though in the results on this repo bun is 2.7x faster than deno with oak. Like I said, the benchmarks don't seem to agree on much...

I am not involved with this repository whatsoever, but I think it would be appreciated to find a problem with their methodology instead of dropping a claim like this. For example:

  • Is there an issue with oha the load generator used, how would wrk or bombardier be better?
  • Is there an issue with the code for either deno or bun?
  • What OS is used?
  • Is WSL (Windows Subsystem for Linux) used, if so does it have any impact on the results?
  • What are the version numbers of the frameworks?

Which I think would all be valid questions and would be easier to follow up on. Personally I would love to know version numbers and which OS is used for example. This would also make it easier to validate these results.

JasperVanEsveld avatar Sep 13 '23 14:09 JasperVanEsveld

For simple benchmarking, wrk or wrk2 should be fine. It provides a Lua API for you to work with and wrk is the most performant benchmarking tools.

The result mentioned above has an issue in POST /json test, which made the result inaccurate. @JasperVanEsveld

aquapi avatar Sep 13 '23 23:09 aquapi

bun:  89,218.22 req/s
deno: 70,772.89 req/s
node: 28,542.56 req/s

Intel(R) Core(TM) i7-8565U CPU @ 1.80GHz Linux Mint 21.2 kernel 5.15.0-82-generic

node v20.3.1 bun 1.0.0 deno 1.36.1 (release, x86_64-unknown-linux-gnu) v8 11.6.189.12 typescript 5.1.6

deno.ts

Deno.serve({ port: 3000 }, () => new Response("Hello, Bench!"));

bun.ts

export default { port: 3000, fetch: () => new Response('Hello, Bench!') };

node.js

require('http').createServer((request, response) => (response.write('Hello, Bench!'), response.end())).listen(3000);

bombardier --fasthttp -c 500 -d 10s http://localhost:3000/

All the best deno frameworks stay around 67,689.84 req/s (oak)

icetbr avatar Sep 14 '23 01:09 icetbr

WSL may have some impact on the result as it has some limitation as a virtual machine. Idk if Deno has improved or not, but still.

I suggest you wrap the node handler with setImmediate and see the result

aquapi avatar Sep 15 '23 11:09 aquapi

Like this? No difference

const handler = response => () => {
    response.write('Hello, Bench!');
    response.end();
}
require('http').createServer((request, response) => setImmediate(handler(response))).listen(3000);

icetbr avatar Sep 15 '23 16:09 icetbr

@icetbr test on MacOS or other distro than Linux Mint Bun seems to be performing bad in Linux Mint for some reasons? (I suggest using Ubuntu)

aquapi avatar Sep 18 '23 16:09 aquapi

Mint is Ubuntu. I don't have MacOS, sorry. My benchmarks seems consistent with some articles I found online by others. Post your results if you can.

icetbr avatar Sep 18 '23 16:09 icetbr

@icetbr bunsvr/benchmark is mine

aquapi avatar Sep 24 '23 04:09 aquapi

Cool! I can only see bun frameworks for now, are you updating the repo?

icetbr avatar Sep 24 '23 12:09 icetbr

@icetbr Can you give me a framework code because I don't want to install Deno lint

aquapi avatar Oct 06 '23 07:10 aquapi

Which framework? Deno itself? I've used the ones from here https://github.com/SaltyAom/bun-http-framework-benchmark/tree/main/src/deno

And pure deno

Deno.serve({ port: 3000 }, () => new Response("Hello, Bench!"));

icetbr avatar Oct 06 '23 11:10 icetbr

@eliassjogreen Why are benchmark results flunctuating on every single update? You should check the benchmarking tool again or use wrk2.

aquapi avatar Nov 13 '23 11:11 aquapi

@icetbr Bun performs better in Ubuntu than in Linux Mint

aquapi avatar Nov 20 '23 00:11 aquapi

Do you have the stats? That would be unusual, as I'm not aware of any performance difference between any program between any linux distro.

icetbr avatar Nov 20 '23 01:11 icetbr

@icetbr if you have Mint you can for example test node:test with bun:test. On my ubuntu machine bun is 3x faster It may varies between distros as I see other results on Mint states that node:test is 3x faster

aquapi avatar Nov 21 '23 14:11 aquapi

Do you have any reference of this? It makes no sense, its like saying strawberry ice cream is colder then chocolate ice cream. Mint IS Ubuntu. Even Arch, which is way different, there is no performance difference. I don't doubt your results, but I'm curious of the circumstances of the test.

Even MacOs has the same performance. At least it used to be, I stopped following the benchmarks of dev environments a while ago, when i was deciding between a PC vs Mac for Node development.

icetbr avatar Nov 21 '23 15:11 icetbr

@icetbr See strager YT video

aquapi avatar Nov 23 '23 05:11 aquapi

https://github.com/Vanilla-IceCream/fastify-uws?tab=readme-ov-file#benchmarks

Version Language Router Requests/sec
bun 1.0.33 TypeScript/Bun ✗ 157,740.9324
deno 1.41.3 TypeScript/Deno ✗ 124,009.3235
node 20.11.1 JavaScript/Node ✗ 78,002.1550

Bun is 2x faster than Node, not Deno.

Shyam-Chen avatar Mar 31 '24 15:03 Shyam-Chen

@Shyam-Chen It was

aquapi avatar Apr 01 '24 03:04 aquapi