perf(req): use `URLSearchParams` for parsing query params

[yusukebe]

This PR changes the implementation of parsing query params using c.req.query(). This change affects performance and file size. In addition, it fixes the problem that can't parse invalid percent strings.

File size

The logic to parse query params like ?name=hono is written by ourown method. The method started with following:

https://github.com/honojs/hono/blob/3a59d865333a2121316fd1d4b2f937a931847022/src/utils/url.ts#L213-L217

This enables them to be parsed fast, but the logic and code are complicated.

With this PR, it uses URLSearchParams instead of that logic. So, the code will decrease, and the application file size will be smaller. The below compares the current release version and this PR. The application is "Hello World" using hono/tiny preset and minified with esbuild:

The result is that this change decreases 642bytes. This seems to be a small change for the non-Hono user, but I think it's a big diff for us.

The important point is that hono should not provide a lot of code for the user that does not use a lot of features. For example, we don't want to take much code to parse a query if the application does not use c.req.query(). So, using URLSearchParams built-in API makes sense.

Performance

Instead, performance will decrease. The following is the result:

The application code:

import { Hono } from 'hono'

const app = new Hono()

app.get('/', (c) => c.text(c.req.query('name') ?? 'no name'))

export default app

At a first grance, you may be impressed that this difference is enormous, but it's a slight, 4% slowdown. I think there's more advantage in using URLSearchParasm for making it short and easy to understand.

Small vs Fast

In this place, you may think "small vs. fast "—which should we choose, small or fast? I think the answer is "both." We have to consider it case by case.

In this parsing query param matter, "small" is better. Being small has some time-saving effects because a small application can be faster in an environment where resources such as Cloudflare Workers are limited. So, in some cases, "small" makes "fast".

Parting invalid percent strings

This PR will fix the problem of paring invalid percent strings. Before this PR, it will throw the error if you want to do c.req.query('q') for the following URL:

http://example.com/?q=%h

In this PR, it can parse the query and you can get a %h.

Conclusion

This PR using URLSearchParams introduce decrease of the performance to parse query params, but reducing the file size and making the code clean is more efficient. BUT, this is just my current thought. We can discuss it.

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 91.29%. Comparing base (c277c75) to head (2bf7b5f).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3565      +/-   ##
==========================================
- Coverage   91.34%   91.29%   -0.05%     
==========================================
  Files         168      168              
  Lines       10700    10641      -59     
  Branches     3181     3081     -100     
==========================================
- Hits         9774     9715      -59     
  Misses        925      925              
  Partials        1        1              

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:

• :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[yusukebe]

Hi @usualoma ! I want to hear your opinion!

[usualoma]

Another benchmark

When we do local benchmarking of query string parsing, I confirm that using URLSearchParams is, on average, several times slower. However, in some exceptional cases, URLSearchParams may be faster (depending on the runtime).

diff --git a/benchmarks/query-param/src/bench.mts b/benchmarks/query-param/src/bench.mts
index e8be60e0..857d24c9 100644
--- a/benchmarks/query-param/src/bench.mts
+++ b/benchmarks/query-param/src/bench.mts
@@ -1,4 +1,5 @@
 import { run, group, bench } from 'mitata'
+import { getQueryStrings } from '../../../src/utils/url'
 import fastQuerystring from './fast-querystring.mts'
 import hono from './hono.mts'
 ;[
@@ -36,6 +37,17 @@ import hono from './hono.mts'
   group(JSON.stringify(data), () => {
     bench('hono', () => hono(url, key))
     bench('fastQuerystring', () => fastQuerystring(url, key))
+    bench('URLSearchParams', () => {
+      const params = new URLSearchParams(getQueryStrings(url))
+      if (key) {
+        return params.get(key)
+      }
+      const obj = {}
+      for (const [k, v] of params) {
+        obj[k] = v
+      }
+      return obj
+    })
   })
 })
 

% npm run bench:node

> bench:node
> tsx ./src/bench.mts

(node:19644) ExperimentalWarning: `--experimental-loader` may be removed in the future; instead use `register()`:
--import 'data:text/javascript,import { register } from "node:module"; import { pathToFileURL } from "node:url"; register("file%3A///Users/taku/src/github.com/honojs/hono/benchmarks/query-param/node_modules/tsx/dist/loader.js", pathToFileURL("./"));'
(Use `node --trace-warnings ...` to show where the warning was created)
cpu: Apple M2 Pro
runtime: node v20.14.0 (arm64-darwin)

benchmark            time (avg)             (min … max)       p75       p99      p995
------------------------------------------------------- -----------------------------
• {"url":"http://example.com/?page=1","key":"page"}
------------------------------------------------------- -----------------------------
hono              55.29 ns/iter   (48.29 ns … 72.54 ns)  56.93 ns  64.82 ns  67.65 ns
fastQuerystring   72.86 ns/iter    (67.3 ns … 83.71 ns)  74.87 ns  82.22 ns  82.56 ns
URLSearchParams   87.86 ns/iter  (81.46 ns … 104.03 ns)  89.55 ns  96.67 ns     98 ns

summary for {"url":"http://example.com/?page=1","key":"page"}
  hono
   1.32x faster than fastQuerystring
   1.59x faster than URLSearchParams

• {"url":"http://example.com/?url=http://example.com&page=1","key":"page"}
------------------------------------------------------- -----------------------------
hono              77.51 ns/iter   (70.67 ns … 92.51 ns)  78.74 ns  87.02 ns  89.45 ns
fastQuerystring  206.31 ns/iter (196.72 ns … 227.85 ns)  207.4 ns 226.08 ns 226.62 ns
URLSearchParams  235.62 ns/iter (227.06 ns … 279.42 ns) 239.84 ns 258.11 ns 264.34 ns

summary for {"url":"http://example.com/?url=http://example.com&page=1","key":"page"}
  hono
   2.66x faster than fastQuerystring
   3.04x faster than URLSearchParams

• {"url":"http://example.com/?page=1"}
------------------------------------------------------- -----------------------------
hono              97.54 ns/iter  (89.05 ns … 113.99 ns) 100.43 ns  111.7 ns 113.39 ns
fastQuerystring   73.91 ns/iter   (70.23 ns … 85.17 ns)  76.53 ns  80.67 ns  81.42 ns
URLSearchParams  138.78 ns/iter (130.23 ns … 155.33 ns) 140.43 ns 152.97 ns  154.7 ns

summary for {"url":"http://example.com/?page=1"}
  fastQuerystring
   1.32x faster than hono
   1.88x faster than URLSearchParams

• {"url":"http://example.com/?url=http://example.com&page=1"}
------------------------------------------------------- -----------------------------
hono             174.13 ns/iter (165.96 ns … 184.61 ns) 175.66 ns 180.56 ns 181.09 ns
fastQuerystring  205.18 ns/iter  (197.78 ns … 223.4 ns) 205.85 ns 219.93 ns 222.64 ns
URLSearchParams  329.48 ns/iter  (314.24 ns … 351.4 ns) 334.73 ns 348.99 ns  351.4 ns

summary for {"url":"http://example.com/?url=http://example.com&page=1"}
  hono
   1.18x faster than fastQuerystring
   1.89x faster than URLSearchParams

• {"url":"http://example.com/?url=http://example.com/very/very/deep/path/to/something&search=very-long-search-string"}
------------------------------------------------------- -----------------------------
hono             214.94 ns/iter  (201.22 ns … 245.8 ns) 220.49 ns 242.58 ns 244.95 ns
fastQuerystring  410.29 ns/iter (402.25 ns … 462.79 ns) 410.93 ns 446.06 ns 462.79 ns
URLSearchParams  556.61 ns/iter (539.25 ns … 591.04 ns) 567.25 ns  587.7 ns 591.04 ns

summary for {"url":"http://example.com/?url=http://example.com/very/very/deep/path/to/something&search=very-long-search-string"}
  hono
   1.91x faster than fastQuerystring
   2.59x faster than URLSearchParams

• {"url":"http://example.com/?search=Hono+is+a+small,+simple,+and+ultrafast+web+framework+for+the+Edge.&page=1"}
------------------------------------------------------- -----------------------------
hono               3.21 µs/iter     (3.18 µs … 3.29 µs)   3.22 µs   3.29 µs   3.29 µs
fastQuerystring    3.31 µs/iter      (3.25 µs … 3.4 µs)   3.39 µs    3.4 µs    3.4 µs
URLSearchParams  723.09 ns/iter (680.95 ns … 783.58 ns) 740.12 ns 783.58 ns 783.58 ns

summary for {"url":"http://example.com/?search=Hono+is+a+small,+simple,+and+ultrafast+web+framework+for+the+Edge.&page=1"}
  URLSearchParams
   4.44x faster than hono
   4.58x faster than fastQuerystring

• {"url":"http://example.com/?a=1&b=2&c=3&d=4&e=5&f=6&g=7&h=8&i=9&j=10"}
------------------------------------------------------- -----------------------------
hono             415.17 ns/iter (400.33 ns … 434.02 ns) 417.63 ns 433.17 ns 434.02 ns
fastQuerystring  395.34 ns/iter (382.32 ns … 426.68 ns) 398.51 ns 411.61 ns 426.68 ns
URLSearchParams  587.83 ns/iter (575.63 ns … 618.53 ns) 594.34 ns 618.53 ns 618.53 ns

summary for {"url":"http://example.com/?a=1&b=2&c=3&d=4&e=5&f=6&g=7&h=8&i=9&j=10"}
  fastQuerystring
   1.05x faster than hono
   1.49x faster than URLSearchParams

% npm run bench:bun

> bench:bun
> bun run ./src/bench.mts

cpu: Apple M2 Pro
runtime: bun 1.1.30 (arm64-darwin)

benchmark            time (avg)             (min … max)       p75       p99      p995
------------------------------------------------------- -----------------------------
• {"url":"http://example.com/?page=1","key":"page"}
------------------------------------------------------- -----------------------------
hono              56.17 ns/iter  (50.86 ns … 144.73 ns)  56.02 ns  93.92 ns  102.5 ns
fastQuerystring   88.13 ns/iter   (80.9 ns … 158.18 ns)  88.23 ns 136.54 ns 144.22 ns
URLSearchParams  300.22 ns/iter (238.77 ns … 722.29 ns)  311.5 ns 655.42 ns 722.29 ns

summary for {"url":"http://example.com/?page=1","key":"page"}
  hono
   1.57x faster than fastQuerystring
   5.34x faster than URLSearchParams

• {"url":"http://example.com/?url=http://example.com&page=1","key":"page"}
------------------------------------------------------- -----------------------------
hono             105.29 ns/iter  (97.54 ns … 239.88 ns) 105.29 ns 147.07 ns 154.31 ns
fastQuerystring  150.95 ns/iter (139.22 ns … 230.01 ns) 152.44 ns 199.83 ns 217.17 ns
URLSearchParams  521.94 ns/iter   (435.99 ns … 1.89 µs)    516 ns   1.11 µs   1.89 µs

summary for {"url":"http://example.com/?url=http://example.com&page=1","key":"page"}
  hono
   1.43x faster than fastQuerystring
   4.96x faster than URLSearchParams

• {"url":"http://example.com/?page=1"}
------------------------------------------------------- -----------------------------
hono              92.67 ns/iter  (85.98 ns … 182.63 ns)  92.57 ns 136.81 ns 140.65 ns
fastQuerystring   85.43 ns/iter  (79.54 ns … 141.13 ns)  86.14 ns 131.65 ns  134.6 ns
URLSearchParams  534.39 ns/iter  (428.79 ns … 699.6 ns) 577.35 ns 690.42 ns  699.6 ns

summary for {"url":"http://example.com/?page=1"}
  fastQuerystring
   1.08x faster than hono
   6.26x faster than URLSearchParams

• {"url":"http://example.com/?url=http://example.com&page=1"}
------------------------------------------------------- -----------------------------
hono             174.34 ns/iter (162.04 ns … 285.48 ns) 175.02 ns 215.94 ns 219.05 ns
fastQuerystring  152.69 ns/iter (140.99 ns … 205.06 ns) 154.36 ns 199.17 ns  200.7 ns
URLSearchParams  832.56 ns/iter   (717.03 ns … 1.38 µs) 863.65 ns   1.38 µs   1.38 µs

summary for {"url":"http://example.com/?url=http://example.com&page=1"}
  fastQuerystring
   1.14x faster than hono
   5.45x faster than URLSearchParams

• {"url":"http://example.com/?url=http://example.com/very/very/deep/path/to/something&search=very-long-search-string"}
------------------------------------------------------- -----------------------------
hono             213.67 ns/iter (200.91 ns … 277.14 ns) 215.71 ns  258.9 ns 259.39 ns
fastQuerystring  236.22 ns/iter (221.84 ns … 293.94 ns) 237.02 ns 281.23 ns 284.47 ns
URLSearchParams    1.05 µs/iter   (967.05 ns … 1.18 µs)   1.07 µs   1.18 µs   1.18 µs

summary for {"url":"http://example.com/?url=http://example.com/very/very/deep/path/to/something&search=very-long-search-string"}
  hono
   1.11x faster than fastQuerystring
   4.89x faster than URLSearchParams

• {"url":"http://example.com/?search=Hono+is+a+small,+simple,+and+ultrafast+web+framework+for+the+Edge.&page=1"}
------------------------------------------------------- -----------------------------
hono             599.99 ns/iter (539.21 ns … 758.78 ns) 612.83 ns 758.78 ns 758.78 ns
fastQuerystring  498.18 ns/iter (479.87 ns … 620.67 ns) 497.73 ns 554.57 ns 620.67 ns
URLSearchParams  947.22 ns/iter   (833.05 ns … 1.11 µs) 998.14 ns   1.11 µs   1.11 µs

summary for {"url":"http://example.com/?search=Hono+is+a+small,+simple,+and+ultrafast+web+framework+for+the+Edge.&page=1"}
  fastQuerystring
   1.2x faster than hono
   1.9x faster than URLSearchParams

• {"url":"http://example.com/?a=1&b=2&c=3&d=4&e=5&f=6&g=7&h=8&i=9&j=10"}
------------------------------------------------------- -----------------------------
hono             318.32 ns/iter (302.19 ns … 389.28 ns) 320.43 ns 369.13 ns 389.28 ns
fastQuerystring  219.94 ns/iter (206.72 ns … 288.48 ns) 224.32 ns 271.62 ns 274.78 ns
URLSearchParams    2.09 µs/iter     (1.97 µs … 2.24 µs)   2.14 µs   2.24 µs   2.24 µs

summary for {"url":"http://example.com/?a=1&b=2&c=3&d=4&e=5&f=6&g=7&h=8&i=9&j=10"}
  fastQuerystring
   1.45x faster than hono
   9.5x faster than URLSearchParams

[usualoma]

Is it an app that works in a persistent environment?

From the benchmark mentioned above, we can see that using URLSearchParams takes several times longer. However, in that case, it is only a few hundred nanoseconds. If this change reduces the code by 600 bytes, it will reduce the time it takes for the runtime to parse the code. This will vary depending on the environment, but I think there is a good chance that it will be a reduction of more than a few hundred nanoseconds. Therefore, if the code is parsed each time it is called (which is likely to be the case in some cloud runtime environments), I think it is more appropriate to use the code that uses URLSearchParams.

[usualoma]

As you mentioned in your comment, there is a trade-off between “small” and “fast” (and “small causes fast " complicates the problem), so I think it's a difficult point.

When using c.req. query()`, there is often corresponding backend processing (accessing the DB, proxying to the outside), so I don't think optimizing a few hundred nanoseconds here is very meaningful in a real-world production app.

The results of specific benchmarks will be worse, so we have to think about that.

As an alternative, I think there is also the option of “using URLSearchParams when using the hono/tiny preset.” The maintenance cost will be a little higher, though.

[usualoma]

@yusukebe It's difficult to evaluate the trade-offs, but overall, I agree with the changes in this pull request!

[nakasyou]

How about that code?:

import { Hono } from 'hono/hono-base'
import { FastURLSearchParams } from 'hono/...'

const app = new Hono({
  URLSearchParams: URLSearchParams
})

and implementing using URLSearchParams when using the hono/tiny preset like @usualoma said is easier.

[yusukebe]

@usualoma Thank you for the comments!

Is it an app that works in a persistent environment?

As you know, Hono has to support both "one-time" and "persistent" apps.

For example, each popular environment has a different life cycle:

• Node.js/Bun on Fly.io - Once it starts, it's up and running forever.
• Cloudflare Workers - Once it starts, it stays alive for a while, but dies soon after.
• Fastly Compute - Once it starts, it dies immediately.

The following benchmarks for Cloudflare Workers are of interest. The smaller file size is faster.

https://github.com/TigersWay/cloudflare-playground

In that benchmark, the application receives one access every 60 seconds, which is not a lot. My guess is that once the access is handled, the application shuts down. And then it starts anew for the next access. In a real-world application, it will stay running and Hono will perform better because it handles more accesses than once every 60 seconds.

However, many people make judgments based on benchmark results, so it is also important to be fast, even "one-time".

When using c.req. query()`, there is often corresponding backend processing (accessing the DB, proxying to the outside), so I don't think optimizing a few hundred nanoseconds here is very meaningful in a real-world production app.

On the other hand, there is this way of thinking about it.

For example, if you use a Hono app as a proxy server, as in the Issue #3518 , the code is short. However, even though you are not using c.req.query(), the code to parse queries is included. I would like to avoid having code for other functions which is not used in your apps.

As an alternative, I think there is also the option of “using URLSearchParams when using the hono/tiny preset.” The maintenance cost will be a little higher, though.

This is interesting! It is truly tiny. We can consider it together with @nakasyou's opinion, but it would certainly complicate the code.

---

Either way, I believe this PR is effective for now. But there is no need to rush!

[yusukebe]

In this case, the important point is that URLSearchParams is a runtime API. So, if the performance of the runtime implementation improves, the performance of Hono will also improve. In fact, the runtime-sides are working hard to do it.

[rafaell-lycan]

@yusukebe from what I see it makes sense to ship the fastest (default) with Hono preset and change it to URLSearchParams when using a different preset such as hono/tiny as mentioned by @usualoma

Comparing Hono with Express is unfair, but one of the reasons why we've decided to use it instead of Fastify was its performance. I'm not asking to change its DNA or roots, but users might need to be aware of the trade-off.

[yusukebe]

Hi @rafaell-lycan

Thank you for the suggestion and your opinion!

Comparing Hono with Express is unfair, but one of the reasons why we've decided to use it instead of Fastify was its performance. I'm not asking to change its DNA or roots, but users might need to be aware of the trade-off.

You may saying right thing for us. We have to focus on speed.

The only way to achieve both is by taking the idea to make hono/tiny using URLSearchParams. However, that would complicate the code a bit. I think putting this issue aside for now is better.

[bompi88]

One consideration: it could be useful to provide an option for overriding the parseQuery method. For example, this would allow for the use of alternative parsers like qs, if needed. A performant and sensible default would, of course, be ideal.

I added an issue https://github.com/honojs/hono/issues/3667 right before I saw this PR.

[yusukebe]

Hi @bompi88

By the way, the qs is slower than the current Hono's query parsing method: https://github.com/honojs/hono/pull/3674

Do you have any motivation to use the qs?

[EdamAme-x]

I like the idea of having URLSearchParams only on hono/tiny.

[bompi88]

@yusukebe My motivation is that I want to migrate my current APIs to Hono :) I'm using a special query "syntax" which is supported by qs. For example I can do ?amount[lte]=300 which resolves to:

{
  amount: {
    lte: 300
  }
}

If there is another way to do this, I'm happy to ditch qs.

[yusukebe]

@bompi88

I'm using a special query "syntax" which is supported by qs.

I see! But it's not good that the API will be changed depending on the implementation, such as Hono's current logic, URLSearchParams, or qs. If you want the feature, we should add it as a Hono feature, though it introduces a breaking change. Can you create an issue for the feature request?

[yusukebe]

@bompi88

Sorry, I missed #3667. Let's discuss in it.

[github-actions[bot]]

Bundle size check

	main (15a83b1)	#3565 (61d50b9)	+/-
Bundle Size (B)	18,904B	18,395B	-509B
Bundle Size (KB)	18.46K	17.96K	-0.5K

Compiler Diagnostics

	main (15a83b1)	#3565 (61d50b9)	+/-
Files	261	261	0
Lines	116,434	116,370	-64
Identifiers	114,428	114,332	-96
Symbols	303,852	303,858	6
Types	214,828	214,836	8
Instantiations	3,091,594	3,091,648	54
Memory used	445,293K	447,522K	2,229K
I/O read	0.03s	0.03s	0s
I/O write	0s	0s	0s
Parse time	0.66s	0.71s	0.05s
Bind time	0.28s	0.28s	0s
Check time	5.84s	6.05s	0.21s
Emit time	0s	0s	0s
Total time	6.78s	7.04s	0.26s

---

Reported by octocov

[github-actions[bot]]

Bundle size check

	main (c277c75)	#3565 (baf1269)	+/-
Bundle Size (B)	18,933B	18,424B	-509B
Bundle Size (KB)	18.49K	17.99K	-0.5K

Compiler Diagnostics

	main (c277c75)	#3565 (baf1269)	+/-
Files	261	261	0
Lines	116,438	116,374	-64
Identifiers	114,433	114,337	-96
Symbols	303,855	303,861	6
Types	214,832	214,840	8
Instantiations	3,091,594	3,091,648	54
Memory used	444,751K	446,439K	1,688K
I/O read	0.02s	0.03s	0.01s
I/O write	0s	0s	0s
Parse time	0.67s	0.71s	0.04s
Bind time	0.28s	0.27s	-0.01s
Check time	5.77s	5.82s	0.05s
Emit time	0s	0s	0s
Total time	6.72s	6.8s	0.08s

---

Reported by octocov