minify.compress may cause memory overflow?

[noyobo]

Describe the bug

Today I encountered a problem. I suddenly found that swc compression code was very slow. After investigation, I found that it was caused by the following code.

When minify.compress is enabled, swc will merge "a" + "b" into "ab", but if a is a variable, it will be repeated many times, which will cause it to be very slow. And after exceeding a certain threshold, memory overflows.

For example, in my example, the variable is in the current scope and does not need to be merged. It seems that this compression cooperation operation can be skipped?

// build.js
import {transformSync} from "@swc/core";

function genCode(len) {
  return `
    export function add(a) {
      return ${'a + "hello swc, minify" + '.repeat(len)} 'c'
    }
  `;
}

function minifyLen(len, fn) {
  try {
    const start = Date.now();
    const code = fn(len);
    transformSync(code, {
      jsc: {
        parser: {
          syntax: "ecmascript",
        },
        transform: {},
        minify: {
          mangle: true,
          compress: true,
        },
      },
      minify: true,
    });
    console.log(fn.name, len, 'cost:', Date.now() - start, 'ms');
  } catch (e) {
    console.log(fn.name, len, 'error:', e);
  }
}

minifyLen(100, genCode)
minifyLen(1000, genCode)
minifyLen(2000, genCode)
minifyLen(3000, genCode)

case1: ${'a + "hello swc, minify" + '.repeat(len)} 'c'

genCode 100 cost: 113 ms
genCode 1000 cost: 222 ms
genCode 2000 cost: 1019 ms
[1]    57128 segmentation fault  node ./build.js

case2: ${'"a" + "hello swc, minify" + '.repeat(len)} 'c'

genCode2 100 cost: 94 ms
genCode2 1000 cost: 6 ms
genCode2 2000 cost: 18 ms
[1]    57423 segmentation fault  node ./build2.js

case3: ${'a + b + '.repeat(len)} c;

genCode3 100 cost: 10 ms
genCode3 1000 cost: 148 ms
genCode3 2000 cost: 638 ms
[1]    57461 segmentation fault  node ./build3.js

Input code

Config

Playground link (or link to the minimal reproduction)

https://github.com/noyobo/swc-issues/tree/swc-minify

SWC Info output

    Operating System:
        Platform: darwin
        Arch: x64
        Machine Type: x86_64
        Version: Darwin Kernel Version 23.1.0: Mon Oct  9 21:27:27 PDT 2023; root:xnu-10002.41.9~6/RELEASE_X86_64
        CPU: (12 cores)
            Models: Intel(R) Core(TM) i7-8750H CPU @ 2.20GHz

    Binaries:
        Node: 20.12.0
        npm: 10.5.0
        Yarn: 1.22.22
        pnpm: 9.4.0

    Relevant Packages:
        @swc/core: 1.11.10
        @swc/helpers: N/A
        @swc/types: N/A


    SWC Config:
        output: N/A
        .swcrc path: N/A

    Next.js info:
        output: N/A

Expected behavior

Compress code fast

Actual behavior

No response

Version

1.11.10

Additional context

No response

[kdy1]

I didn't think about such input while implementing, so I'm sure there's a time complexity bug in string concat logic.

[CodeMan62]

Let me investigate and try to fix the bug in string concat

[CodeMan62]

@kdy1 can you give me a bit more info like where is the bug actually so it can be easy to solve it

[kdy1]

@CodeMan62 I think one method of https://github.com/swc-project/swc/blob/164cbaa2c02a3216096fa7b969fcfa6575954892/crates/swc_ecma_minifier/src/compress/pure/strings.rs or https://github.com/swc-project/swc/blob/164cbaa2c02a3216096fa7b969fcfa6575954892/crates/swc_ecma_minifier/src/compress/optimize/strings.rs is the cause but not sure which method it is.

[CodeMan62]

@CodeMan62 I think one method of swc-project/swc@164cbaa/crates/swc_ecma_minifier/src/compress/pure/strings.rs or swc-project/swc@164cbaa/crates/swc_ecma_minifier/src/compress/optimize/strings.rs is the cause but not sure which method it is.

I will figure it out and fix 👍

[CodeMan62]

fixed in #10430

[CodeMan62]

@kdy1 I think you should close this now

[kdy1]

No it's not fixed

[noyobo]

After the optimization of https://github.com/swc-project/swc/commit/1434571477f5f8576a268a2bd32631eb9ce77229, run under the 1.12.3 time-consuming has been improved.

➜ swc-issues $ node ./build.js
genCode 100 cost: 1 ms
genCode 1000 cost: 3 ms
genCode 2000 cost: 7 ms
[1]    18450 segmentation fault  node ./build.js
➜ swc-issues $ node ./build2.js
genCode2 100 cost: 1 ms
genCode2 1000 cost: 4 ms
genCode2 2000 cost: 12 ms
genCode2 3000 cost: 24 ms
➜ swc-issues $ node ./build3.js
genCode3 100 cost: 36 ms
genCode3 1000 cost: 3 ms
genCode3 2000 cost: 6 ms
[1]    19336 segmentation fault  node ./build3.js

A 'segmentation fault' is only caused when the string and variable concat are present

[noyobo]

If increase the repeat length of case2 https://github.com/noyobo/swc-issues/blob/b6369b10267cb1afd4701eeacf3ccbe83cd42728/build2.js#L40, will also cause segmentation fault

➜ swc-issues $ node ./build2.js
genCode2 100 cost: 35 ms
genCode2 1000 cost: 4 ms
genCode2 2000 cost: 13 ms
genCode2 3000 cost: 23 ms
[1]    20551 segmentation fault  node ./build2.js

[noyobo]

Even more amazing was that when I tried to run all the tests through the Makefile at once, nothing went wrong. I don't know what the difference is 🤯🤯🤯🤯

https://github.com/noyobo/swc-issues/blob/4fea034dc4ea0809e73da92cf7b730315e973af4/Makefile#L14

➜ swc-issues $ node ./build2.js
genCode2 100 cost: 24 ms
genCode2 1000 cost: 4 ms
genCode2 2000 cost: 11 ms
genCode2 3000 cost: 23 ms
[1]    23971 segmentation fault  node ./build2.js


➜ swc-issues $ make test-2
npx swc --version

@swc/cli: 0.4.0
@swc/core: 1.12.3

node ./build2.js
genCode2 100 cost: 1 ms
genCode2 1000 cost: 4 ms
genCode2 2000 cost: 15 ms
genCode2 3000 cost: 35 ms
genCode2 4000 cost: 52 ms
genCode2 5000 cost: 66 ms
genCode2 10000 cost: 217 ms