yorkie-js-sdk icon indicating copy to clipboard operation
yorkie-js-sdk copied to clipboard

Add removed benchmark test

Open parkeunae opened this issue 1 year ago • 3 comments

What this PR does / why we need it?

Add benchmark Document/equals_test and Document/nested_update_test to updated benchmark scripts

Any background context you want to provide?

What are the relevant tickets?

Fixes #

Checklist

  • [ ] Added relevant tests or not required
  • [ ] Didn't break anything

parkeunae avatar Jul 28 '22 03:07 parkeunae

@parkeunae Thanks for your contribution.

The benchmark test seems to be using a lot of memory. How about trying to find the cause? https://github.com/yorkie-team/yorkie-js-sdk/runs/7552659343?check_suite_focus=true#step:10:13

> [email protected] benchmark
> ts-node-script ./test/bench

Document#constructor x 883 ops/sec ±10.[6](https://github.com/yorkie-team/yorkie-js-sdk/runs/7552659343?check_suite_focus=true#step:10:7)0% ([7](https://github.com/yorkie-team/yorkie-js-sdk/runs/7552659343?check_suite_focus=true#step:10:8)[8](https://github.com/yorkie-team/yorkie-js-sdk/runs/7552659343?check_suite_focus=true#step:10:9) runs sampled)
Document#equals x 166 ops/sec ±43.85% (38 runs sampled)
Document#nested update x 17.87 ops/sec ±5[9](https://github.com/yorkie-team/yorkie-js-sdk/runs/7552659343?check_suite_focus=true#step:10:10).17% (12 runs sampled)

<--- Last few GCs --->

[3155:0x66f46d0]    31134 ms: Mark-sweep (reduce) 2047.0 (2083.0) -> 2045.8 (2083.2) MB, 1400.4 / 0.0 ms  (+ 70.8 ms in 25 steps since start of marking, biggest step [10](https://github.com/yorkie-team/yorkie-js-sdk/runs/7552659343?check_suite_focus=true#step:10:11).6 ms, walltime since start of marking 1508 ms) (average mu = 0.258, current mu = 0.199[3155:0x66f46d0]    32959 ms: Mark-sweep (reduce) 2046.9 (2083.2) -> 2045.7 (2083.5) MB, 1823.0 / 0.0 ms  (average mu = 0.133, current mu = 0.001) allocation failure scavenge might not succeed


<--- JS stacktrace --->

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0xb0a860 node::Abort() [node]
 2: 0xa1c193 node::FatalError(char const*, char const*) [node]
 3: 0xcf9a6e v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xcf9de7 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xeb1685  [node]
 6: 0xeb2166  [node]
 7: 0xec068e  [node]
 8: 0xec10d0 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
 9: 0xec404e v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
10: 0xe8558a v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
[11](https://github.com/yorkie-team/yorkie-js-sdk/runs/7552659343?check_suite_focus=true#step:10:12): 0x11fe2d6 v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
[12](https://github.com/yorkie-team/yorkie-js-sdk/runs/7552659343?check_suite_focus=true#step:10:13): 0x[15](https://github.com/yorkie-team/yorkie-js-sdk/runs/7552659343?check_suite_focus=true#step:10:16)f2d39  [node]
Aborted (core dumped)

hackerwins avatar Jul 28 '22 03:07 hackerwins

Codecov Report

Merging #358 (458010a) into main (09e921f) will not change coverage. The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #358   +/-   ##
=======================================
  Coverage   89.05%   89.05%           
=======================================
  Files          73       73           
  Lines        5491     5491           
  Branches      544      544           
=======================================
  Hits         4890     4890           
  Misses        410      410           
  Partials      191      191           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

codecov[bot] avatar Jul 28 '22 08:07 codecov[bot]

@hackerwins I fixed it so that it no longer throws javascript heap out of memory error. I guessed the probloem is that it adds all tests to benchmark suite and runs them in turn. So, I updated it as follows.

  1. Delete the For loop that runs 100 times as the benchmark automatically runs the tests.
  2. Fix toSortedJson to toJson.
  3. Set benchmark options to run asynchronously.

But it still takes a long time. It would be okay to create new Suite instance for each test and run it immediately. Could I try like this? I'd like to confirm this in advance. (The test results will not be in order)

runners.forEach(({ name, run }) => {
  new Benchmark.Suite()
    .add(name, run)
    .on('cycle', (event: Benchmark.Event) => {
      console.log(String(event.target));
    })
    .run({ async: true });
});

parkeunae avatar Jul 28 '22 17:07 parkeunae