'while' loop consumes up to 2 GB of RAM

[czerwinskilukasz1]

After iterating 30-60 000 times when running the AskScript program (below) node process consumed up to 2GB of RAM.

let a = 1
while(a>0) {
  if ((a/1000):equals(floor(a/1000))) {
    log(a)
  }
  a=a+1
}

[mhagmajer]

@czerwinskilukasz1 thanks Łukasz, this is a great report!

[mhagmajer]

@czerwinskilukasz1 how did you check how much memory does this program consume?

[czerwinskilukasz1]

By watching Activity Monitor. It was enough for the first estimate. Did you get different results yourself?

[mhagmajer]

@czerwinskilukasz1 not yet, I'm trying to figure out how we can test it

[czerwinskilukasz1]

@mhagmajer How about /usr/bin/time -l (and time -v on Linux)?

Lukaszs-MacBook-Pro:askql milimetr$ /usr/bin/time -l npm run cli

> [email protected] cli /Users/milimetr/Desktop/askql/askql
> node dist/cli.js

🦄 .editor
// Entering editor mode (^D to finish, ^C to cancel)
let a = 1
while(a>0) {
  if ((a/1000):equals(floor(a/1000))) {
    log(a)
  }
  a=a+1
}


1000
2000
3000
4000
5000
6000
7000
8000
9000
10000
11000
12000
13000
14000
15000
16000
17000
18000
19000
20000
21000
22000
23000
24000
25000
26000
27000
28000
29000
30000
31000
32000
33000
34000
35000
36000
37000
38000
39000
40000
41000
42000
43000
44000
45000
46000
47000
48000
49000
50000
51000
52000
53000
54000
55000
56000
57000
58000
59000
60000
61000
62000
63000
64000
RangeError: Value undefined out of range for undefined options property undefined
    at Map.set (<anonymous>)
    at AsyncHook.init (domain.js:68:15)
    at PromiseWrap.emitInitNative (internal/async_hooks.js:127:43)
    at Promise.then (<anonymous>)
    at step (/Users/milimetr/Desktop/askql/askql/dist/askvm/lib/run.js:7:91)
    at /Users/milimetr/Desktop/askql/askql/dist/askvm/lib/run.js:8:9
    at new Promise (<anonymous>)
    at __awaiter (/Users/milimetr/Desktop/askql/askql/dist/askvm/lib/run.js:4:12)
    at Object.run (/Users/milimetr/Desktop/askql/askql/dist/askvm/lib/run.js:29:12)
    at /Users/milimetr/Desktop/askql/askql/dist/askvm/resources/core/call.js:29:84
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] cli: `node dist/cli.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] cli script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/milimetr/.npm/_logs/2020-06-27T12_29_08_323Z-debug.log
      145.37 real       133.90 user         3.67 sys
1998094336  maximum resident set size
         0  average shared memory size
         0  average unshared data size
         0  average unshared stack size
    697857  page reclaims
         0  page faults
         0  swaps
         0  block input operations
        55  block output operations
         0  messages sent
         0  messages received
         1  signals received
        10  voluntary context switches
    199411  involuntary context switches
Lukaszs-MacBook-Pro:askql milimetr$ 

[czerwinskilukasz1]

We could plug time to npm run ask <.ask filename> as soon as issue https://github.com/xFAANG/askql/issues/248 is solved.

[mhagmajer]

A lot of this is the cost of being asynchronous language. Let's see how much we can optimize it. That program should use constant memory

[mhagmajer]

@czerwinskilukasz1 this can be solved with tail recursion

[mhagmajer]

Not all programming languages require tail call elimination. However, in functional programming languages, tail call elimination is often guaranteed by the language standard, allowing tail recursion to use a similar amount of memory as an equivalent loop. The special case of tail recursive calls, when a function calls itself, may be more amenable to call elimination than general tail calls. When the language semantics do not explicitly support general tail calls, a compiler can often still optimize sibling calls, or tail calls to functions which take and return the same types as the caller.[3]

https://en.wikipedia.org/wiki/Tail_call

[mhagmajer]

Seems like while will only be able to return the final value anyway @czerwinskilukasz1, https://github.com/xFAANG/askql/issues/239

[cpelican]

Hi @czerwinskilukasz1 , I tried to reproduce this issue with the program bit you showed, but I only got as what looks to me as an other error. Plus the program didn't logged anything.

/usr/bin/time -l npm run cli

> [email protected] cli /Users/carolinepellet/Documents/exos/askql
> node dist/cli.js

Debugger listening on ws://127.0.0.1:51296/a140e1af-2610-4b61-9b7d-cf1234c70ce2
For help, see: https://nodejs.org/en/docs/inspector
Debugger attached.
Welcome to AskQL CLI!

For multi-line mode please type:
     .editor
🦄 .editor
// Entering editor mode (^D to finish, ^C to cancel)
let a = 1
while(a>0) {
  if ((a/1000):equals(floor(a/1000))) {
    log(a)
  }
  a=a+1
}

Uncaught Error: Over steps limit!
    at Object.<anonymous> (/Users/carolinepellet/Documents/exos/askql/dist/askvm/lib/run.js:34:19)
    at Generator.next (<anonymous>)
    at /Users/carolinepellet/Documents/exos/askql/dist/askvm/lib/run.js:8:71
    at new Promise (<anonymous>)
    at __awaiter (/Users/carolinepellet/Documents/exos/askql/dist/askvm/lib/run.js:4:12)
    at Object.run (/Users/carolinepellet/Documents/exos/askql/dist/askvm/lib/run.js:28:12)

Only if I did (a/100):equals(floor(a/100)) I was able to have 2 logs but still the same error. Is there something else to do to reproduce this bug? Or did I understand something incorrectly? Thanks!

[czerwinskilukasz1]

@cpelican , good catch. After I submitted this error, mhagmajer reduced the allowed operation limit to just 10 000 operations (which is roughly around 600 iterations of a while loop, if I recall correctly), so no longer is it reproducible but if you changed the limit in https://github.com/xFAANG/askql/blob/master/src/askvm/index.ts#L20 to let's say 10 000 000, you should be able to reproduce.